Python DatastreamClient.create_stream not working

Google Cloud Serverless Applications
1

I am trying to create a datastream stream using the python client library:

stream = datastream_v1.Stream(
            display_name="TEST_STREAM",
            source_config=datastream_v1.SourceConfig(
                source_connection_profile="projects/{PROJECT_NUMBER}/locations/europe-west9/connectionProfiles/{CONNECTION_PROFILE_NAME}",
                mysql_source_config=datastream_v1.MysqlSourceConfig(
                    include_objects=datastream_v1.MysqlRdbms(
                        mysql_databases=[
                            datastream_v1.MysqlDatabase(
                                database="DB1",
                                mysql_tables=[
                                    datastream_v1.MysqlTable(table="TABLE1")
                                ],
                            )
                        ]
                    )
                ),
            ),
            destination_config=datastream_v1.DestinationConfig(
                destination_connection_profile="projects/{PROJECT_NUMBER}/locations/europe-west9/connectionProfiles/{CONNECTION_PROFILE_NAME}",
                gcs_destination_config=datastream_v1.GcsDestinationConfig(
                    file_rotation_mb=50,
                    file_rotation_interval=Duration(seconds=60),
                    json_file_format=datastream_v1.JsonFileFormat(
                        schema_file_format=datastream_v1.JsonFileFormat.SchemaFileFormat.NO_SCHEMA_FILE,
                        compression=datastream_v1.JsonFileFormat.JsonCompression.GZIP,
                    ),
                ),
            ),
            backfill_none=datastream_v1.Stream.BackfillNoneStrategy(),
        )

        request = datastream_v1.CreateStreamRequest(
            parent=f"projects/{PROJECT_NAME}/locations/europe-west9",
            stream_id="TEST_STREAM",
            stream=stream,
            validate_only=False,
        )

        response = self._client.create_stream(request=request)
        print(response.result())

But when it runs it creates a stream without a name and in an “unspecified” state.

Stev0198_0-1724948848470.png

The response result printed above ends with:
File “/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/google/api_core/future/polling.py”, line 261, in result
raise self._exception
google.api_core.exceptions.FailedPrecondition: 400 We failed to execute the command. 9: We failed to execute the command.

Does anyone have any thoughts/advice on how I can approach this problem?

2

Hi @Stev0198 ,

The issue you’re encountering, where the Datastream stream is created with an “unspecified” state and a FailedPrecondition error, often indicates that there is a problem with the configuration of the stream or the request. Here are a few things to check that may help resolve this issue:

  • Follow Documentation Guidelines: Ensure that all steps and configurations detailed in this documentation are correctly implemented. This includes proper formatting of resource names, valid field values, and adherence to required parameter constraints.
  • Enable and check Cloud Logging or Error Reporting: Utilize logging mechanisms to capture detailed information about the request and response, which can assist in troubleshooting. To do this you can either check the logs in Cloud Logging or errors in Error Reporting.
  • Refer to Troubleshooting Guides: Review this troubleshooting guide for detailed steps that you can follow to help check if there’s any error associated with the stream, like if there’s an issue with the data source or its destination.
  • Contact Google Cloud Support: If the issue persists despite following the above options, consider reaching out to Google Cloud Support. Provide them with detailed information about your configuration, request parameters, and error messages for more targeted assistance.

I hope the above information is helpful.