Datastream unsupported events

Google Cloud Data Analytics
1

So I have been using MySQL to GCS Datastream pipeline. When I was inspecting the job , I found out that as shown in the pic

There are 105 events that were unsupported and were not loaded within last 7 days.

RC1_0-1669195945979.png
RC1_0-1669195945979.png1370Ă—198 9.28 KB

In the help section , I found out the following note

The number of events DataStream was not able to process in the last 7 days. 
These events are not loaded to the destination, 
and you are not charged for them. 
More details about unprocessed events can be found in the logs.

When I checked out logs I did not find any logs about unprocessed events. Mostly I found out logs like CDC fetch failed: An unexpected error occurred. Contact Google Support… (which is mostly a recoverable error). There was almost no info about the unprocessed event. Any way or method via which I can get details of these unprocessed events or any suggestion on how these events can be avoided ?

cc: @etaim

1 Like
2

As this would be hard to reproduce, I just want to share with you some GCP documentations and other resources [1,2,3,4,5] that could help you with audit logging and system logging for MySQL to GCS Datastream.

[1] https://cloud.google.com/datastream/docs/audit-logging
[2] https://cloud.google.com/logging/docs/audit#system-event
[3] https://cloud.google.com/datastream/docs/faq
[4] https://cloud.google.com/sql/docs/mysql/replication#bin-log-impact
[5] https://dev.mysql.com/doc/refman/5.6/en/binary-log.html

1 Like
3

@RC1 What do you mean by “which is mostly a recoverable error”?
I’m having the error: “CDC fetch failed: An unexpected error occurred. Contact Google Support..” when I started the stream. Do you know why this is happening?

2 Likes
4

I’m having a similar error “Fetch failed: An unexpected error occurred. Contact Google Support..” and validation tests all passed ok. Has anyone found a solution?