Currently, we are sending aggregated data from the AWS environment to GCP’s BigQuery on an hourly or daily basis, but in the past 1-2 months, we have frequently encountered internal Google API errors in the job within AWS’s Step Functions that performs this process.
{
“errorMessage”: “googleapi: Error 500: An internal error occurred and the request could not be completed. This is usually caused by a transient issue. Retrying the job with back-off as described in the BigQuery SLA should solve the problem: https://cloud.google.com/bigquery/sla. If the error continues to occur please contact support at https://cloud.google.com/support. Error: 1001262, internalError”,
“errorType”: “Error”
}
The data transfer from AWS to GCP itself completes without any problems, but the job execution itself is not completed, and we are addressing the situation by retrying on the AWS side. If you have any insights regarding the causes or possible workarounds for the above issue, your guidance would be greatly appreciated.