Batch: some jobs fail with no logs or status

Google Cloud Compute Infrastructure
1

Hi all,

I have a Batch that normally succeeds but sometimes it fails.

When it fails there are no logs and also no statusEvent useful info…

It seems not a quota issue.

Here the relevant lines when cgloud batch jobs describe:

logsPolicy:
  destination: CLOUD_LOGGING
name: projects/{PROJECT_ID}/locations/europe-west6/jobs/{JOB_ID}
state: FAILED
statusEvents:
  - description: Job state is set from QUEUED to FAILED for job projects/{PROJECT_ID}/locations/europe-west6/jobs/{JOB_ID}.
    eventTime: '2023-02-15T07:24:07.877377371Z'
    type: STATUS_CHANGED
taskGroups:
...

As you can see, from QUEUED to FAILED with no explanation or logs…

Any help please? (if I restart the same job it works)

1 Like
2

Hi @Jorge_blom ,

Could you please attach the full log of the job that is having the issue.

Regards,
Marc

3

Hi @marcanthonyb there is no logs for this job at all… :disappointed_face:

4

Hi @Jorge_blom

In addition to batch_task_logs as log_name by default if you are directed from the Batch job detail page in the console, could you select batch_agent_logs as well to see if you can find anything?

If not, could you provide us the full output from “gcloud batch jobs describe” for us to look further?

5

In general lack of output from any failures that happen by the batch agent runnables has been a frequent issue for me also. (Like no errors on mispelling a bucket to mount).

2 Likes