I want to run Job A with 10,000 tasks and Job B with 10,000 tasks. The tasks within each Job is fully parallel but I would like Job B to start after all tasks in Job A fully completes, is there a way to do that? The closest I can find is something called dependencies https://cloud.google.com/batch/docs/reference/rest/v1alpha/projects.locations.jobs in the v1alpha REST api but it says “Not yet implemented”
1 Like
@Wen_gcp @wenyhu @bolianyin @Shamel Dear staff members, could you help me when you see this?
1 Like
The best way to achieve such dependency would to leverage Cloud Workflows to submit Job A and once that job completes, have the next step in the workflow submit Job B. Here is an example of using the Workflow connector for Batch.
Batch does not support job dependencies natively in the API yet, but it is something we are looking into.
1 Like
Greetings @gradientopt ,
You can Run a Batch Job using Workflows. In utilizing Workflows, it allows you to execute the job based on the order that you defined.
You can also refer to this GCC Post in scheduling a batch job using Cloud Scheduler to trigger a cloud workflow. This creates a new Batch job based on the existing job definition.
Let me know if it helped, thanks!
1 Like