Batch - Specify image-family

Google Cloud Compute Infrastructure
1

Is there any way to specify the image family for a job in the InstancePolicy alongside machineType?

Am I correct in understanding that the only way to specify a custom image is to create an Instance Template?

Would be really helpful to be able to specify the instance family alongside a machineType in a submit call!

2

Hello jacksonwb,

Yes, you are correct that the only way to specify a custom image in a batch job is to use an instance template. You cannot specify an image family in an InstancePolicy alongside the machineType. However, you can specify the image family when you create an instance template, and then use that instance template in your job.

Here’s an example of how you can specify an image family when creating an instance template:

gcloud compute instance-templates create my-template \
--image-family ubuntu-2004-lts \
--image-project ubuntu-os-cloud \
--machine-type n1-standard-1

Then, you can use this instance template in your job like this:

gcloud beta dataflow jobs run my-job \
--gcs-location gs://dataflow-templates/latest/GCS_Text_to_BigQuery \
--parameters \
inputFile=gs://my-bucket/input.txt,outputTableSpec=my-project:my-dataset.my_table \
--instance-template my-template
3

Your understanding is correct in that an instance template is the only way to specify a custom image today. The InstancePolicy boot_disk field within the v1alpha API allows for the image family to be specified, and this capability will soon be introduced to the v1 API.

1 Like
4

Thanks Shamel, makes sense. Any indication on when this might make its way into the v1 API?

Also is this listed on the v1alpha API public docs? I don’t see that field in the InstancePolicy REST object:

https://cloud.google.com/batch/docs/reference/rest/v1alpha/projects.locations.jobs#instancepolicy

5

We had some delay in updating the API reference doc. The bootDisk field is available in v1 now: https://cloud.google.com/batch/docs/reference/rest/v1/projects.locations.jobs#instancepolicy

1 Like