Reduce GPU Allocation Time in GKE Autopilot Cluster

Google Cloud Compute Infrastructure
1

Hi all,
I’m currently experiencing a delay of approximately 2 to 3 minutes in GPU allocation for my GKE Autopilot cluster. Here are the details:

  • Region: asia-south1
  • Cluster Type: Autopilot
  • Kubernetes Version: 1.29.6-gke.1254000

deepaksingh_0-1724763888692.png
deepaksingh_0-1724763888692.png900×356 20.8 KB

deepaksingh_1-1724763961646.png
deepaksingh_1-1724763961646.png836×196 16.3 KB

Is there anything I can do to decrease the time it takes for GPUs to be allocated? Are there any specific configurations or optimizations that can help speed up this process in an Autopilot cluster?

Any guidance or recommendations would be greatly appreciated!

2

@knet your insights on this issue would be especially valuable. Any guidance or recommendations you could provide would be greatly appreciated!

3

I’m sorry, I don’t work on GKE Autopilot. Though this doesn’t sound too out of the ordinary.

If you’re looking for fast startup times, Cloud Run just launched a preview of GPU support! It’s signup-only at the moment, we’re steadily adding people. https://cloud.google.com/run/docs/configuring/services/gpu

1 Like