I’ve noticed some strange behavior recently when attempting to insert VMs that I’m trying to understand. Google Cloud seems to be automatically adding quotas with a value of 0. Here is the sequence of operations:
Check existing quotas. Find a machine type family and region combination that doesn’t have an existing quota (I.e. N4 vm_family and us-west4 region).
Attempt to insert a new VM via the API using that machine type family and region (I.e. n4-highcpu-4 and zone us-west4-a).
Inserting the VM fails with error Quota ‘CPUS_PER_VM_FAMILY’ exceeded. Limit: 0.0 in region us-west4.
Check quotas again. There is now a new quota for region us-west4 and vm_family N4 with a value of 0.
I’ve tried several different regions and several different machine type families and hit this same issue many times. I can’t be sure I’ve hit it every time, but at least most of the time.
I don’t actually care about getting this working for my own account. My problem is that I maintain an application that provisions VMs into Google Cloud which sometimes includes automatically picking a machine type. I don’t want my application to fail because it chose a machine type and Google Cloud auto-created a 0 quota for it. I understand that we can tell the customer to go through the process of increasing the quota but I would prefer to avoid that.
The documentation that I have read seems to indicate that Google Cloud doesn’t automatically create quotas so either this is a bug in Google Cloud, the documentation is wrong, or what I am seeing isn’t actually automatically created quotas.
Thanks for responding. My step 1 was done manually in the Google Cloud console. I just started looking at quotas the last few days and still haven’t been able to correlate the API quota data with the console quota data reliably so I’m relying on the console for now. Here are the quotas that shows up after the attempt to insert the VM:
and
I may have been wrong about the documentation. I thought I saw something in the actual Google documentation, but I can’t find it now. The only thing I can find now is a Google Search AI response that starts with
Google Cloud doesn’t offer fully “automatic quota adjustments” for all resources; however, it provides tools and features to help manage them.
I do see the following document that describes how to create an automatic quota adjuster which I know is not enabled in my project.
FYI, I can get the quota data from the API if that would be helpful. There’s just a lot of data and I don’t always know which parts are relevant.
Also, I’m happy to be proven wrong about automatic quotas since it seems I probably didn’t see that in the documentation. If there is documentation showing that automatic quota adjustments do happen and why, I’d be interested to see that documentation. My own search for anything along those lines hasn’t turned up anything.
The quota limit of 0 for the specific combination (e.g., N4 VM family in us-west4) likely isn’t being created when you make the first VM insertion attempt. It was always 0, but the system did not pre-populate the Quotas page with every possible quota limit for every resource, region, and dimension.
Said another way, the quota wasn’t created by the failed attempt; the attempt simply made the existing 0 limit visible.
This does appear to be a bug in the Console UI, so I will report this. Appreciate you highlighting this.
So my next question would be, how can I determine beforehand if the quota is actually 0? I see some system limits for regions (but not vm_family) set to 0. For example:
But then I also see System Limits for vm_family with non-zero values:
Also, I see one for the zone and vm_family:
So, is there a document somewhere that explains how the quotas and system limits are applied so that I can take the quota data and make a determination of whether the limit is actually 0?Obviously the available quota is affected by other tenancy’s usage of a given machine type in the same region and the availability of physical hardware resources. However, assuming there are still resources available in the region/zone, is there a way to figure out what the “effective” quota would be?
View and manage quotas | Cloud Quotas | Google Cloud is the public doc for quota. Each quota bucket has its own effective limit. A quota bucket is determined by a unique project, service (API), limit name, dimension (region/resource/user, custom dimension etc). The system limits are non-adjustable quota, the limit values are pre-configured by Google compute team.
Thanks for that, @Adam_Qin. That makes sense but I’m still unsure how to take all of the existing limits and quotas and figure out the effective quota. The link only really shows how to look at the existing quotas (or modify them) but doesn’t explain how to interpret them. Even the “quota bucket” term you used is something I haven’t seen elsewhere.
For example, in my tenancy, all of the system limits for Name “CPUs per VM family (default)” with dimensions that include only the region have a value of 0. Obviously, you can’t create a VM without a machine type/vm_family so when is that system limit relevant?
According to Andrew_B’s earlier message, there is a a quota of 0 on the region/vm_family combinations I was trying but I still don’t know how to tell if that is the case before trying to insert a VM. It seems like the only 0-value quotas and limits in “CPUs per VM family (default)” (excluding the ones that showed up after a VM insertion attempt) are system limits on regions but there are some 0-value system limits on regions that don’t prevent insertion of VMs to that region.
So, I’m still looking for a way to find the effective quota given all of the system limits and quotas I see.
The first row shows the effective quota limit of CPUs per vm family for your project in region us-west4 and vm N4 is 0. This value was pre-configured by Google Compute Engine team. It prevents the user creating any N4 VMs in that region. Since the quota type is Quota and Adjustable col is Yes, it means you could request a quota increase for this quota bucket.
The next row shows CPUS per VM family in zone us-west4-a, vm_family N4 has a Quota type of System limit. The Adjustable col shows No. This means this quota bucket has a non-adjustable system limit that you cannot request any quota increase or decrease.
In general system limit is non-adjustable, quota is adjustable.
Okay, thanks again. I must not be making myself clear, I am sorry about that. I understand that system limits are not adjustable. I am not interested in in adjusting quotas at this time. I maintain an application that provisions VMs in Google Cloud for customers and sometimes it has to choose a machine type automatically. I don’t want it to choose a machine type that has no available quota. So I’m trying to understand how I can determine–before inserting the VM–whether there is likely to be a quota issue. I understand that even if the effective quota is non-zero, there may not actually be available resources. That is fine, I just don’t want to pick a machine type that has a zero quota.
Interpreting each row as a standalone quota is straight forward as you suggest. But combining multiple rows is the hard part. I want to understand how to combine multiple rows into a single effective quota.
I previously posted these three rows:
How would I determine that those three rows tell me that the effective quota is 0 when one is 0, one is 300 and one is unlimited?
There are other system limits where the dimension is only the region and the value is zero but I am able to provision to that region. So it appears that system limit is not always applied.
