We set up horizontal auto scaling.
This is basically to save money.
Originally the minimum nodes were set rather high and no matter how much we lowered the min replicas on a deployment the number of nodes did not go down thus we were not saving any money.
So we lowered the minimum nodes in Autoscaling.
It works for the most part however we just tried a heavier load and we are receiving outofpods errors.
The maximum amount of nodes in autoscaling is set much higher and it doesn’t seem to need that many nodes. I am not understanding why we are receiving these issues. Shouldn’t a new node just be created when an existing node is out of pods?
Should we configure the minimum nodes in Autoscaling higher? However if we do that we won’t be saving as much money. Is there another option? Or is there another reason why we are receiving this error?
Hi @GoogleDevGuy,
You might consider using Spot VMs in GKE, Spot VMs in GKE are priced lower than standard VMs. You can use Spot VMs in your clusters and node pools to run stateless, batch, or fault-tolerant workloads that can tolerate disruptions caused by the ephemeral nature of Spot VMs. Please take note that Spot VMs provide no guarantee of availability.
In addition, you can also check Vertical Pod Scaling, considering the CPU and memory resources of your existing nodes. Vertical Pod Scaling lets you analyze and set CPU and memory resources required by Pods. If you were using the two scalers, you can also check their limitations before deploying it.
Regarding the OutofPods error that you are encountering, here are some recommendations:
- Adjusting the Maximum number of pods.
- Review Resource Request and Limits of Pods
- Considering Affinity and anti-affinity
For more details, you can check here in this documentation.
Lastly, you can use Google Cloud Pricing calculator, this tool creates cost estimates based on the assumptions that you provide.
I hope the above information is helpful.