I am encountering two issues with my Kubernetes cluster:
1.Frequent Node Recreation:
Over the past month, I’ve noticed that nodes are being recreated randomly at least once daily. Initially, I identified the nodes as preemptible and disabled this setting, but the issue persists. What could be causing these recreations, and how can I prevent this from happening in the future?
2.Unexpected Node Upgrades:
Despite disabling auto-upgrade and switching the release channel to “No Channel,” my nodes were still upgraded from version 1.30.3-x (approximate) to 1.30.5-gke.1014003.
What could have overridden the auto-upgrade settings, and how can I ensure such upgrades are avoided in the future?
For #1, do your logs say anything? Is it still one or more recreates a day?
For #2, disabling auto-upgrades stops direct upgrade operations, but anything that causes a node recreation will create the new nodes at the current control plane version if the control plane version was different than the node version. You can’t turn off control plane upgrades - disabling node auto-upgrades only stops GKE from upgrading your node versions, putting the responsibility on you to avoid too big of a version skew.
Probably what happened was that some other operation caused your nodes to get recreated (the issue you described in #1?), and they got recreated at the current control plane patch version.
