Hello All,
I’m somewhat new to Rancher, so i’m looking for any help i can get with this issue. I’ve got a few Rancher control planes that were all initially running Rancher 2.4.5 and provisioning to a VMware 6.7U3 environment. I upgraded one of the environments to Rancher 2.5.8, and I upgraded the local clusters K8s version from 1.18.x to 1.20.6. Soon thereafter, I noticed a changed in the way Rancher was provisioning clusters/nodes.
Prior to the upgrade, I could spin up multiple clusters and all the nodes of the provisioned clusters would begin cloning in vCenter simultaneously. The clones might slow down a bit if there were a lot of them, but they would all be created together. When I upgraded the first control plane, I noticed that cluster provisioning was cloning each node one at a time, or sometimes two at a time. I could spin up three clusters and watch the nodes clone one at a time in vCenter. In this particular environment, some automation was being tested so i initially thought it might be related to that.
I then upgraded a second control plane to the same versions, and we started getting the same thing there as well. So i know it must be related to the Rancher version or K8s version, as nothing else as changed. Also, the third control plane that is still on 2.4.5 is still cloning nodes in parallel.
I’ve been researching this but have not found very much in the way of matching symptoms or bugs. I did start reading about in tree and out of tree cloud providers and I know that Rancher incorporated out of tree cloud providers starting at 2.5+, but as far as I can tell it is not a hard requirement to move to in tree cloud providers.
Has anyone run into something like this previously, or does anyone have any ideas or suggestions? Any help is greatly appreciated.
Thanks