Rancher not starting vSphere virtual machines

We are currently setting up a Rancher 2.x environment and want to use RKE with vmwares vSphere to automatically create new virtual machines to deploy pods on. Rancher creates new vm’s but then fails on “Uploading cloud-init.iso” with “Error creating machine: Error in driver during machine creation: 404 Not Found”. I’ve tried to find anything in Ranchers system pod logs that would indicate an authorization failure but have not found anything. Is there something that should be provided by Rancher (via template or cluster settings) that power-on the vm’s? We are cloning an existing vm at creation if that helps!

1 Like

Sorry for reviving this half a year on, but we had the same error and this was one of the only Google hits so I thought I’d share our answer in case it helps someone else in future.

After digging into the source for rancher-machine it turns out that if you see ‘Uploading cloud-init.iso’ but receive an error before seeing ‘Mounting cloud-init.iso’ then rancher-machine has failed to upload the ISO file to the datastore.

Reasons for this may vary but in our case the vSphere account we were using was missing the Host > Configuration > System Management permission required to upload files to the datastore.

This was tough to troubleshoot for a few reasons:

  • The log messages refer to cloud-init.iso but the filename is user-data.iso
  • vSphere’s event log claims that ‘user-data.iso’ was successfully uploaded (but it wasn’t)
  • the SOAP client that rancher-machine uses does not provide any context around the 404 and rancher-machine doesn’t add it

I’m hoping to raise a PR to fix items 1 and 3 when I get a chance. Meanwhile, I hope this helps somebody!

1 Like