Rancher2 vSphere Provisioning 403 error

Hey,

if I want to create a machine over the rancher vsphere plugin the node is created and drives up then comes after some time the following error.

Here is a detail from the rancher container log.

2018/10/25 10:01:41 [INFO] stdout: (node1) Waiting for VMware Tools to come online...
2018-10-25 10:03:17.714156 W | wal: sync duration of 3.751225364s, expected less than 1s
2018-10-25 10:03:27.171785 I | mvcc: store.index: compact 64581
2018-10-25 10:03:27.173783 I | mvcc: finished scheduled compaction at 64581 (took 1.410553ms)
2018-10-25 10:03:29.937074 W | wal: sync duration of 1.077891015s, expected less than 1s
E1025 10:04:27.349715       8 watcher.go:208] watch chan error: etcdserver: mvcc: required revision has been compacted
2018-10-25 10:04:42.169839 W | wal: sync duration of 2.090397449s, expected less than 1s
2018/10/25 10:04:54 [INFO] stdout: (node1) Provisioning certs and ssh keys...
2018/10/25 10:04:56 [INFO] stdout: The default lines below are for a sh/bash shell, you can specify the shell you're using, with the --shell flag.
2018/10/25 10:04:56 [INFO] stdout:
2018/10/25 10:04:56 [INFO] Generating and uploading node config node1
**2018/10/25 10:04:56 [ERROR] NodeController c-dphwn/m-drlvm [node-controller] failed with : Error creating machine: Error in driver during machine creation: 403 Forbidden**
2018/10/25 10:04:56 [INFO] Generating and uploading node config

After the error, the machine is deleted and the preparation process starts again.

someone had ever the same problem?

Greetings
Patrick

In our case the NO_PROXY setting was the problem. We used CIDR, see this topic. While trying to upload the config the firewall blocked the request creating the 403 error.