Problems creating high availability RKE

Hello everyone, I am trying to stand up a high availability Rancher cluster with the air gapped cluster installation instructions and I am having trouble with the RKE install step.

When I perform the rke up command I get some output that looks like this:
Failed to set up SSH tunneling for host [XXX.XXX.XXX.XXX]: Can’t retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Failed to dial ssh using address [XXX.XXX.XXX.XXX]: ssh handshake failed: ssh: signature did not verify.

This is using RKE v0.2.2.

I have checked the troubleshooting page and while there are a number of SSH problems there, this one isn’t covered. I have checked them all anyway:

  • I can ssh to each host as the user and access docker without using sudo
  • I am using SSH server 7.4
  • The user executing rke can access the private key
  • I have confirmed that the private key can be used to log in to the server
  • The key is not encrypted

I tracked down where the “signature did not verify” message comes from, it is from the Go ssh/keys.go file. I’m not really across how this works so I couldn’t work out what causes this message to appear though.

The environment I work in is quite restrictive so I have no doubt that this is a root cause of the problem but I would like to know what I need to get changed to make this work, or at least get further down the troubleshooting path.

Bump Has no one seen this issue before, or have any suggestions?

In the interests of transferring Wisdom of the Ancients I worked out what was causing this after spending some quality time with packet dumps.

I had to modify our corporate sshd_config file to disable all HostKey directives except the RSA one. It’s possible the Go SSH client has a problem with the more exciting types of host keys but I kind of stopped investigating once I got it working.