RancherOS on AHV (Acropolis Hypervisor)

RancherOS
1

Hello,

currently I am using RancherOS as GuestOS on a Hyper-V Cluster for my Rancher Cluster (k8s).

Now the infrastructure team planning to switch to Nutanix with AHV as hypervisor. AHV is based on KVM. The compatibility matrix of Nutanix does not list RancherOS as supported GuestOS. But what I understand so far is that the GuestOS only needs to support the virtio driver. I can’t find any information about RancherOS and virtio. Does the kernel of RancherOS contains the virtio drivers, as most modern linux distributions?

In an test environment I already installed RancherOS on AHV, therefore I used the rancheros.iso without any vendor.
Is this the right image for KVM?
I can start the VM and everthings seems to work, but there is a container in the system-docker rancher/os-hypervvmtools which restart every few seconds.
Why is this container started?

Any help or information would be great.

Best regards,
sike

2

Hello Sike,

I’ve been using RancherOS among rancher and also longhorn on some nutanix boxes with AHV. I enabled the open-vm-tools service and everything looks to me working as expected, without any noticeable issue so far.

Let me know what kind of information are you expecting, so I can provide you further details.

Regards.

Gabriel

3

Hi Gabriel,

the documentation of RancherOS contains the following paragraph:

The RancherOS ISO file can be used to create a fresh RancherOS install on KVM, VMware, VirtualBox, Hyper-V, Proxmox VE, or bare metal servers. You can download the rancheros.iso file from our releases page.

Some hypervisors may require a built-in agent to communicate with the guest, for this, RancherOS precompiles some ISO files.

My question is, which Image should I use for AHV, because I can’t find a special image for KVM, which is the base of AHV. So I used the standard rancher.iso image and after the first boot I was a little bit confused that the hyperv-tools were started.
In the meantime I know why this happens, because RancherOS has an autodiscover of the hypervisor, which seems to have a bug. Because on AHV it detects hyperv as hypervisor. As you can see in the following log:

time=“2020-03-03T15:36:59Z” level=info msg=“[6/23] Starting load modules”
time=“2020-03-03T15:36:59Z” level=debug msg=“Loading module hv_utils”
time=“2020-03-03T15:36:59Z” level=error msg=“Could not load module hv_utils, err exit status 1”
time=“2020-03-03T15:36:59Z” level=debug msg=“Loading module hv_storvsc”
time=“2020-03-03T15:36:59Z” level=error msg=“Could not load module hv_storvsc, err exit status 1”
time=“2020-03-03T15:36:59Z” level=debug msg=“Loading module hv_vmbus”
time=“2020-03-03T15:37:00Z” level=error msg=“Could not load module hv_vmbus, err exit status 1”
time=“2020-03-03T15:37:00Z” level=debug msg=“[6/23] Done load modules”
time=“2020-03-03T15:37:00Z” level=info msg=“[7/23] Starting recovery console”
time=“2020-03-03T15:37:00Z” level=debug msg=“[7/23] Done recovery console”
time=“2020-03-03T15:37:00Z” level=info msg=“[8/23] Starting b2d env”
time=“2020-03-03T15:37:00Z” level=debug msg=“[8/23] Done b2d env”
time=“2020-03-03T15:37:00Z” level=info msg=“[9/23] Starting mount STATE and bootstrap”
time=“2020-03-03T15:37:00Z” level=debug msg=“FsType has been set to ext4”
time=“2020-03-03T15:37:00Z” level=info msg=“Mounting state device /dev/sda1 to /state”
time=“2020-03-03T15:37:00Z” level=debug msg=“[9/23] Done mount STATE and bootstrap”
time=“2020-03-03T15:37:00Z” level=info msg=“[10/23] Starting cloud-init”
time=“2020-03-03T15:37:00Z” level=info msg=“ros init: Detected Hypervisor: hyperv”

I already reported this bug here.

It is possible to disable this autodetection as described here, but on boot up the VM still tries to load the modules hv_utils, hv_storvsc and hv_vmbus. Only the os-hypervvmtools container isn’t started anylonger.

Until now I didn’t recognized any problems with this VM, but I don’t know if this is the right way.

Have you already checked if the container of the open-vm-tools is running as expected (sudo system-docker ps)? Because on my VM the container restarts every few seconds without any logs, if I enable this service.

Best regards,
sike

4

Hello Sike,

I notice this bug long time ago, but never bothered about it because I have disabled the autodetection as described in documentation linked by you too.

I just notice about the open-vm-tools container restarting. I’m not sure what is the expected behavior. It’s a interesting find.

I also disabled the hyperv-vm-tools (enabled by default) in my template keeping enabled only the open-vm-tools and everything seems to work properly.

Gabriel

5

Hello again Sike,

I removed the open-vm-tools mostly because the KVM drivers (virtio) are part of the linux kernel in the majority of the distros and everything is working and also the guest shutdown and guest reboot features from nutanix console.

msgs from syslog:

Mar 5 18:34:19 rancher kernel: [ 0.000000] Hypervisor detected: KVM
…
Mar 5 18:34:19 rancher kernel: [ 0.000000] Booting paravirtualized kernel on KVM

Regards,

Gabriel