Cluster rke : Kubelet has disk pressure

Rancher 2.x
1

Hi,

I set up a rke cluster, registered the nodes on rancher

I did a manipulation (related to volume storage: persistent volume or storage classes) I have the following error message on rancher: “Kubelet has disk pressure”

Do you have an idea ?

Best regards

2

usually it’s due a disk space, try to give more disk to /var/lib/docker or /var/lib

I have this advice when the 90% of disk is full

3

I agree with marcosx. Some more info for you, the k8s doco tell us:

kubelet supports only two filesystem partitions.

  1. The nodefs filesystem that kubelet uses for volumes, daemon logs, etc.
  2. The imagefs filesystem that container runtimes uses for storing images and container writable layers.

Depending on your partition scheme you can be getting disk pressure for either of these sources. Nodefs tends to be the partition with /var/lib/kubernetes while imagefs tends to be the partition with /var/lib/docker. If these are the same then there is no seperate imagefs partition.

A trap for you players is if /var/log is a separate partition to /var then you can get a situation where /var/log has run out of space but disk pressure has not been triggered. Pods will get scheduled to the node but then will not start up but Kubernetes cannot work out why so they sit there attempting to provision forever.