Error Message: replica scheduling failed

Longhorn
1

Hello, I have a single-node k3s cluster installed on Azure VM. On the cluster I have installed Longhorn using kubectl. I have made a PersistentVolumeClaim and Deployment for Clickhouse but I guess this does not matter. Once I applied the k8s config, I bumped every time in this error shown on Longhorn UI:

Replica Scheduling Failure
Error Message: replica scheduling failed

In the logs of the longhorn manager pod I can see this message:
level=error msg="There's no available disk for replica pvc-cb754e28-d7dc-406a-8ddf-ad27f37d7ca4-r-201bab51, size 10737418240"

Here is my k8s yaml file:

# PersistentVolumeClaim
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: clickhouse-data
  namespace: observability
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: longhorn
---
# Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: clickhouse
  namespace: observability
spec:
  replicas: 1
  selector:
    matchLabels:
      app: clickhouse
  template:
    metadata:
      labels:
        app: clickhouse
    spec:
      volumes:
        - name: clickhouse-data
          persistentVolumeClaim:
            claimName: clickhouse-data
      containers:
      - name: clickhouse
        image: clickhouse/clickhouse-server
        ports:
        - containerPort: 8123
        - containerPort: 9000
        resources:
          limits:
            cpu: 600m
            memory: 4Gi
          requests:
            cpu: 300m
            memory: 2Gi
        volumeMounts:
            - name: clickhouse-data
              mountPath: /var/lib/clickhouse/data
              subPath: data

I would be happy to get some assistance cause I literally have no clue why this could happen. Thanks

2

Notice that you are using a single-node cluster. I would recommend:

  1. Change the field numberOfReplicas to 1 for the StorageClass. The default StorageClass is using value 3. You can change it by kubectl -n longhorn-system edit cm longhorn-storageclass
  2. Check if there is at least one available disk on the UI - Node page
1 Like
3

Thanks for the quick response, this solved my issue