Run k3s in Incus container

catav · September 27, 2024, 4:46pm

Hello,
I am trying to install k3s in an Incus container. Creating an Incus container without security.nesting and security.priviliged set to true or not does not solve the problem.

The command I used for installing K3s is:
curl -sfL https://get.k3s.io | K3S_KUBECONFIG_MODE=“644” sh -s -

kubectl get -A all

E0927 16:48:56.232410   15283 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": dial tcp 127.0.0.1:6443: connect: connection refused
E0927 16:48:56.232616   15283 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": dial tcp 127.0.0.1:6443: connect: connection refused
E0927 16:48:56.233836   15283 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": dial tcp 127.0.0.1:6443: connect: connection refused
E0927 16:48:56.233995   15283 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": dial tcp 127.0.0.1:6443: connect: connection refused
E0927 16:48:56.235238   15283 memcache.go:265] couldn't get current server API group list: Get "https://127.0.0.1:6443/api?timeout=32s": dial tcp 127.0.0.1:6443: connect: connection refused
The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?

sudo systemctl status k3s.service

● k3s.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s.service; enabled; preset: enabled)
     Active: active (running) since Fri 2024-09-27 16:50:06 UTC; 415ms ago
       Docs: https://k3s.io
    Process: 15591 ExecStartPre=/bin/sh -xc ! /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service 2>/dev/null (code=exited, status=0/SUCCESS)
    Process: 15593 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
    Process: 15594 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Main PID: 15595 (k3s-server)
      Tasks: 4
     Memory: 346.8M
        CPU: 4.807s
     CGroup: /system.slice/k3s.service
             └─15595 "[k3s-server]"

Sep 27 16:50:06 debk3s k3s[15595]: I0927 16:50:06.816539   15595 container_manager_linux.go:270] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"debk3s","RuntimeCgroupsName":"","SystemCgroupsName":"","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":false,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":null,"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"ExperimentalMemoryManagerPolicy":"None","ExperimentalMemoryManagerReservedMemory":null,"PodPidsLimit":-1,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null}
Sep 27 16:50:06 debk3s k3s[15595]: I0927 16:50:06.816804   15595 topology_manager.go:138] "Creating topology manager with none policy"
Sep 27 16:50:06 debk3s k3s[15595]: I0927 16:50:06.816819   15595 container_manager_linux.go:301] "Creating device plugin manager"
Sep 27 16:50:06 debk3s k3s[15595]: I0927 16:50:06.816865   15595 state_mem.go:36] "Initialized new in-memory state store"
Sep 27 16:50:06 debk3s k3s[15595]: I0927 16:50:06.816956   15595 kubelet.go:401] "Attempting to sync node with API server"
Sep 27 16:50:06 debk3s k3s[15595]: I0927 16:50:06.816970   15595 kubelet.go:302] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests"
Sep 27 16:50:06 debk3s k3s[15595]: I0927 16:50:06.816994   15595 kubelet.go:313] "Adding apiserver pod source"
Sep 27 16:50:06 debk3s k3s[15595]: I0927 16:50:06.817007   15595 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 27 16:50:06 debk3s k3s[15595]: Error: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory
Sep 27 16:50:06 debk3s k3s[15595]: time="2024-09-27T16:50:06Z" level=error msg="kubelet exited: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory"

“sudo journalctl -xeu k3s.service” command produces following errors:

Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949018    4325 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949303    4325 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949421    4325 secure_serving.go:213] Serving securely on 127.0.0.1:6444
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949468    4325 tlsconfig.go:240] "Starting DynamicServingCertificateController"
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949528    4325 system_namespaces_controller.go:67] Starting system namespaces controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949621    4325 controller.go:116] Starting legacy_token_tracking_controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949632    4325 shared_informer.go:313] Waiting for caches to sync for configmaps
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949663    4325 gc_controller.go:78] Starting apiserver lease garbage collector
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949942    4325 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.949951    4325 shared_informer.go:313] Waiting for caches to sync for cluster_authentication_trust_controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950016    4325 apiservice_controller.go:97] Starting APIServiceRegistrationController
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950021    4325 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt::/var/lib/rancher/k3s/server/tls/client-auth-proxy.key"
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950025    4325 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950211    4325 aggregator.go:163] waiting for initial CRD sync...
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950229    4325 controller.go:78] Starting OpenAPI AggregationController
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950262    4325 apf_controller.go:374] Starting API Priority and Fairness config controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950102    4325 controller.go:80] Starting OpenAPI V3 AggregationController
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950110    4325 customresource_discovery_controller.go:289] Starting DiscoveryController
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950124    4325 available_controller.go:423] Starting AvailableConditionController
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950330    4325 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950582    4325 crdregistration_controller.go:111] Starting crd-autoregister controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950593    4325 shared_informer.go:313] Waiting for caches to sync for crd-autoregister
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950687    4325 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt"
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950751    4325 controller.go:139] Starting OpenAPI controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950783    4325 controller.go:87] Starting OpenAPI V3 controller
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950801    4325 naming_controller.go:291] Starting NamingConditionController
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950816    4325 establishing_controller.go:76] Starting EstablishingController
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950837    4325 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950855    4325 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950870    4325 crd_finalizer.go:266] Starting CRDFinalizer
Sep 27 16:08:56 debk3s k3s[4325]: I0927 16:08:56.950750    4325 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050288    4325 cache.go:39] Caches are synced for APIServiceRegistrationController controller
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050328    4325 shared_informer.go:320] Caches are synced for cluster_authentication_trust_controller
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050366    4325 handler.go:286] Adding GroupVersion helm.cattle.io v1 to ResourceManager
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050386    4325 handler.go:286] Adding GroupVersion k3s.cattle.io v1 to ResourceManager
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050452    4325 cache.go:39] Caches are synced for AvailableConditionController controller
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050657    4325 shared_informer.go:320] Caches are synced for crd-autoregister
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050694    4325 aggregator.go:165] initial CRD sync complete...
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050701    4325 autoregister_controller.go:141] Starting autoregister controller
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050707    4325 cache.go:32] Waiting for caches to sync for autoregister controller
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050713    4325 cache.go:39] Caches are synced for autoregister controller
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050296    4325 apf_controller.go:379] Running API Priority and Fairness config worker
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.059179    4325 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.059280    4325 handler_discovery.go:447] Starting ResourceDiscoveryManager
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.050304    4325 shared_informer.go:320] Caches are synced for configmaps
Sep 27 16:08:57 debk3s k3s[4325]: E0927 16:08:57.066680    4325 controller.go:102] Error removing old endpoints from kubernetes service: no API server IP addresses were listed in storage, refusing to erase all endpoints for the kubernetes Service
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.073876    4325 shared_informer.go:320] Caches are synced for node_authorizer
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.077181    4325 shared_informer.go:320] Caches are synced for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.077210    4325 policy_source.go:224] refreshing policies
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.126024    4325 controller.go:615] quota admission added evaluator for: leases.coordination.k8s.io
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=info msg="Password verified locally for node debk3s"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=info msg="certificate CN=debk3s signed by CN=k3s-server-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:08:57 +0000 UTC"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=info msg="certificate CN=system:node:debk3s,O=system:nodes signed by CN=k3s-client-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:08:57 +0000 UTC"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=warning msg="Host resolv.conf includes loopback or multicast nameservers - kubelet will use autogenerated resolv.conf with nameserver 8.8.8.8"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=info msg="Module overlay was already loaded"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=info msg="Module nf_conntrack was already loaded"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=warning msg="Failed to load kernel module br_netfilter with modprobe"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=warning msg="Failed to load kernel module iptable_nat with modprobe"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=warning msg="Failed to load kernel module iptable_filter with modprobe"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=info msg="Set sysctl 'net/bridge/bridge-nf-call-iptables' to 1"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=error msg="Failed to set sysctl: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=error msg="Failed to set sysctl: open /proc/sys/net/netfilter/nf_conntrack_max: permission denied"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
Sep 27 16:08:57 debk3s k3s[4325]: time="2024-09-27T16:08:57Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
Sep 27 16:08:57 debk3s k3s[4325]: I0927 16:08:57.964120    4325 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Waiting for cloud-controller-manager privileges to become available"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Creating k3s-supervisor event broadcaster"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Kube API server is now running"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="ETCD server is now running"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="k3s is up and running"
Sep 27 16:08:58 debk3s systemd[1]: Started k3s.service - Lightweight Kubernetes.
░░ Subject: A start job for unit k3s.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has finished successfully.
░░ 
░░ The job identifier is 1157.
Sep 27 16:08:58 debk3s k3s[4325]: W0927 16:08:58.394491    4325 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Applying CRD addons.k3s.cattle.io"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Applying CRD etcdsnapshotfiles.k3s.cattle.io"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Applying CRD helmcharts.helm.cattle.io"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Applying CRD helmchartconfigs.helm.cattle.io"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-25.0.3+up25.0.0.tgz"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-crd-25.0.3+up25.0.0.tgz"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/local-storage.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-delegator.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-apiservice.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-deployment.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-service.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/ccm.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/aggregated-metrics-reader.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-reader.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/resource-reader.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/rolebindings.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/runtimes.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/traefik.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/coredns.yaml"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Starting dynamiclistener CN filter node controller"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Tunnel server egress proxy mode: agent"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Creating deploy event broadcaster"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.596716    4325 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.643490    4325 controller.go:615] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.648179    4325 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="AppliedManifest" message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.651591    4325 controller.go:615] quota admission added evaluator for: addons.k3s.cattle.io
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.661319    4325 event.go:389] "Event occurred" object="kube-system/coredns" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\""
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Creating helm-controller event broadcaster"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Starting /v1, Kind=Node controller"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.705388    4325 controller.go:615] quota admission added evaluator for: deployments.apps
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Waiting for control-plane node debk3s startup: nodes \"debk3s\" not found"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="containerd is now running"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Cluster dns configmap already exists"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Creating k3s-cert-monitor event broadcaster"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Connecting to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=debk3s --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-ip=10.174.47.117,fd42:f085:b67c:18d1:216:3eff:fee7:36c3 --node-labels= --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/var/lib/rancher/k3s/agent/etc/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Handling backend connection request [debk3s]"
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=info msg="Remotedialer connected to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect"
Sep 27 16:08:58 debk3s k3s[4325]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.
Sep 27 16:08:58 debk3s k3s[4325]: W0927 16:08:58.745195    4325 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:08:58 debk3s k3s[4325]: W0927 16:08:58.745669    4325 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.747850    4325 server.go:479] "Kubelet version" kubeletVersion="v1.30.5+k3s1"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.747878    4325 server.go:481] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 27 16:08:58 debk3s k3s[4325]: W0927 16:08:58.747925    4325 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:08:58 debk3s k3s[4325]: W0927 16:08:58.748003    4325 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.752126    4325 serving.go:387] Generated self-signed cert in-memory
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.755888    4325 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.760367    4325 event.go:389] "Event occurred" object="kube-system/coredns" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="AppliedManifest" message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\""
Sep 27 16:08:58 debk3s k3s[4325]: W0927 16:08:58.774967    4325 fs.go:216] stat failed on /dev/sdb2 with error: no such file or directory
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.786175    4325 event.go:389] "Event occurred" object="kube-system/local-storage" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/local-storage.yaml\""
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.796168    4325 server.go:737] "--cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.796203    4325 server.go:805] "NoSwap is set due to memorySwapBehavior not specified" memorySwapBehavior="" FailSwapOn=false
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.796744    4325 container_manager_linux.go:265] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.796833    4325 container_manager_linux.go:270] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"debk3s","RuntimeCgroupsName":"","SystemCgroupsName":"","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":false,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":null,"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"ExperimentalMemoryManagerPolicy":"None","ExperimentalMemoryManagerReservedMemory":null,"PodPidsLimit":-1,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null}
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.797209    4325 topology_manager.go:138] "Creating topology manager with none policy"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.797314    4325 container_manager_linux.go:301] "Creating device plugin manager"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.797363    4325 state_mem.go:36] "Initialized new in-memory state store"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.797463    4325 kubelet.go:401] "Attempting to sync node with API server"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.797479    4325 kubelet.go:302] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.797505    4325 kubelet.go:313] "Adding apiserver pod source"
Sep 27 16:08:58 debk3s k3s[4325]: I0927 16:08:58.797517    4325 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 27 16:08:58 debk3s k3s[4325]: Error: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory
Sep 27 16:08:58 debk3s k3s[4325]: time="2024-09-27T16:08:58Z" level=error msg="kubelet exited: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory"
Sep 27 16:08:58 debk3s systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ An ExecStart= process belonging to unit k3s.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Sep 27 16:08:58 debk3s systemd[1]: k3s.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service has entered the 'failed' state with result 'exit-code'.
Sep 27 16:08:58 debk3s systemd[1]: k3s.service: Consumed 4.817s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service completed and consumed the indicated resources.
Sep 27 16:09:04 debk3s systemd[1]: k3s.service: Scheduled restart job, restart counter is at 15.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ Automatic restarting of the unit k3s.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Sep 27 16:09:04 debk3s systemd[1]: Stopped k3s.service - Lightweight Kubernetes.
░░ Subject: A stop job for unit k3s.service has finished
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A stop job for unit k3s.service has finished.
░░ 
░░ The job identifier is 1214 and the job result is done.
Sep 27 16:09:04 debk3s systemd[1]: k3s.service: Consumed 4.817s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service completed and consumed the indicated resources.
Sep 27 16:09:04 debk3s systemd[1]: Starting k3s.service - Lightweight Kubernetes...
░░ Subject: A start job for unit k3s.service has begun execution
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has begun execution.
░░ 
░░ The job identifier is 1214.
Sep 27 16:09:04 debk3s sh[4356]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Sep 27 16:09:04 debk3s (modprobe)[4358]: k3s.service: Executable /sbin/modprobe missing, skipping: No such file or directory
░░ Subject: Process /sbin/modprobe could not be executed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The process /sbin/modprobe could not be executed and failed.
░░ 
░░ The error number returned by this process is 2.
Sep 27 16:09:04 debk3s (modprobe)[4359]: k3s.service: Executable /sbin/modprobe missing, skipping: No such file or directory
░░ Subject: Process /sbin/modprobe could not be executed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The process /sbin/modprobe could not be executed and failed.
░░ 
░░ The error number returned by this process is 2.
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Starting k3s v1.30.5+k3s1 (9b586704)"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Configuring sqlite3 database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Database tables and indexes are up to date"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Kine available at unix://kine.sock"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Reconciling bootstrap data between datastore and disk"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --etcd-servers=unix://kine.sock --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Running kube-scheduler --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --leader-elect=false --profiling=false --secure-port=10259"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Waiting for API server to become available"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Running kube-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --configure-cloud-routes=false --controllers=*,tokencleaner,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --leader-elect=false --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=10257 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --use-service-account-credentials=true"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.377192    4360 options.go:221] external host was not specified, using 10.174.47.117
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/k3s/server/etc/cloud-config.yaml --cloud-provider=k3s --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route --feature-gates=CloudDualStackNodeIPs=true --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --leader-elect=false --leader-elect-resource-name=k3s-cloud-controller-manager --node-status-update-frequency=1m0s --profiling=false"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.377783    4360 server.go:156] Version: v1.30.5+k3s1
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.377804    4360 server.go:158] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Server node token is available at /var/lib/rancher/k3s/server/token"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="To join server node to cluster: k3s server -s https://10.174.47.117:6443 -t ${SERVER_NODE_TOKEN}"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Agent node token is available at /var/lib/rancher/k3s/server/agent-token"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="To join agent node to cluster: k3s agent -s https://10.174.47.117:6443 -t ${AGENT_NODE_TOKEN}"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml"
Sep 27 16:09:04 debk3s k3s[4360]: time="2024-09-27T16:09:04Z" level=info msg="Run: k3s kubectl"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.390457    4360 shared_informer.go:313] Waiting for caches to sync for node_authorizer
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.396162    4360 shared_informer.go:313] Waiting for caches to sync for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.399424    4360 plugins.go:157] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.399442    4360 plugins.go:160] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.399647    4360 instance.go:299] Using reconciler: lease
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.414549    4360 handler.go:286] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.414575    4360 genericapiserver.go:733] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.462769    4360 handler.go:286] Adding GroupVersion  v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.462956    4360 instance.go:696] API group "internal.apiserver.k8s.io" is not enabled, skipping.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.500342    4360 instance.go:696] API group "storagemigration.k8s.io" is not enabled, skipping.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.542408    4360 instance.go:696] API group "resource.k8s.io" is not enabled, skipping.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.569433    4360 handler.go:286] Adding GroupVersion authentication.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.569604    4360 genericapiserver.go:733] Skipping API authentication.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.569670    4360 genericapiserver.go:733] Skipping API authentication.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.570147    4360 handler.go:286] Adding GroupVersion authorization.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.570234    4360 genericapiserver.go:733] Skipping API authorization.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.571015    4360 handler.go:286] Adding GroupVersion autoscaling v2 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.571742    4360 handler.go:286] Adding GroupVersion autoscaling v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.571753    4360 genericapiserver.go:733] Skipping API autoscaling/v2beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.571759    4360 genericapiserver.go:733] Skipping API autoscaling/v2beta2 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.573032    4360 handler.go:286] Adding GroupVersion batch v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.573043    4360 genericapiserver.go:733] Skipping API batch/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.573854    4360 handler.go:286] Adding GroupVersion certificates.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.573865    4360 genericapiserver.go:733] Skipping API certificates.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.573871    4360 genericapiserver.go:733] Skipping API certificates.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.574392    4360 handler.go:286] Adding GroupVersion coordination.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.574406    4360 genericapiserver.go:733] Skipping API coordination.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.574464    4360 genericapiserver.go:733] Skipping API discovery.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.575041    4360 handler.go:286] Adding GroupVersion discovery.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.576626    4360 handler.go:286] Adding GroupVersion networking.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.576643    4360 genericapiserver.go:733] Skipping API networking.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.576650    4360 genericapiserver.go:733] Skipping API networking.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.577140    4360 handler.go:286] Adding GroupVersion node.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.577154    4360 genericapiserver.go:733] Skipping API node.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.577163    4360 genericapiserver.go:733] Skipping API node.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.578059    4360 handler.go:286] Adding GroupVersion policy v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.578076    4360 genericapiserver.go:733] Skipping API policy/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.579872    4360 handler.go:286] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.579896    4360 genericapiserver.go:733] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.579905    4360 genericapiserver.go:733] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.580355    4360 handler.go:286] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.580369    4360 genericapiserver.go:733] Skipping API scheduling.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.580376    4360 genericapiserver.go:733] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.582360    4360 handler.go:286] Adding GroupVersion storage.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.582373    4360 genericapiserver.go:733] Skipping API storage.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.582380    4360 genericapiserver.go:733] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.583582    4360 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.584791    4360 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.584805    4360 genericapiserver.go:733] Skipping API flowcontrol.apiserver.k8s.io/v1beta2 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.584812    4360 genericapiserver.go:733] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.588295    4360 handler.go:286] Adding GroupVersion apps v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.588318    4360 genericapiserver.go:733] Skipping API apps/v1beta2 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.588326    4360 genericapiserver.go:733] Skipping API apps/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.590053    4360 handler.go:286] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.590066    4360 genericapiserver.go:733] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.590073    4360 genericapiserver.go:733] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.590582    4360 handler.go:286] Adding GroupVersion events.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.590594    4360 genericapiserver.go:733] Skipping API events.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.593808    4360 handler.go:286] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager
Sep 27 16:09:04 debk3s k3s[4360]: W0927 16:09:04.593829    4360 genericapiserver.go:733] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946132    4360 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946137    4360 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946408    4360 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946579    4360 secure_serving.go:213] Serving securely on 127.0.0.1:6444
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946631    4360 tlsconfig.go:240] "Starting DynamicServingCertificateController"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946802    4360 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946814    4360 shared_informer.go:313] Waiting for caches to sync for cluster_authentication_trust_controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946898    4360 controller.go:116] Starting legacy_token_tracking_controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946907    4360 shared_informer.go:313] Waiting for caches to sync for configmaps
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946953    4360 customresource_discovery_controller.go:289] Starting DiscoveryController
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946972    4360 controller.go:78] Starting OpenAPI AggregationController
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.946995    4360 controller.go:80] Starting OpenAPI V3 AggregationController
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947022    4360 system_namespaces_controller.go:67] Starting system namespaces controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947062    4360 gc_controller.go:78] Starting apiserver lease garbage collector
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947083    4360 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt::/var/lib/rancher/k3s/server/tls/client-auth-proxy.key"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947184    4360 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947259    4360 apf_controller.go:374] Starting API Priority and Fairness config controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947326    4360 aggregator.go:163] waiting for initial CRD sync...
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947346    4360 apiservice_controller.go:97] Starting APIServiceRegistrationController
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947356    4360 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947377    4360 available_controller.go:423] Starting AvailableConditionController
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947385    4360 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947439    4360 controller.go:139] Starting OpenAPI controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947465    4360 controller.go:87] Starting OpenAPI V3 controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947481    4360 naming_controller.go:291] Starting NamingConditionController
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947499    4360 establishing_controller.go:76] Starting EstablishingController
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947521    4360 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947534    4360 crdregistration_controller.go:111] Starting crd-autoregister controller
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947544    4360 shared_informer.go:313] Waiting for caches to sync for crd-autoregister
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947556    4360 crd_finalizer.go:266] Starting CRDFinalizer
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947538    4360 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.947260    4360 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:09:04 debk3s k3s[4360]: I0927 16:09:04.990703    4360 shared_informer.go:320] Caches are synced for node_authorizer
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.047153    4360 handler.go:286] Adding GroupVersion k3s.cattle.io v1 to ResourceManager
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.047250    4360 shared_informer.go:320] Caches are synced for cluster_authentication_trust_controller
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.047392    4360 shared_informer.go:320] Caches are synced for configmaps
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.047592    4360 cache.go:39] Caches are synced for APIServiceRegistrationController controller
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.047906    4360 handler.go:286] Adding GroupVersion helm.cattle.io v1 to ResourceManager
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.048088    4360 apf_controller.go:379] Running API Priority and Fairness config worker
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.048098    4360 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.048830    4360 shared_informer.go:320] Caches are synced for crd-autoregister
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.049838    4360 aggregator.go:165] initial CRD sync complete...
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.049941    4360 autoregister_controller.go:141] Starting autoregister controller
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.050026    4360 cache.go:32] Waiting for caches to sync for autoregister controller
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.050165    4360 cache.go:39] Caches are synced for autoregister controller
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.050686    4360 cache.go:39] Caches are synced for AvailableConditionController controller
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.057343    4360 handler_discovery.go:447] Starting ResourceDiscoveryManager
Sep 27 16:09:05 debk3s k3s[4360]: E0927 16:09:05.063985    4360 controller.go:102] Error removing old endpoints from kubernetes service: no API server IP addresses were listed in storage, refusing to erase all endpoints for the kubernetes Service
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.096284    4360 shared_informer.go:320] Caches are synced for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.096323    4360 policy_source.go:224] refreshing policies
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.133153    4360 controller.go:615] quota admission added evaluator for: leases.coordination.k8s.io
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=info msg="Password verified locally for node debk3s"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=info msg="certificate CN=debk3s signed by CN=k3s-server-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:09:05 +0000 UTC"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=info msg="certificate CN=system:node:debk3s,O=system:nodes signed by CN=k3s-client-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:09:05 +0000 UTC"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=warning msg="Host resolv.conf includes loopback or multicast nameservers - kubelet will use autogenerated resolv.conf with nameserver 8.8.8.8"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=info msg="Module overlay was already loaded"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=info msg="Module nf_conntrack was already loaded"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=warning msg="Failed to load kernel module br_netfilter with modprobe"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=warning msg="Failed to load kernel module iptable_nat with modprobe"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=warning msg="Failed to load kernel module iptable_filter with modprobe"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=info msg="Set sysctl 'net/bridge/bridge-nf-call-iptables' to 1"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=error msg="Failed to set sysctl: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=error msg="Failed to set sysctl: open /proc/sys/net/netfilter/nf_conntrack_max: permission denied"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
Sep 27 16:09:05 debk3s k3s[4360]: time="2024-09-27T16:09:05Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
Sep 27 16:09:05 debk3s k3s[4360]: I0927 16:09:05.964933    4360 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Kube API server is now running"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="ETCD server is now running"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="k3s is up and running"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Waiting for cloud-controller-manager privileges to become available"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Creating k3s-supervisor event broadcaster"
Sep 27 16:09:06 debk3s systemd[1]: Started k3s.service - Lightweight Kubernetes.
░░ Subject: A start job for unit k3s.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has finished successfully.
░░ 
░░ The job identifier is 1214.
Sep 27 16:09:06 debk3s k3s[4360]: W0927 16:09:06.408616    4360 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Applying CRD addons.k3s.cattle.io"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Applying CRD etcdsnapshotfiles.k3s.cattle.io"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Applying CRD helmcharts.helm.cattle.io"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Applying CRD helmchartconfigs.helm.cattle.io"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-25.0.3+up25.0.0.tgz"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-crd-25.0.3+up25.0.0.tgz"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/coredns.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/aggregated-metrics-reader.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-deployment.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-service.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/resource-reader.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/traefik.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/ccm.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/local-storage.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-delegator.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-reader.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-apiservice.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/rolebindings.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/runtimes.yaml"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Starting dynamiclistener CN filter node controller"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Tunnel server egress proxy mode: agent"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Creating deploy event broadcaster"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.618167    4360 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.677651    4360 controller.go:615] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.679347    4360 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="AppliedManifest" message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.681004    4360 controller.go:615] quota admission added evaluator for: addons.k3s.cattle.io
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.688597    4360 event.go:389] "Event occurred" object="kube-system/coredns" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\""
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="containerd is now running"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Creating k3s-cert-monitor event broadcaster"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Connecting to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=debk3s --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-ip=10.174.47.117,fd42:f085:b67c:18d1:216:3eff:fee7:36c3 --node-labels= --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/var/lib/rancher/k3s/agent/etc/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Handling backend connection request [debk3s]"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Remotedialer connected to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect"
Sep 27 16:09:06 debk3s k3s[4360]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.
Sep 27 16:09:06 debk3s k3s[4360]: W0927 16:09:06.713606    4360 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:06 debk3s k3s[4360]: W0927 16:09:06.713851    4360 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.716643    4360 server.go:479] "Kubelet version" kubeletVersion="v1.30.5+k3s1"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.716677    4360 server.go:481] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 27 16:09:06 debk3s k3s[4360]: W0927 16:09:06.716721    4360 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:06 debk3s k3s[4360]: W0927 16:09:06.716793    4360 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.718542    4360 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt"
Sep 27 16:09:06 debk3s k3s[4360]: W0927 16:09:06.726166    4360 fs.go:216] stat failed on /dev/sdb2 with error: no such file or directory
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Creating helm-controller event broadcaster"
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=info msg="Starting /v1, Kind=Node controller"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.741795    4360 server.go:737] "--cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.741833    4360 server.go:805] "NoSwap is set due to memorySwapBehavior not specified" memorySwapBehavior="" FailSwapOn=false
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.742246    4360 container_manager_linux.go:265] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.742276    4360 container_manager_linux.go:270] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"debk3s","RuntimeCgroupsName":"","SystemCgroupsName":"","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":false,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":null,"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"ExperimentalMemoryManagerPolicy":"None","ExperimentalMemoryManagerReservedMemory":null,"PodPidsLimit":-1,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null}
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.742512    4360 topology_manager.go:138] "Creating topology manager with none policy"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.742528    4360 container_manager_linux.go:301] "Creating device plugin manager"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.742567    4360 state_mem.go:36] "Initialized new in-memory state store"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.742657    4360 kubelet.go:401] "Attempting to sync node with API server"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.742671    4360 kubelet.go:302] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.742696    4360 kubelet.go:313] "Adding apiserver pod source"
Sep 27 16:09:06 debk3s k3s[4360]: I0927 16:09:06.742707    4360 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 27 16:09:06 debk3s k3s[4360]: Error: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory
Sep 27 16:09:06 debk3s k3s[4360]: time="2024-09-27T16:09:06Z" level=error msg="kubelet exited: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory"
Sep 27 16:09:06 debk3s systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ An ExecStart= process belonging to unit k3s.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Sep 27 16:09:06 debk3s systemd[1]: k3s.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service has entered the 'failed' state with result 'exit-code'.
Sep 27 16:09:06 debk3s systemd[1]: k3s.service: Consumed 4.570s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service completed and consumed the indicated resources.
Sep 27 16:09:12 debk3s systemd[1]: k3s.service: Scheduled restart job, restart counter is at 16.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ Automatic restarting of the unit k3s.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Sep 27 16:09:12 debk3s systemd[1]: Stopped k3s.service - Lightweight Kubernetes.
░░ Subject: A stop job for unit k3s.service has finished
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A stop job for unit k3s.service has finished.
░░ 
░░ The job identifier is 1271 and the job result is done.
Sep 27 16:09:12 debk3s systemd[1]: k3s.service: Consumed 4.570s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service completed and consumed the indicated resources.
Sep 27 16:09:12 debk3s systemd[1]: Starting k3s.service - Lightweight Kubernetes...
░░ Subject: A start job for unit k3s.service has begun execution
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has begun execution.
░░ 
░░ The job identifier is 1271.
Sep 27 16:09:12 debk3s sh[4391]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Sep 27 16:09:12 debk3s (modprobe)[4393]: k3s.service: Executable /sbin/modprobe missing, skipping: No such file or directory
░░ Subject: Process /sbin/modprobe could not be executed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The process /sbin/modprobe could not be executed and failed.
░░ 
░░ The error number returned by this process is 2.
Sep 27 16:09:12 debk3s (modprobe)[4394]: k3s.service: Executable /sbin/modprobe missing, skipping: No such file or directory
░░ Subject: Process /sbin/modprobe could not be executed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The process /sbin/modprobe could not be executed and failed.
░░ 
░░ The error number returned by this process is 2.
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Starting k3s v1.30.5+k3s1 (9b586704)"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Configuring sqlite3 database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Database tables and indexes are up to date"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Kine available at unix://kine.sock"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Reconciling bootstrap data between datastore and disk"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --etcd-servers=unix://kine.sock --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Running kube-scheduler --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --leader-elect=false --profiling=false --secure-port=10259"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Running kube-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --configure-cloud-routes=false --controllers=*,tokencleaner,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --leader-elect=false --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=10257 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --use-service-account-credentials=true"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Waiting for API server to become available"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.366282    4395 options.go:221] external host was not specified, using 10.174.47.117
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/k3s/server/etc/cloud-config.yaml --cloud-provider=k3s --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route --feature-gates=CloudDualStackNodeIPs=true --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --leader-elect=false --leader-elect-resource-name=k3s-cloud-controller-manager --node-status-update-frequency=1m0s --profiling=false"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.366914    4395 server.go:156] Version: v1.30.5+k3s1
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.366950    4395 server.go:158] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Server node token is available at /var/lib/rancher/k3s/server/token"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="To join server node to cluster: k3s server -s https://10.174.47.117:6443 -t ${SERVER_NODE_TOKEN}"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Agent node token is available at /var/lib/rancher/k3s/server/agent-token"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="To join agent node to cluster: k3s agent -s https://10.174.47.117:6443 -t ${AGENT_NODE_TOKEN}"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml"
Sep 27 16:09:12 debk3s k3s[4395]: time="2024-09-27T16:09:12Z" level=info msg="Run: k3s kubectl"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.375887    4395 shared_informer.go:313] Waiting for caches to sync for node_authorizer
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.379900    4395 shared_informer.go:313] Waiting for caches to sync for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.383538    4395 plugins.go:157] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.383554    4395 plugins.go:160] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.383774    4395 instance.go:299] Using reconciler: lease
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.399254    4395 handler.go:286] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.399276    4395 genericapiserver.go:733] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.443327    4395 handler.go:286] Adding GroupVersion  v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.443477    4395 instance.go:696] API group "internal.apiserver.k8s.io" is not enabled, skipping.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.479371    4395 instance.go:696] API group "storagemigration.k8s.io" is not enabled, skipping.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.512766    4395 instance.go:696] API group "resource.k8s.io" is not enabled, skipping.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.526200    4395 handler.go:286] Adding GroupVersion authentication.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.526227    4395 genericapiserver.go:733] Skipping API authentication.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.526238    4395 genericapiserver.go:733] Skipping API authentication.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.526874    4395 handler.go:286] Adding GroupVersion authorization.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.526890    4395 genericapiserver.go:733] Skipping API authorization.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.528017    4395 handler.go:286] Adding GroupVersion autoscaling v2 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.528847    4395 handler.go:286] Adding GroupVersion autoscaling v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.528861    4395 genericapiserver.go:733] Skipping API autoscaling/v2beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.528869    4395 genericapiserver.go:733] Skipping API autoscaling/v2beta2 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.531156    4395 handler.go:286] Adding GroupVersion batch v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.531175    4395 genericapiserver.go:733] Skipping API batch/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.531828    4395 handler.go:286] Adding GroupVersion certificates.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.531842    4395 genericapiserver.go:733] Skipping API certificates.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.531848    4395 genericapiserver.go:733] Skipping API certificates.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.532400    4395 handler.go:286] Adding GroupVersion coordination.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.532412    4395 genericapiserver.go:733] Skipping API coordination.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.532465    4395 genericapiserver.go:733] Skipping API discovery.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.533402    4395 handler.go:286] Adding GroupVersion discovery.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.548963    4395 handler.go:286] Adding GroupVersion networking.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.548989    4395 genericapiserver.go:733] Skipping API networking.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.548998    4395 genericapiserver.go:733] Skipping API networking.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.549562    4395 handler.go:286] Adding GroupVersion node.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.549578    4395 genericapiserver.go:733] Skipping API node.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.549587    4395 genericapiserver.go:733] Skipping API node.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.550425    4395 handler.go:286] Adding GroupVersion policy v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.550436    4395 genericapiserver.go:733] Skipping API policy/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.551747    4395 handler.go:286] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.551767    4395 genericapiserver.go:733] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.551774    4395 genericapiserver.go:733] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.552220    4395 handler.go:286] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.552234    4395 genericapiserver.go:733] Skipping API scheduling.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.552239    4395 genericapiserver.go:733] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.553787    4395 handler.go:286] Adding GroupVersion storage.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.553801    4395 genericapiserver.go:733] Skipping API storage.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.553807    4395 genericapiserver.go:733] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.554568    4395 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.555509    4395 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.555521    4395 genericapiserver.go:733] Skipping API flowcontrol.apiserver.k8s.io/v1beta2 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.555527    4395 genericapiserver.go:733] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.557841    4395 handler.go:286] Adding GroupVersion apps v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.557861    4395 genericapiserver.go:733] Skipping API apps/v1beta2 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.557866    4395 genericapiserver.go:733] Skipping API apps/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.559033    4395 handler.go:286] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.559046    4395 genericapiserver.go:733] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.559051    4395 genericapiserver.go:733] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.559431    4395 handler.go:286] Adding GroupVersion events.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.559440    4395 genericapiserver.go:733] Skipping API events.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.562256    4395 handler.go:286] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: W0927 16:09:12.562278    4395 genericapiserver.go:733] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.886949    4395 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.887225    4395 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.887412    4395 secure_serving.go:213] Serving securely on 127.0.0.1:6444
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.887460    4395 tlsconfig.go:240] "Starting DynamicServingCertificateController"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.887522    4395 controller.go:80] Starting OpenAPI V3 AggregationController
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.887897    4395 controller.go:116] Starting legacy_token_tracking_controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.887908    4395 shared_informer.go:313] Waiting for caches to sync for configmaps
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.887124    4395 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888079    4395 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888090    4395 shared_informer.go:313] Waiting for caches to sync for cluster_authentication_trust_controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888123    4395 gc_controller.go:78] Starting apiserver lease garbage collector
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888168    4395 apiservice_controller.go:97] Starting APIServiceRegistrationController
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888177    4395 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888201    4395 controller.go:78] Starting OpenAPI AggregationController
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888207    4395 customresource_discovery_controller.go:289] Starting DiscoveryController
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888232    4395 system_namespaces_controller.go:67] Starting system namespaces controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888256    4395 apf_controller.go:374] Starting API Priority and Fairness config controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888311    4395 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt::/var/lib/rancher/k3s/server/tls/client-auth-proxy.key"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888384    4395 available_controller.go:423] Starting AvailableConditionController
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888393    4395 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888412    4395 aggregator.go:163] waiting for initial CRD sync...
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888544    4395 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.888608    4395 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.892115    4395 controller.go:139] Starting OpenAPI controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.892149    4395 controller.go:87] Starting OpenAPI V3 controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.892168    4395 naming_controller.go:291] Starting NamingConditionController
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.892186    4395 establishing_controller.go:76] Starting EstablishingController
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.892209    4395 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.892228    4395 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.892245    4395 crd_finalizer.go:266] Starting CRDFinalizer
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.892269    4395 crdregistration_controller.go:111] Starting crd-autoregister controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.892278    4395 shared_informer.go:313] Waiting for caches to sync for crd-autoregister
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.978307    4395 shared_informer.go:320] Caches are synced for node_authorizer
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.979950    4395 shared_informer.go:320] Caches are synced for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.980012    4395 policy_source.go:224] refreshing policies
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.988454    4395 cache.go:39] Caches are synced for AvailableConditionController controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.988652    4395 handler.go:286] Adding GroupVersion k3s.cattle.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.988691    4395 handler.go:286] Adding GroupVersion helm.cattle.io v1 to ResourceManager
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.988811    4395 shared_informer.go:320] Caches are synced for configmaps
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.988927    4395 apf_controller.go:379] Running API Priority and Fairness config worker
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.988936    4395 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.989014    4395 shared_informer.go:320] Caches are synced for cluster_authentication_trust_controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.989057    4395 cache.go:39] Caches are synced for APIServiceRegistrationController controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.992810    4395 shared_informer.go:320] Caches are synced for crd-autoregister
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.998079    4395 handler_discovery.go:447] Starting ResourceDiscoveryManager
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.998798    4395 aggregator.go:165] initial CRD sync complete...
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.998923    4395 autoregister_controller.go:141] Starting autoregister controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.998991    4395 cache.go:32] Waiting for caches to sync for autoregister controller
Sep 27 16:09:12 debk3s k3s[4395]: I0927 16:09:12.999064    4395 cache.go:39] Caches are synced for autoregister controller
Sep 27 16:09:13 debk3s k3s[4395]: E0927 16:09:13.006441    4395 controller.go:102] Error removing old endpoints from kubernetes service: no API server IP addresses were listed in storage, refusing to erase all endpoints for the kubernetes Service
Sep 27 16:09:13 debk3s k3s[4395]: I0927 16:09:13.047111    4395 controller.go:615] quota admission added evaluator for: leases.coordination.k8s.io
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=info msg="Password verified locally for node debk3s"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=info msg="certificate CN=debk3s signed by CN=k3s-server-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:09:13 +0000 UTC"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=info msg="certificate CN=system:node:debk3s,O=system:nodes signed by CN=k3s-client-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:09:13 +0000 UTC"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=warning msg="Host resolv.conf includes loopback or multicast nameservers - kubelet will use autogenerated resolv.conf with nameserver 8.8.8.8"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=info msg="Module overlay was already loaded"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=info msg="Module nf_conntrack was already loaded"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=warning msg="Failed to load kernel module br_netfilter with modprobe"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=warning msg="Failed to load kernel module iptable_nat with modprobe"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=warning msg="Failed to load kernel module iptable_filter with modprobe"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=info msg="Set sysctl 'net/bridge/bridge-nf-call-iptables' to 1"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=error msg="Failed to set sysctl: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=error msg="Failed to set sysctl: open /proc/sys/net/netfilter/nf_conntrack_max: permission denied"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
Sep 27 16:09:13 debk3s k3s[4395]: time="2024-09-27T16:09:13Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
Sep 27 16:09:13 debk3s k3s[4395]: I0927 16:09:13.891627    4395 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Kube API server is now running"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="ETCD server is now running"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="k3s is up and running"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Waiting for cloud-controller-manager privileges to become available"
Sep 27 16:09:14 debk3s systemd[1]: Started k3s.service - Lightweight Kubernetes.
░░ Subject: A start job for unit k3s.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has finished successfully.
░░ 
░░ The job identifier is 1271.
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Creating k3s-supervisor event broadcaster"
Sep 27 16:09:14 debk3s k3s[4395]: W0927 16:09:14.398986    4395 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Applying CRD addons.k3s.cattle.io"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Applying CRD etcdsnapshotfiles.k3s.cattle.io"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Applying CRD helmcharts.helm.cattle.io"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Applying CRD helmchartconfigs.helm.cattle.io"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-25.0.3+up25.0.0.tgz"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-crd-25.0.3+up25.0.0.tgz"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/traefik.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/coredns.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/aggregated-metrics-reader.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-reader.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-service.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-deployment.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/resource-reader.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/rolebindings.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/runtimes.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/ccm.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/local-storage.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-delegator.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-apiservice.yaml"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Starting dynamiclistener CN filter node controller"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Tunnel server egress proxy mode: agent"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Creating deploy event broadcaster"
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.606497    4395 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="containerd is now running"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Creating k3s-cert-monitor event broadcaster"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Connecting to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=debk3s --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-ip=10.174.47.117,fd42:f085:b67c:18d1:216:3eff:fee7:36c3 --node-labels= --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/var/lib/rancher/k3s/agent/etc/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Handling backend connection request [debk3s]"
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=info msg="Remotedialer connected to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect"
Sep 27 16:09:14 debk3s k3s[4395]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.
Sep 27 16:09:14 debk3s k3s[4395]: W0927 16:09:14.680373    4395 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:14 debk3s k3s[4395]: W0927 16:09:14.680647    4395 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.681877    4395 server.go:479] "Kubelet version" kubeletVersion="v1.30.5+k3s1"
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.681898    4395 server.go:481] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 27 16:09:14 debk3s k3s[4395]: W0927 16:09:14.681943    4395 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:14 debk3s k3s[4395]: W0927 16:09:14.682018    4395 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.683806    4395 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt"
Sep 27 16:09:14 debk3s k3s[4395]: W0927 16:09:14.687381    4395 fs.go:216] stat failed on /dev/sdb2 with error: no such file or directory
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.694227    4395 controller.go:615] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.695928    4395 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="AppliedManifest" message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.697823    4395 controller.go:615] quota admission added evaluator for: addons.k3s.cattle.io
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.720868    4395 server.go:737] "--cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /"
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.720914    4395 server.go:805] "NoSwap is set due to memorySwapBehavior not specified" memorySwapBehavior="" FailSwapOn=false
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.721387    4395 container_manager_linux.go:265] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.721415    4395 container_manager_linux.go:270] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"debk3s","RuntimeCgroupsName":"","SystemCgroupsName":"","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":false,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":null,"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"ExperimentalMemoryManagerPolicy":"None","ExperimentalMemoryManagerReservedMemory":null,"PodPidsLimit":-1,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null}
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.721736    4395 topology_manager.go:138] "Creating topology manager with none policy"
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.721751    4395 container_manager_linux.go:301] "Creating device plugin manager"
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.721789    4395 state_mem.go:36] "Initialized new in-memory state store"
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.721878    4395 kubelet.go:401] "Attempting to sync node with API server"
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.721893    4395 kubelet.go:302] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests"
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.721919    4395 kubelet.go:313] "Adding apiserver pod source"
Sep 27 16:09:14 debk3s k3s[4395]: I0927 16:09:14.721929    4395 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 27 16:09:14 debk3s k3s[4395]: Error: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory
Sep 27 16:09:14 debk3s k3s[4395]: time="2024-09-27T16:09:14Z" level=error msg="kubelet exited: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory"
Sep 27 16:09:14 debk3s systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ An ExecStart= process belonging to unit k3s.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Sep 27 16:09:14 debk3s systemd[1]: k3s.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service has entered the 'failed' state with result 'exit-code'.
Sep 27 16:09:14 debk3s systemd[1]: k3s.service: Consumed 4.402s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service completed and consumed the indicated resources.
Sep 27 16:09:20 debk3s systemd[1]: k3s.service: Scheduled restart job, restart counter is at 17.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ Automatic restarting of the unit k3s.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Sep 27 16:09:20 debk3s systemd[1]: Stopped k3s.service - Lightweight Kubernetes.
░░ Subject: A stop job for unit k3s.service has finished
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A stop job for unit k3s.service has finished.
░░ 
░░ The job identifier is 1328 and the job result is done.
Sep 27 16:09:20 debk3s systemd[1]: k3s.service: Consumed 4.402s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service completed and consumed the indicated resources.
Sep 27 16:09:20 debk3s systemd[1]: Starting k3s.service - Lightweight Kubernetes...
░░ Subject: A start job for unit k3s.service has begun execution
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has begun execution.
░░ 
░░ The job identifier is 1328.
Sep 27 16:09:20 debk3s sh[4434]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Sep 27 16:09:20 debk3s (modprobe)[4436]: k3s.service: Executable /sbin/modprobe missing, skipping: No such file or directory
░░ Subject: Process /sbin/modprobe could not be executed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The process /sbin/modprobe could not be executed and failed.
░░ 
░░ The error number returned by this process is 2.
Sep 27 16:09:20 debk3s (modprobe)[4437]: k3s.service: Executable /sbin/modprobe missing, skipping: No such file or directory
░░ Subject: Process /sbin/modprobe could not be executed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The process /sbin/modprobe could not be executed and failed.
░░ 
░░ The error number returned by this process is 2.
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Starting k3s v1.30.5+k3s1 (9b586704)"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Configuring sqlite3 database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Database tables and indexes are up to date"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Kine available at unix://kine.sock"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Reconciling bootstrap data between datastore and disk"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --etcd-servers=unix://kine.sock --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Running kube-scheduler --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --leader-elect=false --profiling=false --secure-port=10259"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Waiting for API server to become available"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Running kube-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --configure-cloud-routes=false --controllers=*,tokencleaner,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --leader-elect=false --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=10257 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --use-service-account-credentials=true"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.388375    4438 options.go:221] external host was not specified, using 10.174.47.117
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/k3s/server/etc/cloud-config.yaml --cloud-provider=k3s --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route --feature-gates=CloudDualStackNodeIPs=true --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --leader-elect=false --leader-elect-resource-name=k3s-cloud-controller-manager --node-status-update-frequency=1m0s --profiling=false"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.389010    4438 server.go:156] Version: v1.30.5+k3s1
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.389042    4438 server.go:158] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Server node token is available at /var/lib/rancher/k3s/server/token"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="To join server node to cluster: k3s server -s https://10.174.47.117:6443 -t ${SERVER_NODE_TOKEN}"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Agent node token is available at /var/lib/rancher/k3s/server/agent-token"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="To join agent node to cluster: k3s agent -s https://10.174.47.117:6443 -t ${AGENT_NODE_TOKEN}"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml"
Sep 27 16:09:20 debk3s k3s[4438]: time="2024-09-27T16:09:20Z" level=info msg="Run: k3s kubectl"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.398889    4438 shared_informer.go:313] Waiting for caches to sync for node_authorizer
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.402670    4438 shared_informer.go:313] Waiting for caches to sync for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.406282    4438 plugins.go:157] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.406299    4438 plugins.go:160] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.406515    4438 instance.go:299] Using reconciler: lease
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.422379    4438 handler.go:286] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.422405    4438 genericapiserver.go:733] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.469392    4438 handler.go:286] Adding GroupVersion  v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.469581    4438 instance.go:696] API group "internal.apiserver.k8s.io" is not enabled, skipping.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.504779    4438 instance.go:696] API group "storagemigration.k8s.io" is not enabled, skipping.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.539878    4438 instance.go:696] API group "resource.k8s.io" is not enabled, skipping.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.549373    4438 handler.go:286] Adding GroupVersion authentication.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.549397    4438 genericapiserver.go:733] Skipping API authentication.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.549404    4438 genericapiserver.go:733] Skipping API authentication.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.549875    4438 handler.go:286] Adding GroupVersion authorization.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.549886    4438 genericapiserver.go:733] Skipping API authorization.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.550668    4438 handler.go:286] Adding GroupVersion autoscaling v2 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.551392    4438 handler.go:286] Adding GroupVersion autoscaling v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.551403    4438 genericapiserver.go:733] Skipping API autoscaling/v2beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.551410    4438 genericapiserver.go:733] Skipping API autoscaling/v2beta2 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.552705    4438 handler.go:286] Adding GroupVersion batch v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.552719    4438 genericapiserver.go:733] Skipping API batch/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.553402    4438 handler.go:286] Adding GroupVersion certificates.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.553475    4438 genericapiserver.go:733] Skipping API certificates.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.553484    4438 genericapiserver.go:733] Skipping API certificates.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.553914    4438 handler.go:286] Adding GroupVersion coordination.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.553925    4438 genericapiserver.go:733] Skipping API coordination.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.553984    4438 genericapiserver.go:733] Skipping API discovery.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.554429    4438 handler.go:286] Adding GroupVersion discovery.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.556276    4438 handler.go:286] Adding GroupVersion networking.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.556297    4438 genericapiserver.go:733] Skipping API networking.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.556306    4438 genericapiserver.go:733] Skipping API networking.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.557778    4438 handler.go:286] Adding GroupVersion node.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.557797    4438 genericapiserver.go:733] Skipping API node.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.557806    4438 genericapiserver.go:733] Skipping API node.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.558746    4438 handler.go:286] Adding GroupVersion policy v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.559145    4438 genericapiserver.go:733] Skipping API policy/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.573828    4438 handler.go:286] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.573850    4438 genericapiserver.go:733] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.573858    4438 genericapiserver.go:733] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.574284    4438 handler.go:286] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.574299    4438 genericapiserver.go:733] Skipping API scheduling.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.574306    4438 genericapiserver.go:733] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.575961    4438 handler.go:286] Adding GroupVersion storage.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.575982    4438 genericapiserver.go:733] Skipping API storage.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.575991    4438 genericapiserver.go:733] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.576823    4438 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.577880    4438 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.577896    4438 genericapiserver.go:733] Skipping API flowcontrol.apiserver.k8s.io/v1beta2 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.577916    4438 genericapiserver.go:733] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.581143    4438 handler.go:286] Adding GroupVersion apps v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.581168    4438 genericapiserver.go:733] Skipping API apps/v1beta2 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.581175    4438 genericapiserver.go:733] Skipping API apps/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.582555    4438 handler.go:286] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.582577    4438 genericapiserver.go:733] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.582585    4438 genericapiserver.go:733] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.583233    4438 handler.go:286] Adding GroupVersion events.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.583246    4438 genericapiserver.go:733] Skipping API events.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.586326    4438 handler.go:286] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager
Sep 27 16:09:20 debk3s k3s[4438]: W0927 16:09:20.586355    4438 genericapiserver.go:733] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.941368    4438 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.941368    4438 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.941606    4438 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.941762    4438 secure_serving.go:213] Serving securely on 127.0.0.1:6444
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.941810    4438 tlsconfig.go:240] "Starting DynamicServingCertificateController"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.941861    4438 controller.go:78] Starting OpenAPI AggregationController
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.941899    4438 aggregator.go:163] waiting for initial CRD sync...
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.941977    4438 controller.go:116] Starting legacy_token_tracking_controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.941987    4438 shared_informer.go:313] Waiting for caches to sync for configmaps
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942087    4438 customresource_discovery_controller.go:289] Starting DiscoveryController
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942128    4438 available_controller.go:423] Starting AvailableConditionController
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942135    4438 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942150    4438 controller.go:80] Starting OpenAPI V3 AggregationController
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942354    4438 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt::/var/lib/rancher/k3s/server/tls/client-auth-proxy.key"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942433    4438 system_namespaces_controller.go:67] Starting system namespaces controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942472    4438 apf_controller.go:374] Starting API Priority and Fairness config controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942728    4438 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942737    4438 shared_informer.go:313] Waiting for caches to sync for cluster_authentication_trust_controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942788    4438 apiservice_controller.go:97] Starting APIServiceRegistrationController
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942797    4438 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942828    4438 gc_controller.go:78] Starting apiserver lease garbage collector
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942876    4438 crdregistration_controller.go:111] Starting crd-autoregister controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942886    4438 shared_informer.go:313] Waiting for caches to sync for crd-autoregister
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.942944    4438 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.943004    4438 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.943087    4438 controller.go:139] Starting OpenAPI controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.943121    4438 controller.go:87] Starting OpenAPI V3 controller
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.943140    4438 naming_controller.go:291] Starting NamingConditionController
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.943163    4438 establishing_controller.go:76] Starting EstablishingController
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.943903    4438 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.943925    4438 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.943942    4438 crd_finalizer.go:266] Starting CRDFinalizer
Sep 27 16:09:20 debk3s k3s[4438]: I0927 16:09:20.998929    4438 shared_informer.go:320] Caches are synced for node_authorizer
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.003389    4438 shared_informer.go:320] Caches are synced for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.003536    4438 policy_source.go:224] refreshing policies
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.042970    4438 shared_informer.go:320] Caches are synced for crd-autoregister
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043016    4438 cache.go:39] Caches are synced for AvailableConditionController controller
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043188    4438 cache.go:39] Caches are synced for APIServiceRegistrationController controller
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043446    4438 shared_informer.go:320] Caches are synced for configmaps
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043458    4438 apf_controller.go:379] Running API Priority and Fairness config worker
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043469    4438 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043502    4438 handler.go:286] Adding GroupVersion k3s.cattle.io v1 to ResourceManager
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043542    4438 handler.go:286] Adding GroupVersion helm.cattle.io v1 to ResourceManager
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043549    4438 aggregator.go:165] initial CRD sync complete...
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043556    4438 autoregister_controller.go:141] Starting autoregister controller
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043562    4438 cache.go:32] Waiting for caches to sync for autoregister controller
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.043568    4438 cache.go:39] Caches are synced for autoregister controller
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.044632    4438 shared_informer.go:320] Caches are synced for cluster_authentication_trust_controller
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.077869    4438 handler_discovery.go:447] Starting ResourceDiscoveryManager
Sep 27 16:09:21 debk3s k3s[4438]: E0927 16:09:21.086523    4438 controller.go:102] Error removing old endpoints from kubernetes service: no API server IP addresses were listed in storage, refusing to erase all endpoints for the kubernetes Service
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.096532    4438 controller.go:615] quota admission added evaluator for: leases.coordination.k8s.io
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=info msg="Password verified locally for node debk3s"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=info msg="certificate CN=debk3s signed by CN=k3s-server-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:09:21 +0000 UTC"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=info msg="certificate CN=system:node:debk3s,O=system:nodes signed by CN=k3s-client-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:09:21 +0000 UTC"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=warning msg="Host resolv.conf includes loopback or multicast nameservers - kubelet will use autogenerated resolv.conf with nameserver 8.8.8.8"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=info msg="Module overlay was already loaded"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=info msg="Module nf_conntrack was already loaded"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=warning msg="Failed to load kernel module br_netfilter with modprobe"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=warning msg="Failed to load kernel module iptable_nat with modprobe"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=warning msg="Failed to load kernel module iptable_filter with modprobe"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=info msg="Set sysctl 'net/bridge/bridge-nf-call-iptables' to 1"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=error msg="Failed to set sysctl: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=error msg="Failed to set sysctl: open /proc/sys/net/netfilter/nf_conntrack_max: permission denied"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
Sep 27 16:09:21 debk3s k3s[4438]: time="2024-09-27T16:09:21Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
Sep 27 16:09:21 debk3s k3s[4438]: I0927 16:09:21.960920    4438 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Kube API server is now running"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="ETCD server is now running"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="k3s is up and running"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Waiting for cloud-controller-manager privileges to become available"
Sep 27 16:09:22 debk3s systemd[1]: Started k3s.service - Lightweight Kubernetes.
░░ Subject: A start job for unit k3s.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has finished successfully.
░░ 
░░ The job identifier is 1328.
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Creating k3s-supervisor event broadcaster"
Sep 27 16:09:22 debk3s k3s[4438]: W0927 16:09:22.400231    4438 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Applying CRD addons.k3s.cattle.io"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Applying CRD etcdsnapshotfiles.k3s.cattle.io"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Applying CRD helmcharts.helm.cattle.io"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Applying CRD helmchartconfigs.helm.cattle.io"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-25.0.3+up25.0.0.tgz"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-crd-25.0.3+up25.0.0.tgz"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/rolebindings.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/local-storage.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/aggregated-metrics-reader.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-reader.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-deployment.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-service.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/resource-reader.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/runtimes.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/traefik.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/ccm.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/coredns.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-delegator.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-apiservice.yaml"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Tunnel server egress proxy mode: agent"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Starting dynamiclistener CN filter node controller"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Creating deploy event broadcaster"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.609398    4438 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.636391    4438 controller.go:615] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.637991    4438 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="AppliedManifest" message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.639712    4438 controller.go:615] quota admission added evaluator for: addons.k3s.cattle.io
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.646663    4438 event.go:389] "Event occurred" object="kube-system/coredns" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\""
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="containerd is now running"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.695079    4438 controller.go:615] quota admission added evaluator for: deployments.apps
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Creating k3s-cert-monitor event broadcaster"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Connecting to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Running kubelet --address=0.0.0.0 --allowed-unsafe-sysctls=net.ipv4.ip_forward,net.ipv6.conf.all.forwarding --anonymous-auth=false --authentication-token-webhook=true --authorization-mode=Webhook --cgroup-driver=systemd --client-ca-file=/var/lib/rancher/k3s/agent/client-ca.crt --cloud-provider=external --cluster-dns=10.43.0.10 --cluster-domain=cluster.local --container-runtime-endpoint=unix:///run/k3s/containerd/containerd.sock --containerd=/run/k3s/containerd/containerd.sock --eviction-hard=imagefs.available<5%,nodefs.available<5% --eviction-minimum-reclaim=imagefs.available=10%,nodefs.available=10% --fail-swap-on=false --feature-gates=CloudDualStackNodeIPs=true --healthz-bind-address=127.0.0.1 --hostname-override=debk3s --kubeconfig=/var/lib/rancher/k3s/agent/kubelet.kubeconfig --node-ip=10.174.47.117,fd42:f085:b67c:18d1:216:3eff:fee7:36c3 --node-labels= --pod-manifest-path=/var/lib/rancher/k3s/agent/pod-manifests --read-only-port=0 --resolv-conf=/var/lib/rancher/k3s/agent/etc/resolv.conf --serialize-image-pulls=false --tls-cert-file=/var/lib/rancher/k3s/agent/serving-kubelet.crt --tls-private-key-file=/var/lib/rancher/k3s/agent/serving-kubelet.key"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Handling backend connection request [debk3s]"
Sep 27 16:09:22 debk3s k3s[4438]: Flag --containerd has been deprecated, This is a cadvisor flag that was mistakenly registered with the Kubelet. Due to legacy concerns, it will follow the standard CLI deprecation timeline before being removed.
Sep 27 16:09:22 debk3s k3s[4438]: W0927 16:09:22.707585    4438 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:22 debk3s k3s[4438]: W0927 16:09:22.707809    4438 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Remotedialer connected to proxy" url="wss://127.0.0.1:6443/v1-k3s/connect"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.709338    4438 server.go:479] "Kubelet version" kubeletVersion="v1.30.5+k3s1"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.709359    4438 server.go:481] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 27 16:09:22 debk3s k3s[4438]: W0927 16:09:22.709404    4438 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:22 debk3s k3s[4438]: W0927 16:09:22.709499    4438 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.714192    4438 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/agent/client-ca.crt"
Sep 27 16:09:22 debk3s k3s[4438]: W0927 16:09:22.719461    4438 fs.go:216] stat failed on /dev/sdb2 with error: no such file or directory
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.732177    4438 event.go:389] "Event occurred" object="kube-system/coredns" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="AppliedManifest" message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\""
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Creating helm-controller event broadcaster"
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=info msg="Starting /v1, Kind=Node controller"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.743903    4438 server.go:737] "--cgroups-per-qos enabled, but --cgroup-root was not specified.  defaulting to /"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.744021    4438 server.go:805] "NoSwap is set due to memorySwapBehavior not specified" memorySwapBehavior="" FailSwapOn=false
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.744496    4438 container_manager_linux.go:265] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.744590    4438 container_manager_linux.go:270] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"debk3s","RuntimeCgroupsName":"","SystemCgroupsName":"","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":false,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":null,"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"ExperimentalMemoryManagerPolicy":"None","ExperimentalMemoryManagerReservedMemory":null,"PodPidsLimit":-1,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null}
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.746038    4438 topology_manager.go:138] "Creating topology manager with none policy"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.746067    4438 container_manager_linux.go:301] "Creating device plugin manager"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.746122    4438 state_mem.go:36] "Initialized new in-memory state store"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.746273    4438 kubelet.go:401] "Attempting to sync node with API server"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.746291    4438 kubelet.go:302] "Adding static pod path" path="/var/lib/rancher/k3s/agent/pod-manifests"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.746317    4438 kubelet.go:313] "Adding apiserver pod source"
Sep 27 16:09:22 debk3s k3s[4438]: I0927 16:09:22.746337    4438 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 27 16:09:22 debk3s k3s[4438]: Error: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory
Sep 27 16:09:22 debk3s k3s[4438]: time="2024-09-27T16:09:22Z" level=error msg="kubelet exited: failed to run Kubelet: failed to create kubelet: open /dev/kmsg: no such file or directory"
Sep 27 16:09:22 debk3s systemd[1]: k3s.service: Main process exited, code=exited, status=1/FAILURE
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ An ExecStart= process belonging to unit k3s.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 1.
Sep 27 16:09:22 debk3s systemd[1]: k3s.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service has entered the 'failed' state with result 'exit-code'.
Sep 27 16:09:22 debk3s systemd[1]: k3s.service: Consumed 4.574s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service completed and consumed the indicated resources.
Sep 27 16:09:28 debk3s systemd[1]: k3s.service: Scheduled restart job, restart counter is at 18.
░░ Subject: Automatic restarting of a unit has been scheduled
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ Automatic restarting of the unit k3s.service has been scheduled, as the result for
░░ the configured Restart= setting for the unit.
Sep 27 16:09:28 debk3s systemd[1]: Stopped k3s.service - Lightweight Kubernetes.
░░ Subject: A stop job for unit k3s.service has finished
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A stop job for unit k3s.service has finished.
░░ 
░░ The job identifier is 1385 and the job result is done.
Sep 27 16:09:28 debk3s systemd[1]: k3s.service: Consumed 4.574s CPU time.
░░ Subject: Resources consumed by unit runtime
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The unit k3s.service completed and consumed the indicated resources.
Sep 27 16:09:28 debk3s systemd[1]: Starting k3s.service - Lightweight Kubernetes...
░░ Subject: A start job for unit k3s.service has begun execution
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has begun execution.
░░ 
░░ The job identifier is 1385.
Sep 27 16:09:28 debk3s sh[4469]: + /usr/bin/systemctl is-enabled --quiet nm-cloud-setup.service
Sep 27 16:09:28 debk3s (modprobe)[4471]: k3s.service: Executable /sbin/modprobe missing, skipping: No such file or directory
░░ Subject: Process /sbin/modprobe could not be executed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The process /sbin/modprobe could not be executed and failed.
░░ 
░░ The error number returned by this process is 2.
Sep 27 16:09:28 debk3s (modprobe)[4472]: k3s.service: Executable /sbin/modprobe missing, skipping: No such file or directory
░░ Subject: Process /sbin/modprobe could not be executed
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ The process /sbin/modprobe could not be executed and failed.
░░ 
░░ The error number returned by this process is 2.
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Starting k3s v1.30.5+k3s1 (9b586704)"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Configuring sqlite3 database connection pooling: maxIdleConns=2, maxOpenConns=0, connMaxLifetime=0s"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Configuring database table schema and indexes, this may take a moment..."
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Database tables and indexes are up to date"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Kine available at unix://kine.sock"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Reconciling bootstrap data between datastore and disk"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Running kube-apiserver --advertise-port=6443 --allow-privileged=true --anonymous-auth=false --api-audiences=https://kubernetes.default.svc.cluster.local,k3s --authorization-mode=Node,RBAC --bind-address=127.0.0.1 --cert-dir=/var/lib/rancher/k3s/server/tls/temporary-certs --client-ca-file=/var/lib/rancher/k3s/server/tls/client-ca.crt --egress-selector-config-file=/var/lib/rancher/k3s/server/etc/egress-selector-config.yaml --enable-admission-plugins=NodeRestriction --enable-aggregator-routing=true --enable-bootstrap-token-auth=true --etcd-servers=unix://kine.sock --kubelet-certificate-authority=/var/lib/rancher/k3s/server/tls/server-ca.crt --kubelet-client-certificate=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.crt --kubelet-client-key=/var/lib/rancher/k3s/server/tls/client-kube-apiserver.key --kubelet-preferred-address-types=InternalIP,ExternalIP,Hostname --profiling=false --proxy-client-cert-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt --proxy-client-key-file=/var/lib/rancher/k3s/server/tls/client-auth-proxy.key --requestheader-allowed-names=system:auth-proxy --requestheader-client-ca-file=/var/lib/rancher/k3s/server/tls/request-header-ca.crt --requestheader-extra-headers-prefix=X-Remote-Extra- --requestheader-group-headers=X-Remote-Group --requestheader-username-headers=X-Remote-User --secure-port=6444 --service-account-issuer=https://kubernetes.default.svc.cluster.local --service-account-key-file=/var/lib/rancher/k3s/server/tls/service.key --service-account-signing-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --service-node-port-range=30000-32767 --storage-backend=etcd3 --tls-cert-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt --tls-cipher-suites=TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305 --tls-private-key-file=/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Running kube-scheduler --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --bind-address=127.0.0.1 --kubeconfig=/var/lib/rancher/k3s/server/cred/scheduler.kubeconfig --leader-elect=false --profiling=false --secure-port=10259"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Waiting for API server to become available"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Running kube-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --bind-address=127.0.0.1 --cluster-cidr=10.42.0.0/16 --cluster-signing-kube-apiserver-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kube-apiserver-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-client-cert-file=/var/lib/rancher/k3s/server/tls/client-ca.nochain.crt --cluster-signing-kubelet-client-key-file=/var/lib/rancher/k3s/server/tls/client-ca.key --cluster-signing-kubelet-serving-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-kubelet-serving-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --cluster-signing-legacy-unknown-cert-file=/var/lib/rancher/k3s/server/tls/server-ca.nochain.crt --cluster-signing-legacy-unknown-key-file=/var/lib/rancher/k3s/server/tls/server-ca.key --configure-cloud-routes=false --controllers=*,tokencleaner,-service,-route,-cloud-node-lifecycle --kubeconfig=/var/lib/rancher/k3s/server/cred/controller.kubeconfig --leader-elect=false --profiling=false --root-ca-file=/var/lib/rancher/k3s/server/tls/server-ca.crt --secure-port=10257 --service-account-private-key-file=/var/lib/rancher/k3s/server/tls/service.current.key --service-cluster-ip-range=10.43.0.0/16 --use-service-account-credentials=true"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.356493    4473 options.go:221] external host was not specified, using 10.174.47.117
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Running cloud-controller-manager --allocate-node-cidrs=true --authentication-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --authorization-kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --bind-address=127.0.0.1 --cloud-config=/var/lib/rancher/k3s/server/etc/cloud-config.yaml --cloud-provider=k3s --cluster-cidr=10.42.0.0/16 --configure-cloud-routes=false --controllers=*,-route --feature-gates=CloudDualStackNodeIPs=true --kubeconfig=/var/lib/rancher/k3s/server/cred/cloud-controller.kubeconfig --leader-elect=false --leader-elect-resource-name=k3s-cloud-controller-manager --node-status-update-frequency=1m0s --profiling=false"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.357089    4473 server.go:156] Version: v1.30.5+k3s1
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.357112    4473 server.go:158] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Server node token is available at /var/lib/rancher/k3s/server/token"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="To join server node to cluster: k3s server -s https://10.174.47.117:6443 -t ${SERVER_NODE_TOKEN}"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Agent node token is available at /var/lib/rancher/k3s/server/agent-token"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="To join agent node to cluster: k3s agent -s https://10.174.47.117:6443 -t ${AGENT_NODE_TOKEN}"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Wrote kubeconfig /etc/rancher/k3s/k3s.yaml"
Sep 27 16:09:28 debk3s k3s[4473]: time="2024-09-27T16:09:28Z" level=info msg="Run: k3s kubectl"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.373755    4473 shared_informer.go:313] Waiting for caches to sync for node_authorizer
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.379237    4473 shared_informer.go:313] Waiting for caches to sync for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.382749    4473 plugins.go:157] Loaded 12 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,NodeRestriction,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,StorageObjectInUseProtection,RuntimeClass,DefaultIngressClass,MutatingAdmissionWebhook.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.382766    4473 plugins.go:160] Loaded 13 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,PodSecurity,Priority,PersistentVolumeClaimResize,RuntimeClass,CertificateApproval,CertificateSigning,ClusterTrustBundleAttest,CertificateSubjectRestriction,ValidatingAdmissionPolicy,ValidatingAdmissionWebhook,ResourceQuota.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.382952    4473 instance.go:299] Using reconciler: lease
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.389180    4473 handler.go:286] Adding GroupVersion apiextensions.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.389199    4473 genericapiserver.go:733] Skipping API apiextensions.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.441035    4473 handler.go:286] Adding GroupVersion  v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.441240    4473 instance.go:696] API group "internal.apiserver.k8s.io" is not enabled, skipping.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.479905    4473 instance.go:696] API group "storagemigration.k8s.io" is not enabled, skipping.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.515418    4473 instance.go:696] API group "resource.k8s.io" is not enabled, skipping.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.525246    4473 handler.go:286] Adding GroupVersion authentication.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.525273    4473 genericapiserver.go:733] Skipping API authentication.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.525280    4473 genericapiserver.go:733] Skipping API authentication.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.525665    4473 handler.go:286] Adding GroupVersion authorization.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.525675    4473 genericapiserver.go:733] Skipping API authorization.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.526443    4473 handler.go:286] Adding GroupVersion autoscaling v2 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.527093    4473 handler.go:286] Adding GroupVersion autoscaling v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.527103    4473 genericapiserver.go:733] Skipping API autoscaling/v2beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.527110    4473 genericapiserver.go:733] Skipping API autoscaling/v2beta2 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.528478    4473 handler.go:286] Adding GroupVersion batch v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.528495    4473 genericapiserver.go:733] Skipping API batch/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.529220    4473 handler.go:286] Adding GroupVersion certificates.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.529232    4473 genericapiserver.go:733] Skipping API certificates.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.529240    4473 genericapiserver.go:733] Skipping API certificates.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.529731    4473 handler.go:286] Adding GroupVersion coordination.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.529742    4473 genericapiserver.go:733] Skipping API coordination.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.529788    4473 genericapiserver.go:733] Skipping API discovery.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.530268    4473 handler.go:286] Adding GroupVersion discovery.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.531589    4473 handler.go:286] Adding GroupVersion networking.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.531600    4473 genericapiserver.go:733] Skipping API networking.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.531607    4473 genericapiserver.go:733] Skipping API networking.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.532008    4473 handler.go:286] Adding GroupVersion node.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.532020    4473 genericapiserver.go:733] Skipping API node.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.532028    4473 genericapiserver.go:733] Skipping API node.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.532978    4473 handler.go:286] Adding GroupVersion policy v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.532994    4473 genericapiserver.go:733] Skipping API policy/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.550306    4473 handler.go:286] Adding GroupVersion rbac.authorization.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.550336    4473 genericapiserver.go:733] Skipping API rbac.authorization.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.550345    4473 genericapiserver.go:733] Skipping API rbac.authorization.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.550818    4473 handler.go:286] Adding GroupVersion scheduling.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.550836    4473 genericapiserver.go:733] Skipping API scheduling.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.550843    4473 genericapiserver.go:733] Skipping API scheduling.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.552438    4473 handler.go:286] Adding GroupVersion storage.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.552457    4473 genericapiserver.go:733] Skipping API storage.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.552464    4473 genericapiserver.go:733] Skipping API storage.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.553215    4473 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.553901    4473 handler.go:286] Adding GroupVersion flowcontrol.apiserver.k8s.io v1beta3 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.553912    4473 genericapiserver.go:733] Skipping API flowcontrol.apiserver.k8s.io/v1beta2 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.553917    4473 genericapiserver.go:733] Skipping API flowcontrol.apiserver.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.556311    4473 handler.go:286] Adding GroupVersion apps v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.556332    4473 genericapiserver.go:733] Skipping API apps/v1beta2 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.556338    4473 genericapiserver.go:733] Skipping API apps/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.557677    4473 handler.go:286] Adding GroupVersion admissionregistration.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.557690    4473 genericapiserver.go:733] Skipping API admissionregistration.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.557696    4473 genericapiserver.go:733] Skipping API admissionregistration.k8s.io/v1alpha1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.558163    4473 handler.go:286] Adding GroupVersion events.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.558173    4473 genericapiserver.go:733] Skipping API events.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.560786    4473 handler.go:286] Adding GroupVersion apiregistration.k8s.io v1 to ResourceManager
Sep 27 16:09:28 debk3s k3s[4473]: W0927 16:09:28.560806    4473 genericapiserver.go:733] Skipping API apiregistration.k8s.io/v1beta1 because it has no resources.
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.914866    4473 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.914947    4473 dynamic_serving_content.go:132] "Starting controller" name="serving-cert::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.crt::/var/lib/rancher/k3s/server/tls/serving-kube-apiserver.key"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.914866    4473 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915417    4473 secure_serving.go:213] Serving securely on 127.0.0.1:6444
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915460    4473 tlsconfig.go:240] "Starting DynamicServingCertificateController"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915549    4473 controller.go:80] Starting OpenAPI V3 AggregationController
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915577    4473 system_namespaces_controller.go:67] Starting system namespaces controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915590    4473 apf_controller.go:374] Starting API Priority and Fairness config controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915690    4473 cluster_authentication_trust_controller.go:440] Starting cluster_authentication_trust_controller controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915700    4473 shared_informer.go:313] Waiting for caches to sync for cluster_authentication_trust_controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915844    4473 gc_controller.go:78] Starting apiserver lease garbage collector
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915864    4473 aggregator.go:163] waiting for initial CRD sync...
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915942    4473 available_controller.go:423] Starting AvailableConditionController
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915952    4473 cache.go:32] Waiting for caches to sync for AvailableConditionController controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.915992    4473 apiservice_controller.go:97] Starting APIServiceRegistrationController
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916000    4473 cache.go:32] Waiting for caches to sync for APIServiceRegistrationController controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916016    4473 controller.go:78] Starting OpenAPI AggregationController
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916019    4473 crdregistration_controller.go:111] Starting crd-autoregister controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916028    4473 shared_informer.go:313] Waiting for caches to sync for crd-autoregister
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916023    4473 controller.go:116] Starting legacy_token_tracking_controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916047    4473 shared_informer.go:313] Waiting for caches to sync for configmaps
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916067    4473 dynamic_cafile_content.go:157] "Starting controller" name="client-ca-bundle::/var/lib/rancher/k3s/server/tls/client-ca.crt"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916147    4473 dynamic_cafile_content.go:157] "Starting controller" name="request-header::/var/lib/rancher/k3s/server/tls/request-header-ca.crt"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916196    4473 customresource_discovery_controller.go:289] Starting DiscoveryController
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916235    4473 controller.go:139] Starting OpenAPI controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916259    4473 controller.go:87] Starting OpenAPI V3 controller
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916278    4473 naming_controller.go:291] Starting NamingConditionController
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916295    4473 establishing_controller.go:76] Starting EstablishingController
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916316    4473 nonstructuralschema_controller.go:192] Starting NonStructuralSchemaConditionController
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916335    4473 apiapproval_controller.go:186] Starting KubernetesAPIApprovalPolicyConformantConditionController
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916353    4473 crd_finalizer.go:266] Starting CRDFinalizer
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.916411    4473 dynamic_serving_content.go:132] "Starting controller" name="aggregator-proxy-cert::/var/lib/rancher/k3s/server/tls/client-auth-proxy.crt::/var/lib/rancher/k3s/server/tls/client-auth-proxy.key"
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.975298    4473 shared_informer.go:320] Caches are synced for node_authorizer
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.980263    4473 shared_informer.go:320] Caches are synced for *generic.policySource[*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicy,*k8s.io/api/admissionregistration/v1.ValidatingAdmissionPolicyBinding,k8s.io/apiserver/pkg/admission/plugin/policy/validating.Validator]
Sep 27 16:09:28 debk3s k3s[4473]: I0927 16:09:28.980304    4473 policy_source.go:224] refreshing policies
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.016501    4473 shared_informer.go:320] Caches are synced for configmaps
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.016724    4473 shared_informer.go:320] Caches are synced for crd-autoregister
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.016533    4473 cache.go:39] Caches are synced for APIServiceRegistrationController controller
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.017249    4473 handler.go:286] Adding GroupVersion k3s.cattle.io v1 to ResourceManager
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.017298    4473 handler.go:286] Adding GroupVersion helm.cattle.io v1 to ResourceManager
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.020092    4473 cache.go:39] Caches are synced for AvailableConditionController controller
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.020617    4473 shared_informer.go:320] Caches are synced for cluster_authentication_trust_controller
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.026775    4473 aggregator.go:165] initial CRD sync complete...
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.026806    4473 autoregister_controller.go:141] Starting autoregister controller
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.026814    4473 cache.go:32] Waiting for caches to sync for autoregister controller
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.026825    4473 cache.go:39] Caches are synced for autoregister controller
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.016672    4473 apf_controller.go:379] Running API Priority and Fairness config worker
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.026970    4473 apf_controller.go:382] Running API Priority and Fairness periodic rebalancing process
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.026996    4473 handler_discovery.go:447] Starting ResourceDiscoveryManager
Sep 27 16:09:29 debk3s k3s[4473]: E0927 16:09:29.033833    4473 controller.go:102] Error removing old endpoints from kubernetes service: no API server IP addresses were listed in storage, refusing to erase all endpoints for the kubernetes Service
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.037018    4473 controller.go:615] quota admission added evaluator for: leases.coordination.k8s.io
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=info msg="Password verified locally for node debk3s"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=info msg="certificate CN=debk3s signed by CN=k3s-server-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:09:29 +0000 UTC"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=info msg="certificate CN=system:node:debk3s,O=system:nodes signed by CN=k3s-client-ca@1727453222: notBefore=2024-09-27 16:07:02 +0000 UTC notAfter=2025-09-27 16:09:29 +0000 UTC"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=warning msg="Host resolv.conf includes loopback or multicast nameservers - kubelet will use autogenerated resolv.conf with nameserver 8.8.8.8"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=info msg="Module overlay was already loaded"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=info msg="Module nf_conntrack was already loaded"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=warning msg="Failed to load kernel module br_netfilter with modprobe"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=warning msg="Failed to load kernel module iptable_nat with modprobe"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=warning msg="Failed to load kernel module iptable_filter with modprobe"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=info msg="Set sysctl 'net/bridge/bridge-nf-call-iptables' to 1"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=error msg="Failed to set sysctl: open /proc/sys/net/bridge/bridge-nf-call-iptables: no such file or directory"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=info msg="Set sysctl 'net/netfilter/nf_conntrack_max' to 131072"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=error msg="Failed to set sysctl: open /proc/sys/net/netfilter/nf_conntrack_max: permission denied"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=info msg="Logging containerd to /var/lib/rancher/k3s/agent/containerd/containerd.log"
Sep 27 16:09:29 debk3s k3s[4473]: time="2024-09-27T16:09:29Z" level=info msg="Running containerd -c /var/lib/rancher/k3s/agent/etc/containerd/config.toml -a /run/k3s/containerd/containerd.sock --state /run/k3s/containerd --root /var/lib/rancher/k3s/agent/containerd"
Sep 27 16:09:29 debk3s k3s[4473]: I0927 16:09:29.939148    4473 storage_scheduling.go:111] all system priority classes are created successfully or already exist.
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Waiting for cloud-controller-manager privileges to become available"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Kube API server is now running"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="ETCD server is now running"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="k3s is up and running"
Sep 27 16:09:30 debk3s systemd[1]: Started k3s.service - Lightweight Kubernetes.
░░ Subject: A start job for unit k3s.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░ 
░░ A start job for unit k3s.service has finished successfully.
░░ 
░░ The job identifier is 1385.
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Creating k3s-supervisor event broadcaster"
Sep 27 16:09:30 debk3s k3s[4473]: W0927 16:09:30.392541    4473 feature_gate.go:246] Setting GA feature gate CloudDualStackNodeIPs=true. It will be removed in a future release.
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Applying CRD addons.k3s.cattle.io"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Applying CRD etcdsnapshotfiles.k3s.cattle.io"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Applying CRD helmcharts.helm.cattle.io"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Applying CRD helmchartconfigs.helm.cattle.io"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-25.0.3+up25.0.0.tgz"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing static file: /var/lib/rancher/k3s/server/static/charts/traefik-crd-25.0.3+up25.0.0.tgz"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/ccm.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-delegator.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-deployment.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-server-service.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/resource-reader.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/rolebindings.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/runtimes.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/coredns.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/local-storage.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/aggregated-metrics-reader.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/auth-reader.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/metrics-server/metrics-apiservice.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Writing manifest: /var/lib/rancher/k3s/server/manifests/traefik.yaml"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Tunnel server egress proxy mode: agent"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Starting dynamiclistener CN filter node controller"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Starting k3s.cattle.io/v1, Kind=Addon controller"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Creating deploy event broadcaster"
Sep 27 16:09:30 debk3s k3s[4473]: I0927 16:09:30.612881    4473 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:09:30 debk3s k3s[4473]: I0927 16:09:30.673108    4473 controller.go:615] quota admission added evaluator for: rolebindings.rbac.authorization.k8s.io
Sep 27 16:09:30 debk3s k3s[4473]: I0927 16:09:30.675054    4473 event.go:389] "Event occurred" object="kube-system/ccm" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="AppliedManifest" message="Applied manifest at \"/var/lib/rancher/k3s/server/manifests/ccm.yaml\""
Sep 27 16:09:30 debk3s k3s[4473]: I0927 16:09:30.677590    4473 controller.go:615] quota admission added evaluator for: addons.k3s.cattle.io
Sep 27 16:09:30 debk3s k3s[4473]: I0927 16:09:30.688314    4473 event.go:389] "Event occurred" object="kube-system/coredns" fieldPath="" kind="Addon" apiVersion="k3s.cattle.io/v1" type="Normal" reason="ApplyingManifest" message="Applying manifest at \"/var/lib/rancher/k3s/server/manifests/coredns.yaml\""
Sep 27 16:09:30 debk3s k3s[4473]: I0927 16:09:30.711282    4473 serving.go:387] Generated self-signed cert in-memory
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Creating helm-controller event broadcaster"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Starting /v1, Kind=Node controller"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Waiting for control-plane node debk3s startup: nodes \"debk3s\" not found"
Sep 27 16:09:30 debk3s k3s[4473]: time="2024-09-27T16:09:30Z" level=info msg="Cluster dns configmap already exists"
Sep 27 16:09:30 debk3s k3s[4473]: I0927 16:09:30.738226    4473 controller.go:615] quota admission added evaluator for: deployments.apps

What is the correct way to do this?
Thank you.

bpedersen2 · September 30, 2024, 10:02am

These resources consumed massages indicate you are running out of cpu/memory for k3s.

Topic		Replies	Views
Kubectl cannot reach API server k3s, k3OS, and k3d	0	2982	March 3, 2023
Running k3s on existing runtime k3s, k3OS, and k3d	0	605	March 22, 2022
K3s with etcd: nodes cannot connect to first node k3s, k3OS, and k3d	1	1124	October 15, 2020
K3s Release - v0.5.0 Announcements	1	1214	May 6, 2019
Localhost:8080 was refused k3s, k3OS, and k3d	10	16703	June 11, 2024

Run k3s in Incus container

Related topics