Stuck at "Waiting for API to be available"

Rancher
1

I am trying to import an existing Kubernetes cluster (which was set up by using this installation instructions).

Basic overview:

  • CentOS 8
  • Kubernetes 1.22
  • CRI-O 1.22
  • Rancher 2.5.11

Unfortunately, Rancher is stuck with the message “Waiting for API to be available”. By taking a closer look at the API configuration (which, I presume, was created from information reported back by the cattle cluster agent), I kind of understand why. The API endpoint is set to “https://10.96.0.1:443”. This is the internal IP of the “kubernetes” service, so of course the Rancher server can’t connect to that.

$ kubectl get service --all-namespaces
NAMESPACE          NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
calico-apiserver   calico-api                        ClusterIP   10.98.207.137    <none>        443/TCP                  112m
calico-system      calico-kube-controllers-metrics   ClusterIP   10.97.0.222      <none>        9094/TCP                 114m
calico-system      calico-typha                      ClusterIP   10.102.202.143   <none>        5473/TCP                 115m
default            kubernetes                        ClusterIP   10.96.0.1        <none>        443/TCP                  59d
kube-system        kube-dns                          ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   59d

What I don’t understand is, why that IP was reported back to Rancher as API endpoint. Shouldn’t this be the same address, which is reported by the kubectl config view command and which is the actual physical IP adress of my kubernetes CTL node?

$ kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://172.20.37.107:6443
  name: kubernetes
[....]

$ kubectl cluster-info
Kubernetes control plane is running at https://172.20.37.107:6443
CoreDNS is running at https://172.20.37.107:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
[...]

$ ifconfig
[...]
ens192: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 172.20.37.107  netmask 255.255.255.0  broadcast 172.20.37.255
[...]

What am I missing? How can successfully “connect” my Kubernetes cluster with Rancher?

2

We saw this with k8s 1.21. Dropped back to k8s 1.20 and that resolved “Waiting for API to be available”

3

Found this in the Rancher release notes:
Importing a Kubernetes v1.21 cluster might not work properly. We are planning to add support for Kubernetes v1.21 in the future.

4

After reading this post , I got rid of K8s V1.21 after it was stuck at “Waiting for API to be available” . Based on the suggestion, retried the setup after thoroughly cleaning all docker residue from the nodes [Rancher V2.6.3 with K8 1.20.13] now again it seems to be stuck at “Waiting for API to be available” . Please help .

image
image1820×925 24.9 KB

5

Actually I realized some searching that “Waiting for API to be available” usually points some misconfiguration somewhere. So I decided to tear down all the nodes and then did the following :

  1. Prepared the nodes based on their roles [which I think is what I underestimated in the previous occasion]
  2. Ran the RKE binary 1.2.13 with its corresponding K8s 1.20 version which it supports.
  3. Boom ! - It worked like a charm and then imported that into the UI…

I will post a detailed documentation what I exactly did , but thats for later. Thanks

6

is there any update for this issue, i am getting same error for AKS Cluster.

7

image
image2420×618 38.9 KB

Same, I can’t seem to provision AKS clusters using rancher. I let it provision over night and it just sits on “Waiting for API to become available”. I am still relatively new to rancher so I could be doing something wrong.

Any help on this would be greatly appreciated.