Rancher on GKE Waiting for API to be available

Deploy Rancher multiple times on hosted k8s GKE using helm
But having trouble trying to create or import other clusters, always stuck on Waiting for API to be available

currently running rancher version: 2.6.3
on a gke 1.20.11-gke.1801

helm values files and set

value files
--------
debug: false

hostname: rc.google-test.domain.example
ingress:
  extraAnnotations:
    nginx.org/websocket-services: rancher
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
  #   cert-manager.io/cluster-issuer: "acme-clusterissuer-stage"
  #   acme.cert-manager.io/http01-ingress-class: "nginx"


  tls:
    source: letsEncrypt

letsEncrypt:
  email: cloud@sdomain.example
  environment: staging
  # ingress:
  #   class: "nginx"

replicas: 2
bootstrapPassword: "cxxxxxx"

------------

helm upgrade --install rancher rancher-latest/rancher \
  --namespace cattle-system \
  --create-namespace \
  --wait \
  --set ingress.tls.source=letsEncrypt \
  --set letsEncrypt.email=cloud@sdomain.example \
  --set letsEncrypt.environment=staging \
  --set replicas=3 \
  --version 2.6.3 --values ../rancher-lab/values.yaml

logs

01/18 10:18:38 [ERROR] error syncing '_all_': handler user-controllers-controller: failed to start user controllers for cluster c-klbc2: ClusterUnavailable 503: cluster not found, requeuing
2022/01/18 10:18:38 [ERROR] error syncing '_all_': handler user-controllers-controller: failed to start user controllers for cluster c-klbc2: ClusterUnavailable 503: cluster not found, requeuing
2022/01/18 10:18:38 [ERROR] error syncing '_all_': handler user-controllers-controller: failed to start user controllers for cluster c-klbc2: ClusterUnavailable 503: cluster not found, requeuing
2022/01/18 10:18:38 [ERROR] error syncing '_all_': handler user-controllers-controller: failed to start user controllers for cluster c-klbc2: ClusterUnavailable 503: cluster not found, requeuing
2022/01/18 10:18:38 [ERROR] error syncing '_all_': handler user-controllers-controller: failed to start user controllers for cluster c-klbc2: ClusterUnavailable 503: cluster not found, requeuing
2022/01/18 10:18:39 [INFO] Starting cluster controllers for c-klbc2
2022/01/18 10:18:40 [ERROR] defaultSvcAccountHandler: Sync: error handling default ServiceAccount of namespace key=cattle-impersonation-system, err=Operation cannot be fulfilled on namespaces "cattle-impersonation-system": the object has been modified; please apply your changes to the latest version and try again
2022/01/18 10:33:14 [ERROR] Error during subscribe websocket: close sent

What am I missing so I can deploy new or import gke or eks clusters

1 Like

maybe firewall blocked some of your ports that rancher or kubernetes uses.
check the ports requirements in documents.