In your air-gapped env (three baremetal servers, Centos 7) we have setup our private Docker repository and loaded images on all three nodes as per Rancher scripts and instructions.
We are stuck on deploying RKE :
These are the logs we are getting :
NFO[0000] Running RKE version: v1.0.8
INFO[0000] Initiating Kubernetes cluster
INFO[0000] [certificates] Generating admin certificates and kubeconfig
INFO[0000] Successfully Deployed state file at [./cluster.rkestate]
INFO[0000] Building Kubernetes cluster
INFO[0000] [dialer] Setup tunnel for host [172.16.100.150]
INFO[0000] [dialer] Setup tunnel for host [172.16.100.149]
INFO[0000] [dialer] Setup tunnel for host [172.16.100.148]
WARN[0000] Failed to set up SSH tunneling for host [172.16.100.148]: Can’t retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access node with address [172.16.100.148:22] using SSH. Please check if you are able to SSH to the node using the specified SSH Private Key and if you have configured the correct SSH username. Error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
WARN[0000] Failed to set up SSH tunneling for host [172.16.100.150]: Can’t retrieve Docker Info: error during connect: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.24/info: Unable to access node with address [172.16.100.150:22] using SSH. Please check if you are able to SSH to the node using the specified SSH Private Key and if you have configured the correct SSH username. Error: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
WARN[0000] Removing host [172.16.100.148] from node lists
WARN[0000] Removing host [172.16.100.150] from node lists
INFO[0000] [network] No hosts added existing cluster, skipping port check
INFO[0000] [certificates] Deploying kubernetes certificates to Cluster nodes
INFO[0000] Checking if container [cert-deployer] is running on host [172.16.100.149], try #1
INFO[0000] Image [rancher/rancher/rke-tools:v0.1.56] exists on host [172.16.100.149]
INFO[0000] Starting container [cert-deployer] on host [172.16.100.149], try #1
INFO[0001] Checking if container [cert-deployer] is running on host [172.16.100.149], try #1
INFO[0006] Checking if container [cert-deployer] is running on host [172.16.100.149], try #1
INFO[0006] Removing container [cert-deployer] on host [172.16.100.149], try #1
INFO[0006] [reconcile] Rebuilding and updating local kube config
INFO[0006] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml]
INFO[0006] [reconcile] host [172.16.100.149] is active master on the cluster
INFO[0006] [certificates] Successfully deployed kubernetes certificates to Cluster nodes
INFO[0006] [reconcile] Reconciling cluster state
INFO[0006] [reconcile] Check etcd hosts to be deleted
INFO[0006] [reconcile] Check etcd hosts to be added
INFO[0006] [reconcile] Rebuilding and updating local kube config
INFO[0006] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml]
INFO[0006] [reconcile] host [172.16.100.149] is active master on the cluster
INFO[0006] [reconcile] Reconciled cluster state successfully
INFO[0006] Pre-pulling kubernetes images
INFO[0006] Image [rancher/rancher/hyperkube:v1.17.5-rancher1] exists on host [172.16.100.149]
INFO[0006] Kubernetes images pulled successfully
INFO[0006] [etcd] Building up etcd plane…
INFO[0006] [etcd] Successfully started etcd plane… Checking etcd cluster health
INFO[0006] [authz] Creating rke-job-deployer ServiceAccount
INFO[0006] [authz] rke-job-deployer ServiceAccount created successfully
INFO[0006] [authz] Creating system:node ClusterRoleBinding
INFO[0006] [authz] system:node ClusterRoleBinding created successfully
INFO[0006] [authz] Creating kube-apiserver proxy ClusterRole and ClusterRoleBinding
INFO[0006] [authz] kube-apiserver proxy ClusterRole and ClusterRoleBinding created successfully
INFO[0006] Successfully Deployed state file at [./cluster.rkestate]
INFO[0006] [state] Saving full cluster state to Kubernetes
INFO[0006] [state] Successfully Saved full cluster state to Kubernetes ConfigMap: cluster-state
INFO[0006] [worker] Building up Worker Plane…
INFO[0006] [worker] Successfully started Worker Plane…
INFO[0006] Image [rancher/rancher/rke-tools:v0.1.56] exists on host [172.16.100.149]
INFO[0006] Starting container [rke-log-cleaner] on host [172.16.100.149], try #1
INFO[0007] [cleanup] Successfully started [rke-log-cleaner] container on host [172.16.100.149]
INFO[0007] Removing container [rke-log-cleaner] on host [172.16.100.149], try #1
INFO[0007] [remove/rke-log-cleaner] Successfully removed container on host [172.16.100.149]
INFO[0007] [sync] Syncing nodes Labels and Taints
INFO[0007] [sync] Successfully synced nodes Labels and Taints
INFO[0007] [network] Setting up network plugin: canal
INFO[0007] [addons] Saving ConfigMap for addon rke-network-plugin to Kubernetes
INFO[0007] [addons] Successfully saved ConfigMap for addon rke-network-plugin to Kubernetes
INFO[0007] [addons] Executing deploy job rke-network-plugin
FATA[0072] Failed to get job complete status for job rke-network-plugin-deploy-job in namespace kube-system