Has anyone got the rke via the command line working with AWS and EC2? As I can’t! 
I run ‘rke config’ and enter what I think are the right values.
I then run ‘rke up’, but just hangs on the ‘[healthcheck]’ part.
Thanks very much for any replies…
Please supply the cluster.yml and full (debug) log to see what the values are (redacted if sensitive) and what the log exactly indicates.
1 Like
# If you intened to deploy Kubernetes in an air-gapped environment,
# please consult the documentation on how to configure custom RKE images.
nodes:
- address: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
port: "22"
internal_address: 10.0.1.145
role:
- controlplane
- worker
- etcd
hostname_override: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
user: ubuntu
docker_socket: /var/run/docker.sock
ssh_key: ""
ssh_key_path: keys/rke-ec2-example-summary-alien-ec2.pem
ssh_cert: ""
ssh_cert_path: ""
labels: {}
taints: []
services:
etcd:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
external_urls: []
ca_cert: ""
cert: ""
key: ""
path: ""
uid: 0
gid: 0
snapshot: null
retention: ""
creation: ""
backup_config: null
kube-api:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
service_cluster_ip_range: 10.43.0.0/16
service_node_port_range: ""
pod_security_policy: false
always_pull_images: false
secrets_encryption_config: null
audit_log: null
admission_configuration: null
event_rate_limit: null
kube-controller:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
cluster_cidr: 10.42.0.0/16
service_cluster_ip_range: 10.43.0.0/16
scheduler:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
kubelet:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
cluster_domain: cluster.local
infra_container_image: ""
cluster_dns_server: 10.43.0.10
fail_swap_on: false
generate_serving_certificate: false
kubeproxy:
image: ""
extra_args: {}
extra_binds: []
extra_env: []
win_extra_args: {}
win_extra_binds: []
win_extra_env: []
network:
plugin: canal
options: {}
mtu: 0
node_selector: {}
update_strategy: null
tolerations: []
authentication:
strategy: x509
sans: []
webhook: null
addons: ""
addons_include: []
system_images:
etcd: rancher/coreos-etcd:v3.4.13-rancher1
alpine: rancher/rke-tools:v0.1.69
nginx_proxy: rancher/rke-tools:v0.1.69
cert_downloader: rancher/rke-tools:v0.1.69
kubernetes_services_sidecar: rancher/rke-tools:v0.1.69
kubedns: rancher/k8s-dns-kube-dns:1.15.10
dnsmasq: rancher/k8s-dns-dnsmasq-nanny:1.15.10
kubedns_sidecar: rancher/k8s-dns-sidecar:1.15.10
kubedns_autoscaler: rancher/cluster-proportional-autoscaler:1.8.1
coredns: rancher/coredns-coredns:1.7.0
coredns_autoscaler: rancher/cluster-proportional-autoscaler:1.8.1
nodelocal: rancher/k8s-dns-node-cache:1.15.13
kubernetes: rancher/hyperkube:v1.19.7-rancher1
flannel: rancher/coreos-flannel:v0.13.0-rancher1
flannel_cni: rancher/flannel-cni:v0.3.0-rancher6
calico_node: rancher/calico-node:v3.16.5
calico_cni: rancher/calico-cni:v3.16.5
calico_controllers: rancher/calico-kube-controllers:v3.16.5
calico_ctl: rancher/calico-ctl:v3.16.5
calico_flexvol: rancher/calico-pod2daemon-flexvol:v3.16.5
canal_node: rancher/calico-node:v3.16.5
canal_cni: rancher/calico-cni:v3.16.5
canal_controllers: rancher/calico-kube-controllers:v3.16.5
canal_flannel: rancher/coreos-flannel:v0.13.0-rancher1
canal_flexvol: rancher/calico-pod2daemon-flexvol:v3.16.5
weave_node: weaveworks/weave-kube:2.7.0
weave_cni: weaveworks/weave-npc:2.7.0
pod_infra_container: rancher/pause:3.2
ingress: rancher/nginx-ingress-controller:nginx-0.35.0-rancher2
ingress_backend: rancher/nginx-ingress-controller-defaultbackend:1.5-rancher1
metrics_server: rancher/metrics-server:v0.3.6
windows_pod_infra_container: rancher/kubelet-pause:v0.1.4
aci_cni_deploy_container: noiro/cnideploy:5.1.1.0.1ae238a
aci_host_container: noiro/aci-containers-host:5.1.1.0.1ae238a
aci_opflex_container: noiro/opflex:5.1.1.0.1ae238a
aci_mcast_container: noiro/opflex:5.1.1.0.1ae238a
aci_ovs_container: noiro/openvswitch:5.1.1.0.1ae238a
aci_controller_container: noiro/aci-containers-controller:5.1.1.0.1ae238a
aci_gbp_server_container: noiro/gbp-server:5.1.1.0.1ae238a
aci_opflex_server_container: noiro/opflex-server:5.1.1.0.1ae238a
ssh_key_path: ~/.ssh/id_rsa
ssh_cert_path: ""
ssh_agent_auth: false
authorization:
mode: rbac
options: {}
ignore_docker_version: null
kubernetes_version: ""
private_registries: []
ingress:
provider: ""
options: {}
node_selector: {}
extra_args: {}
dns_policy: ""
extra_envs: []
extra_volumes: []
extra_volume_mounts: []
update_strategy: null
http_port: 0
https_port: 0
network_mode: ""
tolerations: []
default_backend: null
cluster_name: ""
cloud_provider:
name: "aws"
prefix_path: ""
win_prefix_path: ""
addon_job_timeout: 0
bastion_host:
address: ""
port: ""
user: ""
ssh_key: ""
ssh_key_path: ""
ssh_cert: ""
ssh_cert_path: ""
monitoring:
provider: ""
options: {}
node_selector: {}
update_strategy: null
replicas: null
tolerations: []
restore:
restore: false
snapshot_name: ""
rotate_encryption_key: false
dns: nulljohn-morsley@pc:~/git-hub/terraform-aws-ec2-module/examples/rke$ rke --debug up
DEBU[0000] Loglevel set to [debug]
INFO[0000] Running RKE version: v1.2.5
DEBU[0000] audit log policy found in cluster.yml
INFO[0000] Initiating Kubernetes cluster
DEBU[0000] metadataInitialized: [False] []
DEBU[0000] Loading data.json from local source
DEBU[0000] data.json SHA256 checksum: 9cbaaabdccc46020fc6083b76ec392eb140a6b420c4b79880c363fac5ba4d2f1
DEBU[0000] metadata initialized successfully
DEBU[0000] metadataInitialized: [true] []
DEBU[0000] No DNS provider configured, setting default based on cluster version [1.19.7-rancher1-1]
DEBU[0000] DNS provider set to [coredns]
DEBU[0000] Checking if cluster version [1.19.7-rancher1-1] needs to have kube-api audit log enabled
DEBU[0000] Cluster version [1.19.7-rancher1-1] needs to have kube-api audit log enabled
DEBU[0000] Enabling kube-api audit log for cluster version [v1.19.7-rancher1-1]
DEBU[0000] No input provided for maxUnavailableWorker, setting it to default value of 10 percent
DEBU[0000] No input provided for maxUnavailableControlplane, setting it to default value of 1
DEBU[0000] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: controlplane
DEBU[0000] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: worker
DEBU[0000] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: etcd
DEBU[0000] [state] previous state not found, possible legacy cluster
INFO[0000] [dialer] Setup tunnel for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0000] Connecting to Docker API for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0001] Docker Info found for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: types.Info{ID:"S3M6:KLEQ:QY53:UGZX:FXOS:N47H:TTKA:5KFW:D3HC:HFJQ:ZC2Z:NVNL", Containers:0, ContainersRunning:0, ContainersPaused:0, ContainersStopped:0, Images:0, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "ipvlan", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, KernelMemoryTCP:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, PidsLimit:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:21, OomKillDisable:true, NGoroutines:34, SystemTime:"2021-02-08T18:53:05.922803044Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"5.4.0-1037-aws", OperatingSystem:"Ubuntu 18.04.5 LTS", OSVersion:"", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001ca0e0), NCPU:1, MemTotal:498470912, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"ip-10-0-1-145", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"19.03.15", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil), Warnings:[]string(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"269548fa27e0089a8b8278fc4fc781d7f65a939b", Expected:"269548fa27e0089a8b8278fc4fc781d7f65a939b"}, RuncCommit:types.Commit{ID:"ff819c7e9184c13b7c2607fe6c30ae19403a7aff", Expected:"ff819c7e9184c13b7c2607fe6c30ae19403a7aff"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}, ProductLicense:"", Warnings:[]string{"WARNING: No swap limit support"}}
INFO[0001] Checking if container [cluster-state-deployer] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0001] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0001] Image [rancher/rke-tools:v0.1.69] does not exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Error: No such image: rancher/rke-tools:v0.1.69
INFO[0001] Pulling image [rancher/rke-tools:v0.1.69] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0009] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0009] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0009] Starting container [cluster-state-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0010] [state] Successfully started [cluster-state-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0010] [certificates] Generating CA kubernetes certificates
INFO[0010] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates
INFO[0010] [certificates] GenerateServingCertificate is disabled, checking if there are unused kubelet certificates
INFO[0010] [certificates] Generating Kubernetes API server certificates
INFO[0011] [certificates] Generating Service account token key
INFO[0011] [certificates] Generating Kube Controller certificates
INFO[0011] [certificates] Generating Kube Scheduler certificates
INFO[0011] [certificates] Generating Kube Proxy certificates
INFO[0011] [certificates] Generating Node certificate
INFO[0011] [certificates] Generating admin certificates and kubeconfig
INFO[0011] [certificates] Generating Kubernetes API server proxy client certificates
INFO[0011] [certificates] Generating kube-etcd-10-0-1-145 certificate and key
INFO[0012] Successfully Deployed state file at [./cluster.rkestate]
DEBU[0012] Checking if cluster version [1.19.7-rancher1-1] needs to have kube-api audit log enabled
DEBU[0012] Cluster version [1.19.7-rancher1-1] needs to have kube-api audit log enabled
DEBU[0012] Enabling kube-api audit log for cluster version [v1.19.7-rancher1-1]
DEBU[0012] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: controlplane
DEBU[0012] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: worker
DEBU[0012] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: etcd
INFO[0012] Building Kubernetes cluster
INFO[0012] [dialer] Setup tunnel for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0012] Connecting to Docker API for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0013] Docker Info found for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: types.Info{ID:"S3M6:KLEQ:QY53:UGZX:FXOS:N47H:TTKA:5KFW:D3HC:HFJQ:ZC2Z:NVNL", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:1, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "ipvlan", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, KernelMemoryTCP:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, PidsLimit:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:24, OomKillDisable:true, NGoroutines:36, SystemTime:"2021-02-08T18:53:17.966456875Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"5.4.0-1037-aws", OperatingSystem:"Ubuntu 18.04.5 LTS", OSVersion:"", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc00026e070), NCPU:1, MemTotal:498470912, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"ip-10-0-1-145", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"19.03.15", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil), Warnings:[]string(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"269548fa27e0089a8b8278fc4fc781d7f65a939b", Expected:"269548fa27e0089a8b8278fc4fc781d7f65a939b"}, RuncCommit:types.Commit{ID:"ff819c7e9184c13b7c2607fe6c30ae19403a7aff", Expected:"ff819c7e9184c13b7c2607fe6c30ae19403a7aff"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}, ProductLicense:"", Warnings:[]string{"WARNING: No swap limit support"}}
INFO[0013] [network] Deploying port listener containers
DEBU[0013] [network] Starting deployListener [rke-etcd-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0013] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0013] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0013] Starting container [rke-etcd-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0013] [network] Successfully started [rke-etcd-port-listener] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0013] [network] Starting deployListener [rke-cp-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0013] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0013] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0014] Starting container [rke-cp-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0014] [network] Successfully started [rke-cp-port-listener] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0014] [network] Starting deployListener [rke-worker-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0014] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0014] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0014] Starting container [rke-worker-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0015] [network] Successfully started [rke-worker-port-listener] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0015] [network] Port listener containers deployed successfully
INFO[0015] [network] Running control plane -> etcd port checks
INFO[0015] [network] Checking if host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] can connect to host(s) [10.0.1.145] on port(s) [2379], try #1
DEBU[0015] [remove/rke-port-checker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0015] [remove/rke-port-checker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0015] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0015] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0015] Starting container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0015] [network] Successfully started [rke-port-checker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] [network] containerLog [] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0016] Removing container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0016] [network] Length of containerLog is [0] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0016] [network] Running control plane -> worker port checks
INFO[0016] [network] Checking if host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] can connect to host(s) [10.0.1.145] on port(s) [10250], try #1
DEBU[0016] [remove/rke-port-checker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] [remove/rke-port-checker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0016] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0016] Starting container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0016] [network] Successfully started [rke-port-checker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] [network] containerLog [] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0016] Removing container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0016] [network] Length of containerLog is [0] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0016] [network] Running workers -> control plane port checks
INFO[0016] [network] Checking if host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] can connect to host(s) [10.0.1.145] on port(s) [6443], try #1
DEBU[0016] [remove/rke-port-checker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] [remove/rke-port-checker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0016] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0016] Starting container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0016] [network] Successfully started [rke-port-checker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [network] containerLog [] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0017] Removing container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0017] [network] Length of containerLog is [0] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0017] [network] Checking KubeAPI port Control Plane hosts
DEBU[0017] [network] Checking KubeAPI port [6443] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0017] [network] Removing port listener containers
DEBU[0017] [remove/rke-etcd-port-listener] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-etcd-port-listener] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] Removing container [rke-etcd-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0017] [remove/rke-etcd-port-listener] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-cp-port-listener] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-cp-port-listener] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] Removing container [rke-cp-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0017] [remove/rke-cp-port-listener] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-worker-port-listener] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-worker-port-listener] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] Removing container [rke-worker-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] [network] Port listener containers removed successfully
INFO[0017] [certificates] Deploying kubernetes certificates to Cluster nodes
INFO[0017] Checking if container [cert-deployer] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0017] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0017] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] Starting container [cert-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0018] [certificates] Successfully started Certificate deployer container: cert-deployer
INFO[0018] Checking if container [cert-deployer] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0023] Checking if container [cert-deployer] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0023] Removing container [cert-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0023] [reconcile] Rebuilding and updating local kube config
DEBU[0023] [reconcile] Rebuilding and updating local kube config, creating new kubeconfig
DEBU[0023] Deploying admin Kubeconfig locally at [./kube_config_cluster.yml]
INFO[0023] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml]
DEBU[0023] [version] Using ./kube_config_cluster.yml to connect to Kubernetes cluster..
DEBU[0023] [version] Getting Kubernetes server version..
WARN[0023] [reconcile] host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is a control plane node without reachable Kubernetes API endpoint in the cluster
WARN[0023] [reconcile] no control plane node with reachable Kubernetes API endpoint in the cluster found
INFO[0023] [certificates] Successfully deployed kubernetes certificates to Cluster nodes
INFO[0023] [file-deploy] Deploying file [/etc/kubernetes/cloud-config] to node [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0023] [remove/file-deployer] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0023] [remove/file-deployer] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0023] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0023] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0023] Starting container [file-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0024] Successfully started [file-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0024] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0024] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0024] Container [file-deployer] is still running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: stderr: [], stdout: []
INFO[0025] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] Exit code for [file-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is [0]
DEBU[0025] [remove/file-deployer] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] [remove/file-deployer] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Removing container [file-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0025] [remove/file-deployer] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] [file-deploy] Successfully deployed file [/etc/kubernetes/cloud-config] on node [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] [/etc/kubernetes/cloud-config] Successfully deployed kubernetes cloud config to Cluster nodes
INFO[0025] [file-deploy] Deploying file [/etc/kubernetes/audit-policy.yaml] to node [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] [remove/file-deployer] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] [remove/file-deployer] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0025] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Starting container [file-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0025] Successfully started [file-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Container [file-deployer] is still running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: stderr: [], stdout: []
INFO[0026] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0026] Exit code for [file-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is [0]
DEBU[0026] [remove/file-deployer] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0026] [remove/file-deployer] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0026] Removing container [file-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0026] [remove/file-deployer] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0026] [file-deploy] Successfully deployed file [/etc/kubernetes/audit-policy.yaml] on node [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0026] [/etc/kubernetes/audit-policy.yaml] Successfully deployed audit policy file to Cluster control nodes
INFO[0026] [reconcile] Reconciling cluster state
INFO[0026] [reconcile] This is newly generated cluster
DEBU[0026] Encryption is disabled in both current and new spec; no action is required
INFO[0026] Pre-pulling kubernetes images
DEBU[0026] Checking if image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0026] Image [rancher/hyperkube:v1.19.7-rancher1] does not exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Error: No such image: rancher/hyperkube:v1.19.7-rancher1
INFO[0026] Pulling image [rancher/hyperkube:v1.19.7-rancher1] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0070] Checking if image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0070] Image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0070] Kubernetes images pulled successfully
DEBU[0070] getDefaultKubernetesServicesOptions: getting serviceOptions for cluster version [v1.19.7-rancher1-1]
DEBU[0070] Extracted version [v1.19.7-rancher1] from image [rancher/hyperkube:v1.19.7-rancher1]
DEBU[0070] getDefaultKubernetesServicesOptions: serviceOptions found for cluster major version [v1.19]
DEBU[0070] Extracted version [v3.4.13-rancher1] from image [rancher/coreos-etcd:v3.4.13-rancher1]
DEBU[0070] etcd version [3.4.13-rancher1] is higher than max version [3.4.3-rancher99] for advertising port 4001, not going to advertise port 4001
DEBU[0070] Version [3.4.13-rancher1] is equal or higher than version [3.2.99]
INFO[0070] [etcd] Building up etcd plane..
DEBU[0070] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0070] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0071] Starting container [etcd-fix-perm] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0071] Successfully started [etcd-fix-perm] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0071] Waiting for [etcd-fix-perm] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0071] Waiting for [etcd-fix-perm] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0071] Container [etcd-fix-perm] is still running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: stderr: [], stdout: []
INFO[0072] Waiting for [etcd-fix-perm] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0072] Exit code for [etcd-fix-perm] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is [0]
DEBU[0072] [remove/etcd-fix-perm] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0072] [remove/etcd-fix-perm] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0072] Removing container [etcd-fix-perm] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0072] [remove/etcd-fix-perm] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0072] Checking if image [rancher/coreos-etcd:v3.4.13-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0072] Image [rancher/coreos-etcd:v3.4.13-rancher1] does not exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Error: No such image: rancher/coreos-etcd:v3.4.13-rancher1
INFO[0072] Pulling image [rancher/coreos-etcd:v3.4.13-rancher1] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0078] Checking if image [rancher/coreos-etcd:v3.4.13-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0078] Image [rancher/coreos-etcd:v3.4.13-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0078] Starting container [etcd] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0078] [etcd] Successfully started [etcd] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] Extracted version [v0.1.69] from image [rancher/rke-tools:v0.1.69]
DEBU[0078] Extracted version [v0.1.69] from image [rancher/rke-tools:v0.1.69]
INFO[0078] [etcd] Running rolling snapshot container [etcd-snapshot-once] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] [etcd] Using command [/opt/rke-tools/rke-etcd-backup etcd-backup save --cacert /etc/kubernetes/ssl/kube-ca.pem --cert /etc/kubernetes/ssl/kube-node.pem --key /etc/kubernetes/ssl/kube-node-key.pem --name etcd-rolling-snapshots --endpoints=10.0.1.145:2379 --retention=72h --creation=12h] for rolling snapshot container [etcd-rolling-snapshots] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] [remove/etcd-rolling-snapshots] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] [remove/etcd-rolling-snapshots] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0078] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0078] Starting container [etcd-rolling-snapshots] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0079] [etcd] Successfully started [etcd-rolling-snapshots] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0084] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0084] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0084] Starting container [rke-bundle-cert] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0084] [certificates] Successfully started [rke-bundle-cert] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0084] Waiting for [rke-bundle-cert] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0084] Container [rke-bundle-cert] is still running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: stderr: [], stdout: []
INFO[0085] Waiting for [rke-bundle-cert] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0085] Exit code for [rke-bundle-cert] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is [0]
INFO[0085] [certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0085] Removing container [rke-bundle-cert] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0086] [etcd] Creating log link for Container [etcd-rolling-snapshots] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0086] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0086] Starting container [rke-log-linker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0086] [etcd] Successfully started [rke-log-linker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0086] Removing container [rke-log-linker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0086] [remove/rke-log-linker] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [etcd] Successfully created log link for Container [etcd-rolling-snapshots] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [etcd] Creating log link for Container [etcd] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0086] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0087] Starting container [rke-log-linker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0087] [etcd] Successfully started [rke-log-linker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0087] [remove/rke-log-linker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0087] [remove/rke-log-linker] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0087] Removing container [rke-log-linker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0087] [remove/rke-log-linker] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0087] [etcd] Successfully created log link for Container [etcd] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0087] [etcd] Successfully started etcd plane.. Checking etcd cluster health
DEBU[0087] [etcd] check etcd cluster health on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0088] [etcd] etcd host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] reported healthy=true
DEBU[0088] getDefaultKubernetesServicesOptions: getting serviceOptions for cluster version [v1.19.7-rancher1-1]
DEBU[0088] Extracted version [v1.19.7-rancher1] from image [rancher/hyperkube:v1.19.7-rancher1]
DEBU[0088] getDefaultKubernetesServicesOptions: serviceOptions found for cluster major version [v1.19]
DEBU[0088] Extracted version [v3.4.13-rancher1] from image [rancher/coreos-etcd:v3.4.13-rancher1]
DEBU[0088] etcd version [3.4.13-rancher1] is higher than max version [3.4.3-rancher99] for advertising port 4001, not going to advertise port 4001
DEBU[0088] Version [3.4.13-rancher1] is equal or higher than version [3.2.99]
INFO[0088] [controlplane] Building up Controller Plane..
DEBU[0088] [remove/nginx-proxy] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0088] [remove/nginx-proxy] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0088] Checking if container [service-sidekick] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0088] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0088] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0088] Checking if image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0088] Image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0088] Starting container [kube-apiserver] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0088] [controlplane] Successfully started [kube-apiserver] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0088] [healthcheck] Start Healthcheck on service [kube-apiserver] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0089] [healthcheck] Failed to check https://localhost:6443/healthz for service [kube-apiserver] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Get "https://localhost:6443/healthz": Unable to access the service on localhost:6443. The service might be still starting up. Error: ssh: rejected: connect failed (Connection refused), try #1
DEBU[0236] [healthcheck] Failed to check https://localhost:6443/healthz for service [kube-apiserver] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Get "https://localhost:6443/healthz": Failed to dial ssh using address [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com:22]: ssh: handshake failed: EOF, try #2
^CIt just hangs on this output? This seems like a too low specced machine or a network issue/error I guess. What instance type are you using? Can you still login using SSH and access Docker (docker ps) when it hangs on this output?
1 Like
Hi @superseb ,
Thanks, mate.
Due to an error with my Terraform, the setting of the EC2 instance was left at t2.nano, and not overridden like what it was supposed to be!
Now it works 
Thanks again,
John
1 Like
Hi I am getting below error while running command:
rke up --config ./rancher-cluster.yml
getDefaultKubernetesServicesOptions: No serviceOptions found for cluster version [v1.19.6-rancher1-1] or cluster major version [v2.5]
Please create an issue on https://github.com/rancher/rke/issues/new with all the info requested in the issue template and the complete log so we can take a look at it.
Any update about this topic ? I got the same issue, when i create my local yaml file, and rke up does’nt work
The issue was solved after changing the instance type to a higher specced/larger instance. The other issue was fixed via I am getting below error while running command: rke up --config ./rancher-cluster.yml getDefaultKubernetesServicesOptions: No serviceOptions found for cluster version [v1.19.6-rancher1-1] or cluster major version [v2.5] · Issue #2482 · rancher/rke · GitHub