RKE (on the command line) with AWS and EC2

Has anyone got the rke via the command line working with AWS and EC2? As I can’t! :frowning:

I run ‘rke config’ and enter what I think are the right values.

I then run ‘rke up’, but just hangs on the ‘[healthcheck]’ part.

Thanks very much for any replies…

Please supply the cluster.yml and full (debug) log to see what the values are (redacted if sensitive) and what the log exactly indicates.

1 Like
    # If you intened to deploy Kubernetes in an air-gapped environment,
    # please consult the documentation on how to configure custom RKE images.
nodes:
- address: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
port: "22"
internal_address: 10.0.1.145
role:
- controlplane
- worker
- etcd
hostname_override: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
user: ubuntu
docker_socket: /var/run/docker.sock
ssh_key: ""
ssh_key_path: keys/rke-ec2-example-summary-alien-ec2.pem
ssh_cert: ""
ssh_cert_path: ""
labels: {}
taints: []
services:
etcd:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    win_extra_args: {}
    win_extra_binds: []
    win_extra_env: []
    external_urls: []
    ca_cert: ""
    cert: ""
    key: ""
    path: ""
    uid: 0
    gid: 0
    snapshot: null
    retention: ""
    creation: ""
    backup_config: null
kube-api:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    win_extra_args: {}
    win_extra_binds: []
    win_extra_env: []
    service_cluster_ip_range: 10.43.0.0/16
    service_node_port_range: ""
    pod_security_policy: false
    always_pull_images: false
    secrets_encryption_config: null
    audit_log: null
    admission_configuration: null
    event_rate_limit: null
kube-controller:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    win_extra_args: {}
    win_extra_binds: []
    win_extra_env: []
    cluster_cidr: 10.42.0.0/16
    service_cluster_ip_range: 10.43.0.0/16
scheduler:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    win_extra_args: {}
    win_extra_binds: []
    win_extra_env: []
kubelet:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    win_extra_args: {}
    win_extra_binds: []
    win_extra_env: []
    cluster_domain: cluster.local
    infra_container_image: ""
    cluster_dns_server: 10.43.0.10
    fail_swap_on: false
    generate_serving_certificate: false
kubeproxy:
    image: ""
    extra_args: {}
    extra_binds: []
    extra_env: []
    win_extra_args: {}
    win_extra_binds: []
    win_extra_env: []
network:
plugin: canal
options: {}
mtu: 0
node_selector: {}
update_strategy: null
tolerations: []
authentication:
strategy: x509
sans: []
webhook: null
addons: ""
addons_include: []
system_images:
etcd: rancher/coreos-etcd:v3.4.13-rancher1
alpine: rancher/rke-tools:v0.1.69
nginx_proxy: rancher/rke-tools:v0.1.69
cert_downloader: rancher/rke-tools:v0.1.69
kubernetes_services_sidecar: rancher/rke-tools:v0.1.69
kubedns: rancher/k8s-dns-kube-dns:1.15.10
dnsmasq: rancher/k8s-dns-dnsmasq-nanny:1.15.10
kubedns_sidecar: rancher/k8s-dns-sidecar:1.15.10
kubedns_autoscaler: rancher/cluster-proportional-autoscaler:1.8.1
coredns: rancher/coredns-coredns:1.7.0
coredns_autoscaler: rancher/cluster-proportional-autoscaler:1.8.1
nodelocal: rancher/k8s-dns-node-cache:1.15.13
kubernetes: rancher/hyperkube:v1.19.7-rancher1
flannel: rancher/coreos-flannel:v0.13.0-rancher1
flannel_cni: rancher/flannel-cni:v0.3.0-rancher6
calico_node: rancher/calico-node:v3.16.5
calico_cni: rancher/calico-cni:v3.16.5
calico_controllers: rancher/calico-kube-controllers:v3.16.5
calico_ctl: rancher/calico-ctl:v3.16.5
calico_flexvol: rancher/calico-pod2daemon-flexvol:v3.16.5
canal_node: rancher/calico-node:v3.16.5
canal_cni: rancher/calico-cni:v3.16.5
canal_controllers: rancher/calico-kube-controllers:v3.16.5
canal_flannel: rancher/coreos-flannel:v0.13.0-rancher1
canal_flexvol: rancher/calico-pod2daemon-flexvol:v3.16.5
weave_node: weaveworks/weave-kube:2.7.0
weave_cni: weaveworks/weave-npc:2.7.0
pod_infra_container: rancher/pause:3.2
ingress: rancher/nginx-ingress-controller:nginx-0.35.0-rancher2
ingress_backend: rancher/nginx-ingress-controller-defaultbackend:1.5-rancher1
metrics_server: rancher/metrics-server:v0.3.6
windows_pod_infra_container: rancher/kubelet-pause:v0.1.4
aci_cni_deploy_container: noiro/cnideploy:5.1.1.0.1ae238a
aci_host_container: noiro/aci-containers-host:5.1.1.0.1ae238a
aci_opflex_container: noiro/opflex:5.1.1.0.1ae238a
aci_mcast_container: noiro/opflex:5.1.1.0.1ae238a
aci_ovs_container: noiro/openvswitch:5.1.1.0.1ae238a
aci_controller_container: noiro/aci-containers-controller:5.1.1.0.1ae238a
aci_gbp_server_container: noiro/gbp-server:5.1.1.0.1ae238a
aci_opflex_server_container: noiro/opflex-server:5.1.1.0.1ae238a
ssh_key_path: ~/.ssh/id_rsa
ssh_cert_path: ""
ssh_agent_auth: false
authorization:
mode: rbac
options: {}
ignore_docker_version: null
kubernetes_version: ""
private_registries: []
ingress:
provider: ""
options: {}
node_selector: {}
extra_args: {}
dns_policy: ""
extra_envs: []
extra_volumes: []
extra_volume_mounts: []
update_strategy: null
http_port: 0
https_port: 0
network_mode: ""
tolerations: []
default_backend: null
cluster_name: ""
cloud_provider:
name: "aws"
prefix_path: ""
win_prefix_path: ""
addon_job_timeout: 0
bastion_host:
address: ""
port: ""
user: ""
ssh_key: ""
ssh_key_path: ""
ssh_cert: ""
ssh_cert_path: ""
monitoring:
provider: ""
options: {}
node_selector: {}
update_strategy: null
replicas: null
tolerations: []
restore:
restore: false
snapshot_name: ""
rotate_encryption_key: false
dns: null
john-morsley@pc:~/git-hub/terraform-aws-ec2-module/examples/rke$ rke --debug up
DEBU[0000] Loglevel set to [debug]
INFO[0000] Running RKE version: v1.2.5
DEBU[0000] audit log policy found in cluster.yml
INFO[0000] Initiating Kubernetes cluster
DEBU[0000] metadataInitialized: [False] []
DEBU[0000] Loading data.json from local source
DEBU[0000] data.json SHA256 checksum: 9cbaaabdccc46020fc6083b76ec392eb140a6b420c4b79880c363fac5ba4d2f1
DEBU[0000] metadata initialized successfully
DEBU[0000] metadataInitialized: [true] []
DEBU[0000] No DNS provider configured, setting default based on cluster version [1.19.7-rancher1-1]
DEBU[0000] DNS provider set to [coredns]
DEBU[0000] Checking if cluster version [1.19.7-rancher1-1] needs to have kube-api audit log enabled
DEBU[0000] Cluster version [1.19.7-rancher1-1] needs to have kube-api audit log enabled
DEBU[0000] Enabling kube-api audit log for cluster version [v1.19.7-rancher1-1]
DEBU[0000] No input provided for maxUnavailableWorker, setting it to default value of 10 percent
DEBU[0000] No input provided for maxUnavailableControlplane, setting it to default value of 1
DEBU[0000] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: controlplane
DEBU[0000] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: worker
DEBU[0000] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: etcd
DEBU[0000] [state] previous state not found, possible legacy cluster
INFO[0000] [dialer] Setup tunnel for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0000] Connecting to Docker API for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0001] Docker Info found for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: types.Info{ID:"S3M6:KLEQ:QY53:UGZX:FXOS:N47H:TTKA:5KFW:D3HC:HFJQ:ZC2Z:NVNL", Containers:0, ContainersRunning:0, ContainersPaused:0, ContainersStopped:0, Images:0, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "ipvlan", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, KernelMemoryTCP:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, PidsLimit:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:21, OomKillDisable:true, NGoroutines:34, SystemTime:"2021-02-08T18:53:05.922803044Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"5.4.0-1037-aws", OperatingSystem:"Ubuntu 18.04.5 LTS", OSVersion:"", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc0001ca0e0), NCPU:1, MemTotal:498470912, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"ip-10-0-1-145", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"19.03.15", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil), Warnings:[]string(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"269548fa27e0089a8b8278fc4fc781d7f65a939b", Expected:"269548fa27e0089a8b8278fc4fc781d7f65a939b"}, RuncCommit:types.Commit{ID:"ff819c7e9184c13b7c2607fe6c30ae19403a7aff", Expected:"ff819c7e9184c13b7c2607fe6c30ae19403a7aff"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}, ProductLicense:"", Warnings:[]string{"WARNING: No swap limit support"}}
INFO[0001] Checking if container [cluster-state-deployer] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0001] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0001] Image [rancher/rke-tools:v0.1.69] does not exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Error: No such image: rancher/rke-tools:v0.1.69
INFO[0001] Pulling image [rancher/rke-tools:v0.1.69] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0009] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0009] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0009] Starting container [cluster-state-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0010] [state] Successfully started [cluster-state-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0010] [certificates] Generating CA kubernetes certificates
INFO[0010] [certificates] Generating Kubernetes API server aggregation layer requestheader client CA certificates
INFO[0010] [certificates] GenerateServingCertificate is disabled, checking if there are unused kubelet certificates
INFO[0010] [certificates] Generating Kubernetes API server certificates
INFO[0011] [certificates] Generating Service account token key
INFO[0011] [certificates] Generating Kube Controller certificates
INFO[0011] [certificates] Generating Kube Scheduler certificates
INFO[0011] [certificates] Generating Kube Proxy certificates
INFO[0011] [certificates] Generating Node certificate
INFO[0011] [certificates] Generating admin certificates and kubeconfig
INFO[0011] [certificates] Generating Kubernetes API server proxy client certificates
INFO[0011] [certificates] Generating kube-etcd-10-0-1-145 certificate and key
INFO[0012] Successfully Deployed state file at [./cluster.rkestate]
DEBU[0012] Checking if cluster version [1.19.7-rancher1-1] needs to have kube-api audit log enabled
DEBU[0012] Cluster version [1.19.7-rancher1-1] needs to have kube-api audit log enabled
DEBU[0012] Enabling kube-api audit log for cluster version [v1.19.7-rancher1-1]
DEBU[0012] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: controlplane
DEBU[0012] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: worker
DEBU[0012] Host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com has role: etcd
INFO[0012] Building Kubernetes cluster
INFO[0012] [dialer] Setup tunnel for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0012] Connecting to Docker API for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0013] Docker Info found for host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: types.Info{ID:"S3M6:KLEQ:QY53:UGZX:FXOS:N47H:TTKA:5KFW:D3HC:HFJQ:ZC2Z:NVNL", Containers:1, ContainersRunning:0, ContainersPaused:0, ContainersStopped:1, Images:1, Driver:"overlay2", DriverStatus:[][2]string{[2]string{"Backing Filesystem", "extfs"}, [2]string{"Supports d_type", "true"}, [2]string{"Native Overlay Diff", "true"}}, SystemStatus:[][2]string(nil), Plugins:types.PluginsInfo{Volume:[]string{"local"}, Network:[]string{"bridge", "host", "ipvlan", "macvlan", "null", "overlay"}, Authorization:[]string(nil), Log:[]string{"awslogs", "fluentd", "gcplogs", "gelf", "journald", "json-file", "local", "logentries", "splunk", "syslog"}}, MemoryLimit:true, SwapLimit:false, KernelMemory:true, KernelMemoryTCP:true, CPUCfsPeriod:true, CPUCfsQuota:true, CPUShares:true, CPUSet:true, PidsLimit:true, IPv4Forwarding:true, BridgeNfIptables:true, BridgeNfIP6tables:true, Debug:false, NFd:24, OomKillDisable:true, NGoroutines:36, SystemTime:"2021-02-08T18:53:17.966456875Z", LoggingDriver:"json-file", CgroupDriver:"cgroupfs", NEventsListener:0, KernelVersion:"5.4.0-1037-aws", OperatingSystem:"Ubuntu 18.04.5 LTS", OSVersion:"", OSType:"linux", Architecture:"x86_64", IndexServerAddress:"https://index.docker.io/v1/", RegistryConfig:(*registry.ServiceConfig)(0xc00026e070), NCPU:1, MemTotal:498470912, GenericResources:[]swarm.GenericResource(nil), DockerRootDir:"/var/lib/docker", HTTPProxy:"", HTTPSProxy:"", NoProxy:"", Name:"ip-10-0-1-145", Labels:[]string{}, ExperimentalBuild:false, ServerVersion:"19.03.15", ClusterStore:"", ClusterAdvertise:"", Runtimes:map[string]types.Runtime{"runc":types.Runtime{Path:"runc", Args:[]string(nil)}}, DefaultRuntime:"runc", Swarm:swarm.Info{NodeID:"", NodeAddr:"", LocalNodeState:"inactive", ControlAvailable:false, Error:"", RemoteManagers:[]swarm.Peer(nil), Nodes:0, Managers:0, Cluster:(*swarm.ClusterInfo)(nil), Warnings:[]string(nil)}, LiveRestoreEnabled:false, Isolation:"", InitBinary:"docker-init", ContainerdCommit:types.Commit{ID:"269548fa27e0089a8b8278fc4fc781d7f65a939b", Expected:"269548fa27e0089a8b8278fc4fc781d7f65a939b"}, RuncCommit:types.Commit{ID:"ff819c7e9184c13b7c2607fe6c30ae19403a7aff", Expected:"ff819c7e9184c13b7c2607fe6c30ae19403a7aff"}, InitCommit:types.Commit{ID:"fec3683", Expected:"fec3683"}, SecurityOptions:[]string{"apparmor", "seccomp"}, ProductLicense:"", Warnings:[]string{"WARNING: No swap limit support"}}
INFO[0013] [network] Deploying port listener containers
DEBU[0013] [network] Starting deployListener [rke-etcd-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0013] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0013] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0013] Starting container [rke-etcd-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0013] [network] Successfully started [rke-etcd-port-listener] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0013] [network] Starting deployListener [rke-cp-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0013] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0013] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0014] Starting container [rke-cp-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0014] [network] Successfully started [rke-cp-port-listener] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0014] [network] Starting deployListener [rke-worker-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0014] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0014] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0014] Starting container [rke-worker-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0015] [network] Successfully started [rke-worker-port-listener] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0015] [network] Port listener containers deployed successfully
INFO[0015] [network] Running control plane -> etcd port checks
INFO[0015] [network] Checking if host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] can connect to host(s) [10.0.1.145] on port(s) [2379], try #1
DEBU[0015] [remove/rke-port-checker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0015] [remove/rke-port-checker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0015] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0015] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0015] Starting container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0015] [network] Successfully started [rke-port-checker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] [network] containerLog [] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0016] Removing container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0016] [network] Length of containerLog is [0] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0016] [network] Running control plane -> worker port checks
INFO[0016] [network] Checking if host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] can connect to host(s) [10.0.1.145] on port(s) [10250], try #1
DEBU[0016] [remove/rke-port-checker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] [remove/rke-port-checker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0016] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0016] Starting container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0016] [network] Successfully started [rke-port-checker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] [network] containerLog [] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0016] Removing container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0016] [network] Length of containerLog is [0] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0016] [network] Running workers -> control plane port checks
INFO[0016] [network] Checking if host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] can connect to host(s) [10.0.1.145] on port(s) [6443], try #1
DEBU[0016] [remove/rke-port-checker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] [remove/rke-port-checker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0016] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0016] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0016] Starting container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0016] [network] Successfully started [rke-port-checker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [network] containerLog [] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0017] Removing container [rke-port-checker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0017] [network] Length of containerLog is [0] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0017] [network] Checking KubeAPI port Control Plane hosts
DEBU[0017] [network] Checking KubeAPI port [6443] on host: ec2-3-9-164-183.eu-west-2.compute.amazonaws.com
INFO[0017] [network] Removing port listener containers
DEBU[0017] [remove/rke-etcd-port-listener] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-etcd-port-listener] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] Removing container [rke-etcd-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0017] [remove/rke-etcd-port-listener] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-cp-port-listener] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-cp-port-listener] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] Removing container [rke-cp-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0017] [remove/rke-cp-port-listener] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-worker-port-listener] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0017] [remove/rke-worker-port-listener] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] Removing container [rke-worker-port-listener] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0017] [remove/rke-worker-port-listener] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] [network] Port listener containers removed successfully
INFO[0017] [certificates] Deploying kubernetes certificates to Cluster nodes
INFO[0017] Checking if container [cert-deployer] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0017] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0017] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0017] Starting container [cert-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0018] [certificates] Successfully started Certificate deployer container: cert-deployer
INFO[0018] Checking if container [cert-deployer] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0023] Checking if container [cert-deployer] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0023] Removing container [cert-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0023] [reconcile] Rebuilding and updating local kube config
DEBU[0023] [reconcile] Rebuilding and updating local kube config, creating new kubeconfig
DEBU[0023] Deploying admin Kubeconfig locally at [./kube_config_cluster.yml]
INFO[0023] Successfully Deployed local admin kubeconfig at [./kube_config_cluster.yml]
DEBU[0023] [version] Using ./kube_config_cluster.yml to connect to Kubernetes cluster..
DEBU[0023] [version] Getting Kubernetes server version..
WARN[0023] [reconcile] host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is a control plane node without reachable Kubernetes API endpoint in the cluster
WARN[0023] [reconcile] no control plane node with reachable Kubernetes API endpoint in the cluster found
INFO[0023] [certificates] Successfully deployed kubernetes certificates to Cluster nodes
INFO[0023] [file-deploy] Deploying file [/etc/kubernetes/cloud-config] to node [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0023] [remove/file-deployer] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0023] [remove/file-deployer] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0023] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0023] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0023] Starting container [file-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0024] Successfully started [file-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0024] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0024] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0024] Container [file-deployer] is still running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: stderr: [], stdout: []
INFO[0025] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] Exit code for [file-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is [0]
DEBU[0025] [remove/file-deployer] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] [remove/file-deployer] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Removing container [file-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0025] [remove/file-deployer] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] [file-deploy] Successfully deployed file [/etc/kubernetes/cloud-config] on node [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] [/etc/kubernetes/cloud-config] Successfully deployed kubernetes cloud config to Cluster nodes
INFO[0025] [file-deploy] Deploying file [/etc/kubernetes/audit-policy.yaml] to node [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] [remove/file-deployer] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] [remove/file-deployer] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0025] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0025] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Starting container [file-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0025] Successfully started [file-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0025] Container [file-deployer] is still running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: stderr: [], stdout: []
INFO[0026] Waiting for [file-deployer] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0026] Exit code for [file-deployer] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is [0]
DEBU[0026] [remove/file-deployer] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0026] [remove/file-deployer] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0026] Removing container [file-deployer] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0026] [remove/file-deployer] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0026] [file-deploy] Successfully deployed file [/etc/kubernetes/audit-policy.yaml] on node [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0026] [/etc/kubernetes/audit-policy.yaml] Successfully deployed audit policy file to Cluster control nodes
INFO[0026] [reconcile] Reconciling cluster state
INFO[0026] [reconcile] This is newly generated cluster
DEBU[0026] Encryption is disabled in both current and new spec; no action is required
INFO[0026] Pre-pulling kubernetes images
DEBU[0026] Checking if image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0026] Image [rancher/hyperkube:v1.19.7-rancher1] does not exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Error: No such image: rancher/hyperkube:v1.19.7-rancher1
INFO[0026] Pulling image [rancher/hyperkube:v1.19.7-rancher1] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0070] Checking if image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0070] Image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0070] Kubernetes images pulled successfully
DEBU[0070] getDefaultKubernetesServicesOptions: getting serviceOptions for cluster version [v1.19.7-rancher1-1]
DEBU[0070] Extracted version [v1.19.7-rancher1] from image [rancher/hyperkube:v1.19.7-rancher1]
DEBU[0070] getDefaultKubernetesServicesOptions: serviceOptions found for cluster major version [v1.19]
DEBU[0070] Extracted version [v3.4.13-rancher1] from image [rancher/coreos-etcd:v3.4.13-rancher1]
DEBU[0070] etcd version [3.4.13-rancher1] is higher than max version [3.4.3-rancher99] for advertising port 4001, not going to advertise port 4001
DEBU[0070] Version [3.4.13-rancher1] is equal or higher than version [3.2.99]
INFO[0070] [etcd] Building up etcd plane..
DEBU[0070] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0070] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0071] Starting container [etcd-fix-perm] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0071] Successfully started [etcd-fix-perm] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0071] Waiting for [etcd-fix-perm] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0071] Waiting for [etcd-fix-perm] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0071] Container [etcd-fix-perm] is still running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: stderr: [], stdout: []
INFO[0072] Waiting for [etcd-fix-perm] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0072] Exit code for [etcd-fix-perm] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is [0]
DEBU[0072] [remove/etcd-fix-perm] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0072] [remove/etcd-fix-perm] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0072] Removing container [etcd-fix-perm] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0072] [remove/etcd-fix-perm] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0072] Checking if image [rancher/coreos-etcd:v3.4.13-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0072] Image [rancher/coreos-etcd:v3.4.13-rancher1] does not exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Error: No such image: rancher/coreos-etcd:v3.4.13-rancher1
INFO[0072] Pulling image [rancher/coreos-etcd:v3.4.13-rancher1] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0078] Checking if image [rancher/coreos-etcd:v3.4.13-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0078] Image [rancher/coreos-etcd:v3.4.13-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0078] Starting container [etcd] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0078] [etcd] Successfully started [etcd] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] Extracted version [v0.1.69] from image [rancher/rke-tools:v0.1.69]
DEBU[0078] Extracted version [v0.1.69] from image [rancher/rke-tools:v0.1.69]
INFO[0078] [etcd] Running rolling snapshot container [etcd-snapshot-once] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] [etcd] Using command [/opt/rke-tools/rke-etcd-backup etcd-backup save --cacert /etc/kubernetes/ssl/kube-ca.pem --cert /etc/kubernetes/ssl/kube-node.pem --key /etc/kubernetes/ssl/kube-node-key.pem --name etcd-rolling-snapshots --endpoints=10.0.1.145:2379 --retention=72h --creation=12h] for rolling snapshot container [etcd-rolling-snapshots] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] [remove/etcd-rolling-snapshots] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] [remove/etcd-rolling-snapshots] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0078] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0078] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0078] Starting container [etcd-rolling-snapshots] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0079] [etcd] Successfully started [etcd-rolling-snapshots] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0084] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0084] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0084] Starting container [rke-bundle-cert] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0084] [certificates] Successfully started [rke-bundle-cert] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0084] Waiting for [rke-bundle-cert] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0084] Container [rke-bundle-cert] is still running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: stderr: [], stdout: []
INFO[0085] Waiting for [rke-bundle-cert] container to exit on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0085] Exit code for [rke-bundle-cert] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] is [0]
INFO[0085] [certificates] successfully saved certificate bundle [/opt/rke/etcd-snapshots//pki.bundle.tar.gz] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0085] Removing container [rke-bundle-cert] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0086] [etcd] Creating log link for Container [etcd-rolling-snapshots] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0086] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0086] Starting container [rke-log-linker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0086] [etcd] Successfully started [rke-log-linker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0086] Removing container [rke-log-linker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0086] [remove/rke-log-linker] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [etcd] Successfully created log link for Container [etcd-rolling-snapshots] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [etcd] Creating log link for Container [etcd] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] [remove/rke-log-linker] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0086] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0086] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0087] Starting container [rke-log-linker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0087] [etcd] Successfully started [rke-log-linker] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0087] [remove/rke-log-linker] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0087] [remove/rke-log-linker] Removing container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0087] Removing container [rke-log-linker] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0087] [remove/rke-log-linker] Successfully removed container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0087] [etcd] Successfully created log link for Container [etcd] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0087] [etcd] Successfully started etcd plane.. Checking etcd cluster health
DEBU[0087] [etcd] check etcd cluster health on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0088] [etcd] etcd host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com] reported healthy=true
DEBU[0088] getDefaultKubernetesServicesOptions: getting serviceOptions for cluster version [v1.19.7-rancher1-1]
DEBU[0088] Extracted version [v1.19.7-rancher1] from image [rancher/hyperkube:v1.19.7-rancher1]
DEBU[0088] getDefaultKubernetesServicesOptions: serviceOptions found for cluster major version [v1.19]
DEBU[0088] Extracted version [v3.4.13-rancher1] from image [rancher/coreos-etcd:v3.4.13-rancher1]
DEBU[0088] etcd version [3.4.13-rancher1] is higher than max version [3.4.3-rancher99] for advertising port 4001, not going to advertise port 4001
DEBU[0088] Version [3.4.13-rancher1] is equal or higher than version [3.2.99]
INFO[0088] [controlplane] Building up Controller Plane..
DEBU[0088] [remove/nginx-proxy] Checking if container is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0088] [remove/nginx-proxy] Container doesn't exist on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0088] Checking if container [service-sidekick] is running on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
DEBU[0088] Checking if image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0088] Image [rancher/rke-tools:v0.1.69] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0088] Checking if image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0088] Image [rancher/hyperkube:v1.19.7-rancher1] exists on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0088] Starting container [kube-apiserver] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com], try #1
INFO[0088] [controlplane] Successfully started [kube-apiserver] container on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
INFO[0088] [healthcheck] Start Healthcheck on service [kube-apiserver] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]
DEBU[0089] [healthcheck] Failed to check https://localhost:6443/healthz for service [kube-apiserver] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Get "https://localhost:6443/healthz": Unable to access the service on localhost:6443. The service might be still starting up. Error: ssh: rejected: connect failed (Connection refused), try #1
DEBU[0236] [healthcheck] Failed to check https://localhost:6443/healthz for service [kube-apiserver] on host [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com]: Get "https://localhost:6443/healthz": Failed to dial ssh using address [ec2-3-9-164-183.eu-west-2.compute.amazonaws.com:22]: ssh: handshake failed: EOF, try #2
^C

Hi @superseb ,
I hope the above is what you need.
I’m really stumped! :frowning:
Thanks,
John

It just hangs on this output? This seems like a too low specced machine or a network issue/error I guess. What instance type are you using? Can you still login using SSH and access Docker (docker ps) when it hangs on this output?

1 Like

Hi @superseb ,
Thanks, mate.
Due to an error with my Terraform, the setting of the EC2 instance was left at t2.nano, and not overridden like what it was supposed to be!
Now it works :slight_smile:
Thanks again,
John

1 Like