For unknown reason, I can not boot the RancherOS 0.5 after I reboot the AWS instance!

Hi guys,

Can any body help me?

I run the RancherOS 0.5 in AWS EC2 Instance, and use Rancher v1.1.0 to manage the cattle clustering.

During the AWS EC2 Instance volume backup, One of RancherOS instance stopped to response (can not remote access by ssh). After I reboot this dead instance, I can not even to boot this instance.

Here is the booting log got from AWS admin console. It seems like System-Docker can not bootup.

Can any one help me?

=================== Instance Boot Logs ============

[ 0.145141] x2apic: IRQ remapping doesn’t support X2APIC mode
[ 0.151004] Switched APIC routing to physical flat.
[ 0.157000] …TIMER: vector=0x30 apic1=0 pin1=2 apic2=0 pin2=0
[ 0.174637] clocksource: xen: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[ 0.184019] installing Xen timer for CPU 0
[ 0.188060] smpboot: CPU0: Intel® Xeon® CPU E5-2676 v3 @ 2.40GHz (fam: 06, model: 3f, stepping: 02)
[ 0.192035] cpu 0 spinlock event irq 53
[ 0.193042] Performance Events: unsupported p6 CPU model 63 no PMU driver, software events only.
[ 0.196667] NMI watchdog: disabled (cpu0): hardware events not enabled
[ 0.197013] NMI watchdog: Shutting down hard lockup detector on all cpus
[ 0.198109] installing Xen timer for CPU 1
[ 0.199077] x86: Booting SMP configuration:
[ 0.200010] … node #0, CPUs: #1cpu 1 spinlock event irq 59
[ 0.206031] x86: Booted up 1 node, 2 CPUs
[ 0.207008] smpboot: Total of 2 processors activated (9600.32 BogoMIPS)
[ 0.208399] devtmpfs: initialized
[ 0.212309] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 1911260446275000 ns
[ 0.213092] pinctrl core: initialized pinctrl subsystem
[ 0.214233] NET: Registered protocol family 16
[ 0.221011] cpuidle: using governor ladder
[ 0.229011] cpuidle: using governor menu
[ 0.232135] ACPI: bus type PCI registered
[ 0.235008] acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5
[ 0.242097] dca service started, version 1.12.1
[ 0.248387] PCI: Using configuration type 1 for base access
[ 0.258074] ACPI: Added _OSI(Module Device)
[ 0.259013] ACPI: Added _OSI(Processor Device)
[ 0.260008] ACPI: Added _OSI(3.0 SCP Extensions)
[ 0.261007] ACPI: Added OSI(Processor Aggregator Device)
[ 0.266702] ACPI: Interpreter enabled
[ 0.267011] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [_S1
] (20150619/hwxface-580)
[ 0.270014] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [_S2
] (20150619/hwxface-580)
[ 0.273025] ACPI: (supports S0 S3 S4 S5)
[ 0.274007] ACPI: Using IOAPIC for interrupt routing
[ 0.275052] PCI: Using host bridge windows from ACPI; if necessary, use “pci=nocrs” and report a bug
[ 0.351822] ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff])
[ 0.352016] acpi PNP0A03:00: _OSC: OS supports [ASPM ClockPM Segments MSI]
[ 0.353022] acpi PNP0A03:00: _OSC failed (AE_NOT_FOUND); disabling ASPM
[ 0.354077] acpi PNP0A03:00: fail to add MMCONFIG information, can’t access extended PCI configuration space under this bridge.
[ 0.356020] acpiphp: Slot [0] registered
[ 0.358637] acpiphp: Slot [3] registered
[ 0.359808] acpiphp: Slot [4] registered
[ 0.360410] acpiphp: Slot [5] registered
[ 0.362064] acpiphp: Slot [6] registered
[ 0.363899] acpiphp: Slot [7] registered
[ 0.365011] acpiphp: Slot [8] registered
[ 0.366383] acpiphp: Slot [9] registered
[ 0.367791] acpiphp: Slot [10] registered
[ 0.368423] acpiphp: Slot [11] registered
[ 0.369751] acpiphp: Slot [12] registered
[ 0.370423] acpiphp: Slot [13] registered
[ 0.371896] acpiphp: Slot [14] registered
[ 0.372991] acpiphp: Slot [15] registered
[ 0.373523] acpiphp: Slot [16] registered
[ 0.375076] acpiphp: Slot [17] registered
[ 0.376450] acpiphp: Slot [18] registered
[ 0.377703] acpiphp: Slot [19] registered
[ 0.379027] acpiphp: Slot [20] registered
[ 0.380448] acpiphp: Slot [21] registered
[ 0.381410] acpiphp: Slot [22] registered
[ 0.382723] acpiphp: Slot [23] registered
[ 0.383440] acpiphp: Slot [24] registered
[ 0.385006] acpiphp: Slot [25] registered
[ 0.386388] acpiphp: Slot [26] registered
[ 0.387797] acpiphp: Slot [27] registered
[ 0.388940] acpiphp: Slot [28] registered
[ 0.389398] acpiphp: Slot [29] registered
[ 0.390521] acpiphp: Slot [30] registered
[ 0.391420] acpiphp: Slot [31] registered
[ 0.392743] PCI host bridge to bus 0000:00
[ 0.393010] pci_bus 0000:00: root bus resource [bus 00-ff]
[ 0.394010] pci_bus 0000:00: root bus resource [io 0x0000-0x0cf7 window]
[ 0.395011] pci_bus 0000:00: root bus resource [io 0x0d00-0xffff window]
[ 0.396013] pci_bus 0000:00: root bus resource [mem 0x000a0000-0x000bffff window]
[ 0.397014] pci_bus 0000:00: root bus resource [mem 0xf0000000-0xfbffffff window]
[ 0.409734] pci 0000:00:01.1: legacy IDE quirk: reg 0x10: [io 0x01f0-0x01f7]
[ 0.410010] pci 0000:00:01.1: legacy IDE quirk: reg 0x14: [io 0x03f6]
[ 0.411012] pci 0000:00:01.1: legacy IDE quirk: reg 0x18: [io 0x0170-0x0177]
[ 0.412009] pci 0000:00:01.1: legacy IDE quirk: reg 0x1c: [io 0x0376]
[ 0.414175] * Found PM-Timer Bug on the chipset. Due to workarounds for a bug,
[ 0.414175] * this clock source is slow. Consider trying other clock sources
[ 0.417783] pci 0000:00:01.3: quirk: [io 0xb000-0xb03f] claimed by PIIX4 ACPI
[ 0.429333] ACPI: PCI Interrupt Link [LNKA] (IRQs *5 10 11)
[ 0.431607] ACPI: PCI Interrupt Link [LNKB] (IRQs 5 *10 11)
[ 0.433313] ACPI: PCI Interrupt Link [LNKC] (IRQs 5 10 *11)
[ 0.436000] ACPI: PCI Interrupt Link [LNKD] (IRQs *5 10 11)
[ 0.465950] ACPI: Enabled 2 GPEs in block 00 to 0F
[ 0.466072] xen:balloon: Initialising balloon driver
[ 0.471036] xen_balloon: Initialising balloon driver
[ 0.478075] init_memory_mapping: [mem 0x110000000-0x117ffffff]
[ 0.478107] vgaarb: setting as boot device: PCI:0000:00:02.0
[ 0.478109] vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none
[ 0.478109] vgaarb: loaded
[ 0.478110] vgaarb: bridge control possible 0000:00:02.0
[ 0.483000] SCSI subsystem initialized
[ 0.488037] ACPI: bus type USB registered
[ 0.489034] usbcore: registered new interface driver usbfs
[ 0.490023] usbcore: registered new interface driver hub
[ 0.491049] usbcore: registered new device driver usb
[ 0.492113] PCI: Using ACPI for IRQ routing
[ 0.494062] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[ 0.495021] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[ 0.497400] hpet0: 3 comparators, 64-bit 62.500000 MHz counter
[ 0.499037] clocksource: Switched to clocksource xen
[ 0.510748] pnp: PnP ACPI init
[ 0.514956] system 00:00: [mem 0x00000000-0x0009ffff] could not be reserved
[ 0.524791] system 00:01: [io 0x08a0-0x08a3] has been reserved
[ 0.532441] system 00:01: [io 0x0cc0-0x0ccf] has been reserved
[ 0.539356] system 00:01: [io 0x04d0-0x04d1] has been reserved
[ 0.546872] system 00:07: [io 0x10c0-0x1141] has been reserved
[ 0.554462] system 00:07: [io 0xb044-0xb047] has been reserved
[ 0.595214] pnp: PnP ACPI: found 8 devices
[ 0.607315] clocksource: acpi_pm: mask: 0xffffff max_cycles: 0xffffff, max_idle_ns: 2085701024 ns
[ 0.614650] NET: Registered protocol family 2
[ 0.616915] TCP established hash table entries: 32768 (order: 6, 262144 bytes)
[ 0.624582] TCP bind hash table entries: 32768 (order: 7, 524288 bytes)
[ 0.631474] TCP: Hash tables configured (established 32768 bind 32768)
[ 0.639382] UDP hash table entries: 2048 (order: 4, 65536 bytes)
[ 0.646361] UDP-Lite hash table entries: 2048 (order: 4, 65536 bytes)
[ 0.656512] NET: Registered protocol family 1
[ 0.663019] pci 0000:00:00.0: Limiting direct PCI/PCI transfers
[ 0.668914] pci 0000:00:01.0: PIIX3: Enabling Passive Release
[ 0.676472] pci 0000:00:01.0: Activating ISA DMA hang workarounds
[ 0.683435] Unpacking initramfs…
[ 4.426088] random: nonblocking pool is initialized
[ 5.671823] Freeing initrd memory: 27292K (ffff880034aa2000 - ffff880036549000)
[ 5.683812] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 5.690748] software IO TLB [mem 0xec000000-0xf0000000] (64MB) mapped at [ffff8800ec000000-ffff8800efffffff]
[ 5.708822] RAPL PMU detected, API unit is 2^-32 Joules, 3 fixed counters 655360 ms ovfl timer
[ 5.742023] hw unit of domain pp0-core 2^-14 Joules
[ 5.761190] hw unit of domain package 2^-14 Joules
[ 5.782988] hw unit of domain dram 2^-16 Joules
[ 5.796437] futex hash table entries: 4096 (order: 6, 262144 bytes)
[ 5.815173] Initialise system trusted keyring
[ 5.828663] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[ 5.852704] zpool: loaded
[ 5.861065] VFS: Disk quotas dquot_6.6.0
[ 5.875245] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[ 5.893529] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[ 5.918729] io scheduler noop registered
[ 5.930566] io scheduler deadline registered
[ 5.940489] io scheduler cfq registered (default)
[ 5.958746] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[ 5.979302] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[ 6.041935] Console: switching to colour frame buffer device 100x37
[ 6.070012] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input0
[ 6.089617] ACPI: Power Button [PWRF]
[ 6.106919] input: Sleep Button as /devices/LNXSYSTM:00/LNXSLPBN:00/input/input1
[ 6.133693] ACPI: Sleep Button [SLPF]
[ 6.145287] GHES: HEST is not enabled!
[ 6.157626] ioatdma: Intel® QuickData Technology Driver 4.00
[ 6.183892] xen:grant_table: Grant tables using version 1 layout
[ 6.206271] Grant table initialized
[ 6.209133] Cannot get hvm parameter CONSOLE_EVTCHN (18): -22!
[ 6.230958] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[ 6.310237] 00:06: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[ 6.338697] Linux agpgart interface v0.103
[ 6.341867] [drm] Initialized drm 1.1.0 20060810
[ 6.345485] tun: Universal TUN/TAP device driver, 1.6
[ 6.349245] tun: © 1999-2004 Max Krasnyansky maxk@qualcomm.com
[ 6.353814] ehci_hcd: USB 2.0 ‘Enhanced’ Host Controller (EHCI) Driver
[ 6.358656] ehci-pci: EHCI PCI platform driver
[ 6.362132] ehci-platform: EHCI generic platform driver
[ 6.366303] i8042: PNP: PS/2 Controller [PNP0303:PS2K,PNP0f13:PS2M] at 0x60,0x64 irq 1,12
[ 6.375631] serio: i8042 KBD port at 0x60,0x64 irq 1
[ 6.379620] serio: i8042 AUX port at 0x60,0x64 irq 12
[ 6.385259] rtc_cmos 00:02: rtc core: registered rtc_cmos as rtc0
[ 6.386251] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input2
[ 6.396680] rtc_cmos 00:02: alarms up to one day, 114 bytes nvram, hpet irqs
[ 6.402071] sdhci: Secure Digital Host Controller Interface driver
[ 6.407352] sdhci: Copyright© Pierre Ossman
[ 6.411106] NET: Registered protocol family 10
[ 6.415263] NET: Registered protocol family 17
[ 6.418800] 9pnet: Installing 9P2000 support
[ 6.422220] Loading compiled-in X.509 certificates
[ 6.425858] registered taskstats version 1
[ 6.430928] zswap: loading zswap
[ 6.434611] zpool: no driver for type zbud
[ 6.438342] zswap: zbud zpool not available
[ 6.443082] zswap: zpool creation failed
[ 6.708122] tsc: Refined TSC clocksource calibration: 2399.999 MHz
[ 6.713118] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x2298364cab5, max_idle_ns: 440795214892 ns
[ 11.547102] xenbus_probe_frontend: Waiting for devices to initialise: 25s…20s…15s…10s…5s…0s…
[ 36.452572]
[ 36.453838] xenbus_probe_frontend: Timeout connecting to device: device/vfb/0 (local state 3, remote state 1)
[ 36.458871] xenbus_probe_frontend: Device with no driver: device/vbd/768
[ 36.462707] xenbus_probe_frontend: Device with no driver: device/vif/0
[ 36.466251] rtc_cmos 00:02: setting system clock to 2016-07-25 12:33:21 UTC (1469450001)
[ 36.473077] Freeing unused kernel memory: 1264K (ffffffff81ae4000 - ffffffff81c20000)
[ 36.478691] Write protecting the kernel read-only data: 10240k
[ 36.483354] Freeing unused kernel memory: 324K (ffff8800015af000 - ffff880001600000)
[ 36.489108] Freeing unused kernel memory: 956K (ffff880001911000 - ffff880001a00000)
e[34mINFOe[0m[0000] Launching Bootstrap Docker
[ 36.574688] bridge: automatic filtering via arp/ip/ip6tables has been deprecated. Update your scripts to load br_netfilter if you need this.
e[34mINFOe[0m[0000] Set 127.0.0.1/8 on lo
e[34mINFOe[0m[0000] Waiting for Docker at unix:///var/run/system-docker.sock
[ 36.620472] Bridge firewalling registered
[ 36.624635] nf_conntrack version 0.5.0 (16384 buckets, 65536 max)
[ 36.629815] ip_tables: © 2000-2006 Netfilter Core Team
e[34mINFOe[0m[0000] Connected to Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0000] Loading images from /usr/share/ros/images.tar
e[34mINFOe[0m[0001] Done loading images from /usr/share/ros/images.tar
e[34mINFOe[0m[0001] Running Bootstrap services
e[34mINFOe[0m[0001] Project [bootstrap]: Starting project
e[34mINFOe[0m[0001] [0/2] [udev-bootstrap]: Starting
[ 38.321834] cgroup: docker (114) created nested cgroup for controller “memory” which has incomplete hierarchy support. Nested cgroups may change behavior in the future.
[ 38.328341] cgroup: “memory” requires setting use_hierarchy to 1 on the root
[ 38.368537] udevd[8]: starting version 3.1.5
[ 38.418130] scsi host0: ata_piix
[ 38.423295] microcode: CPU0 sig=0x306f2, pf=0x1, revision=0x25
[ 38.427841] xen_netfront: Initialising Xen virtual ethernet driver
[ 38.433919] microcode: CPU1 sig=0x306f2, pf=0x1, revision=0x25
[ 38.436331] microcode: Microcode Update Driver: v2.00 tigran@aivazian.fsnet.co.uk, Peter Oruba
[ 38.447265] scsi host1: ata_piix
[ 38.449742] ata1: PATA max MWDMA2 cmd 0x1f0 ctl 0x3f6 bmdma 0xc100 irq 14
[ 38.451180] input: ImExPS/2 Generic Explorer Mouse as /devices/platform/i8042/serio1/input/input4
[ 38.461298] ata2: PATA max MWDMA2 cmd 0x170 ctl 0x376 bmdma 0xc108 irq 15
[ 38.470071] AVX2 version of gcm_enc/dec engaged.
[ 38.472548] AES CTR mode by8 optimization enabled
[ 38.478171] piix4_smbus 0000:00:01.3: SMBus base address uninitialized - upgrade BIOS or use force_addr=0xaddr
[ 38.484617] mousedev: PS/2 mouse device common for all mice
[ 38.513857] blkfront: xvda: barrier or flush: disabled; persistent grants: disabled; indirect descriptors: enabled;
[ 38.521045] xvda: xvda1
[ 38.625143] udevd[26]: starting version 3.1.5
e[34mINFOe[0m[0002] [1/2] [udev-bootstrap]: Started
e[34mINFOe[0m[0002] [1/2] [state-script]: Starting
e[34mINFOe[0m[0002] [2/2] [state-script]: Started
e[34mINFOe[0m[0002] Project [bootstrap]: Project started
e[34mINFOe[0m[0002] Mounting state device /dev/xvda1 to /state
[ 38.992837] EXT4-fs (xvda1): recovery complete
[ 38.997509] EXT4-fs (xvda1): mounted filesystem with ordered data mode. Opts: (null)
e[34mINFOe[0m[0002] Launching System Docker
e[34mINFOe[0m[0002] Set 172.18.42.1/16 on docker-sys
e[34mINFOe[0m[0000] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0000] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0000] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0000] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0001] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0003] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0005] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0007] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0009] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0011] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0013] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0015] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0017] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0019] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0021] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0023] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0025] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0027] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0029] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0031] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0033] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0035] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0037] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0039] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0041] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0043] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0045] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0047] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0049] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0051] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0053] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0055] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0057] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0059] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0061] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0063] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0065] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0067] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0069] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0071] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0073] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0075] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0077] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0079] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0081] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0083] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0085] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0087] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0089] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0091] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0093] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0095] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0097] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0099] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0101] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0103] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0105] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0107] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0109] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0111] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0113] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0115] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0117] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0119] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0121] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0123] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0125] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0127] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0129] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0131] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0133] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0135] Waiting for Docker at unix:///var/run/system-docker.sock
e[34mINFOe[0m[0137] Waiting for Docker at unix:///var/run/system-docker.sock

It’s my personal fault to add the wrong parameters to the system-docker extra_args, that caused system-docker failed to startu

The wrong cloud_init.yml config which cause the problem:

hostname: Rancher_Client02
rancher:
system_docker:
extra_args: [ --ulimit nproc=-1 ]
docker:
args: [daemon, --log-opt, max-size=25m, --log-opt, max-file=2, -s, overlay, -G, docker, -H, ‘unix:///var/run/docker.sock’, --insecure-registry, ‘docker-dev.qregistry.local’, --insecure-registry, ‘docker-product.qregistry.local’]

And these two lines:

System_docker:
extra_args: [ --ulimit nproc=-1 ]

Should be deleted from cloud_init.yml!