Upgraded from SLES 12 SP4 + HA to SLES 15 + HA
Using DRBD on devices & HA cluster to have HA on the DRBD devices
after the upgrade, DRBD will not start
ISSUE
nss-sn01:/ # drbdadm status
modinfo: ERROR: Module drbd not found.
modinfo: ERROR: Module drbd not found.
modprobe: FATAL: Module drbd not found in directory /lib/modules/4.12.14-95.3-default
Failed to modprobe drbd (No such file or directory)
Command ‘drbdsetup status’ terminated with exit code 20
command status exited with code 20
nss-sn01:/ # systemctl status drbd -l
â drbd.service - DRBD – please disable. Unless you are NOT using a cluster manager.
Loaded: loaded (/usr/lib/systemd/system/drbd.service; disabled; vendor preset: disabled)
Active: failed (Result: exit-code) since Wed 2019-02-13 22:05:13 MST; 13min ago
Main PID: 13357 (code=exited, status=5)
Feb 13 22:05:13 nss-sn01 systemd[1]: Starting DRBD – please disable. Unless you are NOT using a cluster manager…
Feb 13 22:05:13 nss-sn01 drbd[13357]: /lib/drbd/drbd: line 58: local: can only be used in a function
Feb 13 22:05:13 nss-sn01 modprobe[13370]: FATAL: Module drbd not found in directory /lib/modules/4.12.14-95.3-default
Feb 13 22:05:13 nss-sn01 drbd[13357]: Can not load the drbd module.
Feb 13 22:05:13 nss-sn01 systemd[1]: drbd.service: Main process exited, code=exited, status=5/NOTINSTALLED
Feb 13 22:05:13 nss-sn01 systemd[1]: Failed to start DRBD – please disable. Unless you are NOT using a cluster manager…
Feb 13 22:05:13 nss-sn01 systemd[1]: drbd.service: Unit entered failed state.
Feb 13 22:05:13 nss-sn01 systemd[1]: drbd.service: Failed with result ‘exit-code’.
nss-sn01:/ # uname -a
Linux nss-sn01 4.12.14-95.3-default #1 SMP Wed Dec 5 06:00:48 UTC 2018 (63a8d29) x86_64 x86_64 x86_64 GNU/Linux
nss-sn01:/ # cat /etc/os-release
NAME=“SLES”
VERSION=“15”
VERSION_ID=“15”
PRETTY_NAME=“SUSE Linux Enterprise Server 15”
ID=“sles”
ID_LIKE=“suse”
ANSI_COLOR=“0;32”
CPE_NAME=“cpe:/o:suse:sles:15”
Don’t know if this has anything to do with this, but some files are missing compared to a working SLES12 SP4 + HA
nss-sn01:/lib/modules/4.12.14-95.3-default # l weak-updates/updates/
total 4
drwxr-xr-x 1 root root 16 Feb 13 21:00 ./
drwxr-xr-x 1 root root 14 Dec 14 18:47 …/
lrwxrwxrwx 1 root root 51 Dec 14 18:47 crash.ko → /lib/modules/4.12.14-94.41-default/updates/crash.ko
nss-sn01:/lib/modules/4.12.14-95.3-default #
–
used yast to updated the drbd modules - dd not help
It appears that in the past few days you have not received a response to your
posting. That concerns us, and has triggered this automated reply.
These forums are peer-to-peer, best effort, volunteer run and that if your issue
is urgent or not getting a response, you might try one of the following options:
Visit http://www.suse.com/support and search the knowledgebase and/or check all
the other support options available.
If this is a reply to a duplicate posting or otherwise posted in error, please
ignore and accept our apologies and rest assured we will issue a stern reprimand
to our posting bot…
FATAL: Module drbd not found in directory /lib/modules/4.12.14-95.3-default
what does “rpm -ql drbd-kmp-default-9.0.13+git.b83ade31_k4.12.14_23-3.2.1.x86_64” report? That the file is placed in /lib/modules/4.12.14-23*?
I have not installed SLES15 + HAE yet, but with earlier versions, there was a separate kernel shipped via HAE. Could you please check via according calls to “zypper se -s -i drbd” and “zypper se -s -ikernel”, that the packages come from the same (HAE) repository / channel?
What also caught my eye is the version mismatch 9.0.13 vs. 9.3.1:
S | Name | Type | Version | Arch | Repository
—±-----------------±--------±--------------------------------------±-------±------------------------
i+ | drbd | package | 9.0.13+git.b83ade31-3.2.1 | x86_64 | SLE-Product-HA15-Updates
i | drbd-kmp-default | package | 9.0.13+git.b83ade31_k4.12.14_23-3.2.1 | x86_64 | SLE-Product-HA15-Updates
i | drbd-utils | package | 9.3.1-1.21 | x86_64 | SLE-Product-HA15-Pool
i+ | yast2-drbd | package | 4.0.3-3.3.1 | noarch | SLE-Product-HA15-Updates
Note: For an extended search including not yet activated remote resources please use ‘zypper
search-packages’.
nss-sn01:~ # zypper se -s -i kernel
Loading repository data…
Reading installed packages…
S | Name | Type | Version | Arch | Repository
—±------------------±--------±----------------±-------±-------------------------------
i+ | kernel-default | package | 4.12.14-95.3.1 | x86_64 | (System Packages)
i+ | kernel-default | package | 4.12.14-25.28.1 | x86_64 | SLE-Module-Basesystem15-Updates
“(System packages)” usually points at packages that are not coming from (currently connected) repositories. Might it be you upgraded the kernel package from somewhere else, i.e. a temporarily attached reposirory?
If you’re not depending on the 4.12.14-95.3.1 kernel, you might want to test downgrading to the 4.12.14-25.28.1 kernel as provided by the SLE-Module-Basesystem15-Updates repository - my hope is that the DRBD kernel module (9.0.13+git.b83ade31_k4.12.14_23-3.2.1) might work with that kernel.
If you want to have a look at where that .95 kernel caes from, you could beforehand run “rpm -qi kernel-default” and save that output somewhere.
Looking at the Grub2 menu, the older SLES 15 kernel was listed to boot from - menuentry ‘SLES 15, with Linux 4.12.14-25.28-default (recovery mode)’.
So I rebooted the system and choose - submenu ‘Advanced options for SLES 15’, then the above entry.
Now the system boot and startd DRBD
nss-sn01:~ # drbdadm status
drbd01 role:Secondary
disk:UpToDate
nss-sn02 role:Primary
peer-disk:UpToDate
looking at the install dates of these packages and the “Distribution” info, you can see that the 4.12.14-95.3.1 kernel is from the old SLES12 system. With other words, it’s not “the latest kernel” with regards to your SLES15 channels.
I recommend to remove that old kernel from your system, and a call to “rpm -qa|grep 95.3.1” might reveal some other packages related to that kernel which can also be deleted. "rpm -qa |grep ‘System packages’ " might list more orphans from SLES12, which then will have to be looked at individually - it’s you to decide what to do with them :-/