SLES 11 SP2 and iSCSI HBA

SUSE Linux Enterprise Extensions SLES Hardware
1

Is there a way to insert/load modules at the boot menu right before boot? I’m not able to get into my root partition to add it to /etc/sysconfig/kernel, so I’m really talking about the boot prompt or boot command line at the main menu before SLES boots. Here are the details of the issue I’m seeing:

I’ve had no problems doing both an in-place upgrade from SLES 11 SP1 to SP2 and a fresh SP2 install on VMs. However, I’ve had trouble upgrading and fresh-installing SP2 on a physical machine (Sun Fire x4140).

The in-place upgrade on the physical machine from SP1 to SP2 went smoothly. Upon reboot though, SLES does not see the root partition. We use the iSCSI HBA to boot the OS. The qla4xxx driver seems to load up fine and it is in INITRD_MODULES. Cat /proc/partitions in SP2 rescue mode shows NO partitions, so I can’t mount up root. However, /proc/partitions in the SP1 rescue mode shows me root and I can mount it up just fine. FYI I have never had a problem with this in SP1 or pre-SP1.

A fresh install of SP2 on the physical machine had trouble as well. No hard disks were found when it went through the “probing hard disks” step. Adding withiscsi=1 to the boot line before installation took me to the special iscsi config screen right before install, where I took the defaults, and then the iSCSI lun/disk was seen. I was able to complete the install and it booted up fine.

I found two modules that need to be present in SP2 (per lsmod) that were not needed pre-SP2: iscsi_boot_sysfs and libiscsi. Next time I do an in-place install, I’ll add these into /etc/sysconfig/kernel and run mkinitrd. However, I’ve got this one machine that I screwed up that I can’t get into the root partition, and so hopefully someone can tell me how to add them at the boot menu to force them in. Thanks!

2

On another note, a few months ago we went through on our SLES 11 SP1 servers and changed the devices to use symbolic links in fstab and /boot/grub/menu.lst instead of the disk-by-id version that is default in SLES 11. This could be part of this issue, but I doubt it. I’ll check into it. The reason we changed was due to an issue encountered when moving OS’s from one filer to another, and I guess being on a new lun changed its disk id. Anyway, I don’t think this is the real issue, but thought I’d mention it.

3

Using /dev/disk/by-id/* in fstab and menu.lst did not solve the problem.

4

Per Qlogic on their forums:

SLES 11 SP2 is running the 3.x kernel, so I guess we’re dead in the water until they release an updated driver.

5

Nevermind, I was able to get this going on my own. I’m surprised no one else has this issue…

6

ashbyj,

would you mind sharing how you solved this, just in case someone with the same problem gets directed to your messages via a search?

Thanks & regards,
Jens

7

Certainly. I can’t say if all of these steps are necessary in getting your machine to boot from iSCSI, but here are the steps I take.

1) Perform the in-place upgrade of SLES 11 SP1 to SP2

Yast Software > Patch CD Update I usually go with NFS and then point to the path of my SP2 ISO

After the upgrade you need to configure the iSCSI device at the OS level and also compile some special iSCSI-related modules into the initrd.

2) Configure iSCSI device via yast

Some of this is specific to our environment

Yast
    Network Services > iSCSI Initiator

    Service Start: when booting
    Initiator name: iqn.2009-11.com.mydomain.myhost:qlogic.qla4060c.1
    Offload card: default (software)

I’m really not sure if this below was necessary, but I was matching the settings on a machine where we were successful in installing a fresh SP2:

[CODE]# via command line
cd /etc/iscsi/ifaces

add ipv4.0 suffix to the qla4xxx* iface file

mv qla4xxx.00\:c1\:aa\:0e\:cf\:ab qla4xxx.00\:c1\:aa\:0e\:cf\:ab.ipv4.0

vi eth0-qla4xxx

added this line which is the iSCSI initiator client IP

iface.ipaddress = 172.12.211.2
[/CODE]

3) add modules to initrd

[CODE]
vi /etc/sysconfig/kernel

change initrd_modules line to:

INITRD_MODULES=“sata_nv ata_generic iscsi_boot_sysfs bnx2i”

The two most important modules above I believe to be iscsi_boot_sysfs AND bnx2i

mkinitrd[/CODE]

If you encounter perl-bootloader errors when running mkinitrd, do these steps:

[CODE]cd /boot/grub/
mv device.map device.map.old
grub

At the grub> prompt, type quit

Rerun mkinitrd, it should no longer have the perl-bootloader errors.[/CODE]

4) miscellaneous steps

[CODE]# We disable nscd on our servers, and SP2 seems to reenable it.
/etc/init.d/nscd stop
chkconfig nscd off

rm /etc/ssh/ldap.conf # not sure why ldap.conf gets put in /etc/ssh after SP2 upgrade… could just be me.[/CODE]

5) Reboot and cross your fingers

We’ve never had to configure iSCSI at the software level before, so I’m not sure what is different about the bootloader that doesn’t play nicely with our qlogic card.

8

Thanks for your detailled response and your time to write it! From my own experience I’m sure there will be more than one “just-reader” that will be grateful to find you description and be able to fix his/her problem thanks to it.

Many regards
Jens

9

Novell/SuSE finally published some information about the change to the way Qlogic iscsi is handled, 3 months after SP2 was released :(. Anyway, If you came looking for help on this, my notes above may be helpful when accompanied with their doc:

How to configure and run QLogic Open-iSCSI on SLES11 SP2

FYI

My notes above show you how to offload the iscsi work onto the software (now the default in SP2), but their doc does show how to offload onto hardware (done by default pre-SP2), but I have not tried the latter.