I have SUSE Linux 11 SP2. I recently tried to reboot the server and it failed, throwing errors and finally I got the prompt.
By typing ‘exit’ twice, the bootup process continued and everything worked as expected, although every reboot needs me to type that ‘exit’ twice now. I was wondering what could be causing this.
The errors in boot.log are (is there any to attach a file to this thread?!):
Boot logging started on /dev/tty1(/dev/console) at Fri Jan 4 11:07:00 2013
udevadm settle - timeout of 30 seconds reached, the event queue contains:
/sys/devices/pci0000:00/0000:00:03.0/0000:04:00.0 (1654)
/sys/devices/pci0000:00/0000:00:03.0/0000:04:00.1 (1655)
/init: line 17: cat: command not found
/init: line 17: cat: command not found
Partial mode. Incomplete logical volumes will be processed.
6 logical volume(s) in volume group “vg00” now active
Partial mode. Incomplete logical volumes will be processed.
6 logical volume(s) in volume group “vg00” now active
Partial mode. Incomplete logical volumes will be processed.
6 logical volume(s) in volume group “vg00” now active
udevadm settle - timeout of 30 seconds reached, the event queue contains:
/sys/devices/pci0000:00/0000:00:03.0/0000:04:00.0 (1654)
/sys/devices/pci0000:00/0000:00:03.0/0000:04:00.1 (1655)
/sys/devices/pci0000:00/0000:00:03.0/0000:04:00.0/host4 (2024)
/sys/devices/pci0000:00/0000:00:03.0/0000:04:00.0/host4/scsi_host/host4 (2025)
/sys/devices/pci0000:00/0000:00:03.0/0000:04:00.0/host4/bsg/fc_host4 (2026)
/sys/devices/pci0000:00/0000:00:03.0/0000:04:00.0/host4/fc_host/host4 (2027)
Trying manual resume from /dev/vg00/swap
And so on. Any help on how to troubleshoot? I think the only thing Ive done was to resize the LVM, and this is where my troubleshooting efforts are concentrated so far.
/init: line 17: cat: command not found[color=blue]
/init: line 17: cat: command not found[/color]
If errors like this one are showing up before the hard drives are mounted
I wonder if your initrd (initialization ramdisk) file is corrupt. I’ve
seen this happen a time or two when the disk was full at the time that the
mkinitrd (the command that builds this file) was executed. If this is the
case, be sure you have several MB free both in / (the root of the
filesystem) and /boot (assuming it is a separate filesystem) and then run
‘mkinitrd’ as root (I am pretty sure this is a safe thing to do, but maybe
backup your current /boot/initrd* files first). Once done, reboot and see
if that helps.
Also, post something about your free space with the following command:
Code:
df -h
If the ‘cat’ errors come after mounting the filesystems then… well, but
that’s bad.
–
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…
My version of init has a call to “cat” in line 25:
24 name=$(eval echo $name | sed "s/[-_]/\\*/g")
25 for i in `cat /proc/cmdline`
26 do
27 case $i in
28 ${name}.*=*)
29 param="${param}${param:+ }${i##${name}.}"
30 ;;
31 esac
32 done
Initrd is just a severely stripped-down Linux environment, mostly with only mission-critical files. After unpacking you might check if the “cat” binary is included:
# ls -l /tmp/xxx/bin/cat
-rwxr-xr-x 1 root root 48032 Feb 4 16:03 /tmp/xxx/bin/cat
Further diagnosis depends on the content of your initrd.
The problem is strange indeed - the reported line number is caused by the function call - seems we’re indeed talking about the cat command in line 25. It would be interesting to see what happens if you’d call init manually after a reboot, probably with shell debugging turned on.
BTW, if you reverse the steps to unpack the repos, you can create your own initrd including additional tools you’d like to see… an editor, for example… so that you’ll have a more helpful environment to debug these troubles.
Btw, when I get dropped at the shell, I only have access to a subset of commands (of which ‘ls’ and ‘cat’ are not), although it was included in my ramdisk (see previous post).
We might try a re-install of the OS. And we are also investigating HW + firmware. (then I’ll retire early haha)
Yes, I’m fully aware that the default initrd contains only the most necessary elements. But since you already unpacked the initrd, it’s only a matter of “cp” and “cpio” to enrich that (copy the wanted files to the appropriate initrd directory, repack initrd using cpio, configure grub to use your new initrd - voila).
The error message seems strange enough to make your attempts plausible… otoh, were I in your situation, I’d like to know the root cause and therefore debug the init script. Something to learn and something to find, makes a good match
Should have been more clear. The reason why I was specifying this (not cat command) is that at the start of the boot.msg, you see errors about the command ‘cat’ not found and indeed, when I get dropped to the shell, that command is not available. Hence any script needing it wont run properly.
But then its present in the ramdisk. So what the…? Am I missing properly? I recreated the ramdisk already, it cant be corrupted…
Tks, yes I curious too. Unfortunately, im not the end customer so might have to go with whatever give results.
when I get dropped to the shell, that command is not available
have you tried with path (/bin/cat), or only by specifying “cat”? I recall that when I hit that boot shell, no path settings were in effect.
Another thing that might be remotely possible: Have you checked that the architecture of the files matches the kernel? In my case, that’s a 64 bit kernel, so the “cat” binary matches:
But I couldn’t think of any standard situation that might cause that to go wrong, after all the files are copied from the live system during “mkinitrd”…
Any way this turns out: I’d be interested to hear what steps in the end got things working for you/your customer!
:S - init is a command on the linux, not sure where you are going with this…
sorry for the confusion, I meant to run this when you’re dropped to the shell prompt during boot - to debug the /init script that obviously goes wrong, leading to that situation.
[QUOTE=eolimal;11846]/bin/cat
sh: /bin/cat: Too many levels of symbolic links
Same thing with /bin/ls…[/QUOTE]
This looks more than strange. I’m without access to my test system atm, but will check tonight if any of the config script run inside init are capable of messing up the directory structure.
Does traversing the directory structure work, i.e. “cd …;cd …;cd bin” to go to the /bin directory (the initial "cd …"s are meant to get you up to the top-level directory, you of course could try a “cd /” instead)?
Did this happen with the initrd as created by mkinitrd or was this with a version enhanced by you (I guess the latter, since you tried to run “ls”)? If you would like me to take a look at that initrd, contact me via private message so we can exchange contact info (URL to download the initrd or e-mail to send the file to).
Have you double-checked that you’re actually looking at the initrd you’re using during boot? (No offense intended!)
Do you have a file system structure that would be mountable easily from within initrd’s shell (like a simple partition with ext3 file system, no LVM, no iSCSI etc)? Then you could try to put your tools (ls, vi, bash) there, mount it from the prompt and run the tools from there.
Currently, I’m getting kind of desperate. as you can tell from the nature of my questions… I assume that once the source is found, you’ll go “oh my, that cause leading to so much trouble!”… and I hope we’ll be getting you there soon.
If other measures get the machine up & running, or you have enough of my attempts, please let me know. Right now I’m mostly puzzled.
So… I dont have any conclusive answer to anyone. We decided to re-install and the issue is gone.
yes, sometimes that’s the only way to get back on track… sorry we couldn’t help to isolate the problem, but I’m glad that in the end, things at least got back to working order. And thank you for reporting back!
