Hi,
under certain circumstances i have a lot of error messages during the shutdown procedure. E.g. OOM. I’d really like to examine them further. But they appear only for a short time. Is there a way to log them ? Or does /var/log/boot.msg have a pendant for the shutdown procedure ?
What is about tty10 ? Normally kernel messages are directed to this terminal. Is there a way to capture everything written to tty10 ?
Thanks in advance.
Bernd
Hi Bernd,
[QUOTE=berndgsflinux;38242]Hi,
under certain circumstances i have a lot of error messages during the shutdown procedure. E.g. OOM. I’d really like to examine them further. But they appear only for a short time. Is there a way to log them ? Or does /var/log/boot.msg have a pendant for the shutdown procedure ?
What is about tty10 ? Normally kernel messages are directed to this terminal. Is there a way to capture everything written to tty10 ?[/QUOTE]
“tty10” is usually nothing but some syslog output stream, so you should be able to find these messages either in syslog as is or after configuring an additional output channel for these messages.
The OOMs more likely are some stderr output of a running process, which aren’t piped anywhere - so unless you can at least identify the originating process (to then redirect it’s output to some file) you’re probably out of luck.
Regards,
J
Hi J,
is there a way to capture/read everything which is written to tty1 ? I switched to tty10 during the shutdown procedure, but the system switched back to tty1. All errors seem to be written to tty1. Following the unix paradigma “everything is a file” i think it must be easy to log everything which is written to /dev/tty1 (which is also a file).
Bernd
I’m playing around with script or screen. But during the shutdown procedure the root fs is umounted. If i store the logfile on the root fs, it is like pulling the rug out from under sb.'s feet.
Create a ramdisk and store the logfile there ? But then i have to mount again a disk to copy the file permanently to a disk.
The system does not shut down completely, it resides in a shell (looks like initrd !?!).
Any ideas ?
Bernd
Hi Bernd,
[QUOTE=berndgsflinux;38247]I’m playing around with script or screen. But during the shutdown procedure the root fs is umounted. If i store the logfile on the root fs, it is like pulling the rug out from under sb.'s feet.
Create a ramdisk and store the logfile there ? But then i have to mount again a disk to copy the file permanently to a disk.
The system does not shut down completely, it resides in a shell (looks like initrd !?!).[/QUOTE]
concerning the other message, where you write about output on /dev/tty1: It may well be you’re seeing output that was written to /dev/console instead - reaching the then-current console.
If you have a way to catch the output, you might try to send it to a remote system for logging. Depending on the state of the server (IOW if the network interface is still up) you could use one of a number of transports, netcat coming to mind.
Concerning the final “shell”: I’ve not seen this before. If it’s really the initrd environment, you have a chance to re-mount file systems and then have the tools to further diagnose. Anyhow, using “env” to show the available environment variables might give a clue, i.e. $SHELL, $PPID and other vars may give an indicator of where you actually landed?
Regards,
J
I figured it out. The host is a node of a HA cluster, but currently in testing. I followed some recommendations and didn’t start the cluster (/etc/init.d/openais) automatically.
Ok. I got the trouble when i started the cluster manually via init-script and shut the host down afterwards. I forgot, that if i don’t start the cluster automatically it also does not stop automatically during shutdown. So the ressources of my cluster continued to run during shutdown. That means that i pulled the rug out from under the cluster. Linux does not like that. Then i think i have a kind of a configured kdump (how can i find that out, is this default for SLES 11 SP4 ?), which booted during the shutdown procedure (is that possible, that kdump loads a new kernel although system is going down ?). And finally i landed in initrd. I think that’s it.
Bernd
Gee, that sure’s some situation 
That means that i pulled the rug out from under the cluster. Linux does not like that.
Do you have OCFS2 cluster file systems active on that node? I remember that it’s module tended to bail out on “unclean” shutdowns.
kdump
It’s not default, but a nice feature to catch these type of situations.
[kdump] booted during the shutdown procedure (is that possible, that kdump loads a new kernel although system is going down ?)
kdump actually switches the kernel to perform it’s operations, so yes, this is how it works
And finally i landed in initrd. I think that’s it.
Maybe it’s a problem with the kdump environment instead? IIRC, kdump should automatically boot the original kernel after completing the dump, but that may depend.
Regards,
J