Have a SuSE10sp4 server that is also running OES2 on it, that boots into read only mode only. This is a virtual server on a VMWare server. This virtual box has had power issues of late. Two other SuSE10 servers that had same issue was repaired booting off media and running a repair of the disk, which took care of the issue, but this box file system comes back clean. Does anyone else have any ideas on what can be tried to repair this box?
jlewter wrote:
[color=blue]
Does anyone else have any ideas on what can be tried to
repair this box?[/color]
Before trying to repair it, you have to know what the problem is. 
Have you looked for any clues in /var/log/messages?
–
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…
Yes, have gone all through it but nothing sticks out to me. This server along with two other all had same issue. We believe site it having power issues due to logs on VM Host side showing it. I fixed the other two wtih disk check on drive and they had corruption so I just assumed this one had corruption too. This one came back clean though. Just file system comes up in read only state. Can’t load edirectory on it or most services due to not being able to write to logs needed. I found a post of VMWare site relating to it dealing with issue a while back, but sp2 for SuSE10 supposedly fixed this issue and since this box is sp4 I figure to rule that out. Just looking for suggestions as to if anyone had seen this issue before.
When you try to login from gui console from console in VMWare, you get error:
Error opening /var/lib/pam_devperm/:0: Read Only file system
Then it says GDM could not write to authorization file. I am assuming this is all due to read only issue. It dumps you out at console with login prompt. It want though let you login there as well saying Error in service module. It does let you login using putty though.
jlewter wrote:
[color=blue]
When you try to login from gui console from console in VMWare, you get
error:Error opening /var/lib/pam_devperm/:0: Read Only file system
Then it says GDM could not write to authorization file. I am assuming
this is all due to read only issue. It dumps you out at console with
login prompt. It want though let you login there as well saying Error
in service module. It does let you login using putty though.[/color]
Other Knowledge Partners have more experience with VMWare than I do so
I invite them to jump in if they have any ideas…
Suggestions:
Login via PuTTY.
Is the filesystem mounted “read only” on the VM?
cat /proc/mounts
If yes, what does fstab show?
cat /etc/fstab
Assuming nothing is mounted read only, can you do any writes to the
filesystem?
touch …
If things look normal, can you verify the integrity of the storage on
the VMWare host?
–
Kevin Boyle - Knowledge Partner
If you find this post helpful and are logged into the web interface,
show your appreciation and click on the star below…
“KBOYLE” wrote:
[color=blue]
Other Knowledge Partners have more experience with VMWare than I do so
I invite them to jump in if they have any ideas…[/color]
I don’t think this is VMware-related.
[color=blue]
Suggestions:
Login via PuTTY.
Is the filesystem mounted “read only” on the VM?
cat /proc/mountsIf yes, what does fstab show?
cat /etc/fstab[/color]
This is the file I’d really like to see and I’m wondering if it’s empty or
missing.
What does “mount” output?
[color=blue]
Assuming nothing is mounted read only, can you do any writes to the
filesystem?
touch …[/color]
Can you write to any partition other than / ?
[color=blue]
If things look normal, can you verify the integrity of the storage on
the VMWare host?[/color]
HTH.
Simon
SUSE Knowledge Partner
Here is a screenshot of the mount command output:
Here is output from the other two commands:
Yes, I can write to the other partitions just not /.
I have run across allot of articles pertaining to reasons file system go read only, most point to back end hardware,raid, san issues. I see allot of posts where it was fixed using this command:
mount -o remount /
Have you ever tried it before and results? Is it worth a try?
Hi jlewter,
[QUOTE=jlewter;17208]Here is a screenshot of the mount command output:
[/QUOTE]
if possible, use cut & paste to insert such textual information in a QUOTE block. Makes searching much easier…
Back on topic: quite obviously, your file system driver set the FS to read-only (it’s mounted “rw”) - details on what happened ought to be found in /var/log/messages or the output of “dmesg”.
Two other SuSE10 servers that had same issue was repaired booting off media and running a repair of the disk, which took care of the issue, but this box file system comes back clean.
Had you forced the fsck on the server in question?
Regards,
Jens
I just noticed that this could be hard to read What I intended to say is that it’s not some wrong mount command breaking things, since the FS is reported by “mount” as “mounted rw”… so it most probably was the FS driver that set the fs to “ro” internally because of some error, which reflects in the fact you cannot write to that FS and that /proc/mounts shows the fs as “ro” (I never noticed that such a difference may be displayed - I was under the impression that “mount” uses that information from /proc/mounts… but hey, it’s SLES10 
).
Regards,
Jens
Yes, I booted off media into rescue mode and ran it manually. The file system came back clean. The other two boxes that where having this issue was fixed by running the fsck.
I ran the fsck using this format:
fsck -y /dev/sda3
Like I said, it came back clean.
Hi jlewter,
fsck -y /dev/sda3
I strongly suggest to re-run with “-f” to force an actual check of the file system:
From “man fsck.ext3”:
It may have happened that some part of the file system got corrupted during the power hit, although the file system itself recorded a proper shutdown in the meta data.
Regards,
Jens
Ok, so run it like this:
fsck -f /dev/sda3
Can you run it with the -y as well so it auto answers the prompts?
I believe that did it. I ran it again with the -f option and it found issues and appeared to correct it. It loaded up normal and now services appears to be running correctly. Going to monitor for now, but appears better. Thanks for the help.
Hi jlewter,
good to hear your server is up & running again 
Thanks for reporting back.
Regards,
Jens