I have had a problem for a while now with the /etc/passwd file being deleted on a couple of servers, SLES 11 SP2 on System z, that run
IBM’s WebSphere Application Server, 7.0. WAS runs as root. Note that I didn’t set up the server, only inherited its support.
The problem was occurring about once every other month for close to a year and then stopped happening. Of course, it stopped after I
implemented the auditd service and added a rule to monitor /etc/passwd.
I received a call this morning that the users were unable to PuTTY into one of the two Test servers. I determined that the /etc/passwd
file no longer existed. I shut down (crashed) the server in question and jumped thru some hoops to copy the passwd file from the other
Test server. When I rebooted all was well.
I ran command ‘ausearch -i -f /etc/passwd’ and found that there are 4 entries from yesterday, 11/04, that match. The previous entry is
from 09/17 when the passwd file was modified.
Here are the entries:
type=PATH msg=audit(11/04/12 16:50:40.578:73572) : item=1
name=/xs2files/CustomerMisc/CPREG/OriginalLists/…/…/…/…/…/…/…/…/…/…/…/…/etc/passwd inode=339678 dev=5e:01 mode=file,644
ouid=root ogid=root rdev=00:00
type=PATH msg=audit(11/04/12 16:50:40.578:73572) : item=0
name=/xs2files/CustomerMisc/CPREG/OriginalLists/…/…/…/…/…/…/…/…/…/…/…/…/etc/ inode=336529 dev=5e:01 mode=dir,755 ouid=root
ogid=root rdev=00:00
type=CWD msg=audit(11/04/12 16:50:40.578:73572) : cwd=/opt/IBM/WebSphere/AppServer/profiles/AppSrv01
type=SYSCALL msg=audit(11/04/12 16:50:40.578:73572) : arch=s390x syscall=unlink success=yes exit=0 a0=82a5bdf0 a1=3ffe2835b22 a2=628d8578
a3=20 items=2 ppid=1 pid=49827 auid=wcs4542 uid=root gid=root euid=root suid=root fsuid=root egid=root sgid=root fsgid=root tty=(none)
ses=10699 comm=java exe=/opt/IBM/WebSphere/AppServer/java/jre/bin/java key=fk_passwd
I am assuming that all of these entries are related as they have the same timestamp, 11/04/12 16:50:40.578, and the same appended text,
73572.
In the first two entries the name=/xs2files/… is the path to a directory structure where the application run under WAS stores data.
These entries are showing up, I think, because the application tried to access something in the directory structure that contained
‘/etc/passwd’.
The third entry is identifying the current working directory for WebSphere.
I interpret the last entry as being that java, comm= and exe=, deleted /etc/passwd, syscall=unlink success=yes. The
‘fk_passwd’ in key= parameter is what I specified as the filter key when I set up the rule with auditctl.
If my interpretation that is correct, what is meant by auid=wcs4542? I found out from the team leader for WebSphere development
that user wcs4542 is out of the country and didn’t take his laptop with him. Could his userid be in the entry because he was the last one
who stopped and started WebSphere?
Is there something I’m not interpreting correctly? Is there a way to determine who/what caused the file to be deleted? I’ve checked for
cron entries that could delete the file in /etc/crontab and there aren’t any that use java in the path specified by exe=. There aren’t
any crontab entries for any user in /var/spool/cron/tab.
Any help will be appreciated.