I will post speicific informations next time.
OK. Do a few things *at least*:
1. Remote syslog. Do this first. If the customers network allows outbound and you can set up a single purpose expendable DMZ box configured as a listening syslogd on your side, do so. At least then you have a remote (heh) chance of logging. Make sure it logs everything. If you can't syslog out of the customers network see if any of the other boxen can serve as remote syslogd but verify the integrity of that box first. Now test and watch syslogd output for a bit. If there's too many unexplainable errors skip below and proceed with the Intruder Detection Checklist (CERT):
http://www.cert.org/tech_tips/intrud...checklist.html.
2. Execute "cp /dev/sdN /dev/null" where N is the int of the disk the /home partition is on. Inspect the log. If there's no errors you don't have device but filesystem errors.
3. Verify the system with "rpm --verify". If there's too many unexplainable errors skip the rest and proceed with the Intruder Detection Checklist.
Whatever you do try to start by leaving an audit trail when you access root there. You choice of using (trusting or placing) stuff on the system like running the shell through "script" or "screen" with logging on (commandkey-shift-h by default) or "sudosh". If unsure always pipe command output through "2>&1 | tee -a /dev/shm/log.tee" to capture output.
Dismissing errors w/o investigation and mitigation is a serious situation. I would never tolerate an excuse like "the server is currently in use" for doing nothing. And letting it linger over a three month period is, well, I'm skip posting what I think of that.