Hi Gurus,

I faced a issue yesterday where i found that the System Memory utilization went upto 99%. I was looking at the SAR reports to find these stats where it showed continuos Memory use between 95-99% for some 2-3 hours. I am actually trying to find out the exact root cause of the issue that casued the memory usage to spike.

I looked at /var/log/message to find if anything unusual happened and logged into it during those timings but couldn't find anything

So can you guys please suggest some more ways to troubleshoot, what others things i can look at investigate to really dig deep to catch the roots of the issue.

YOur ideas would be appreciated.

Invoke:
See if any heavy process(s) is/are running. Also what does other log files say like message.1, message.2 etc.?

Also mention your system disto, it's version as well what applications it's running on.

I am on RHEL 5.6, well top won't work as i am looking for yesterday's information only, messages.1, message.2 are involved as current messages file is having all log entries for for last 6 days.

Alright. If you've SAR scripts enabled, then you can find high memory usage details and then check logs for corresponding date or time. So did you find anything unusual in logs for that time period?

And as I said above, what application(s) is/are running on this system?

Yeah thats what i explained earlier. I did that checked with the SAR and it showed there was high RAM for some 2-3 hours but then i checked logs and it showed nothing unsual. There are some Java applicatiosn running, then TSM, HP Open View etc.

The problem is SAR is too coarse to be able to say "display the top 10 resource-hogging processes by name since 03:45 yesterday". Atop can do that but obviously that won't help you until you install and run it...

Well, yes and not quite. For Collectl to do what Atop does by default you should point to http://collectl.sourceforge.net/Process.html specifically. Given that you mentioned Collectl Mark probably will be here soon to explain ;-p