Anyone, Pls give me some step by step procedure on doing forensics for my suspected compromised server. I have been researching this for sometimes and i cant find a step by step procedure on doing that. Also i want to use any open source tools that are avialable for doing this procedure.

Take a look at post 5 of the security references thread by unSpawn. The are a number of forensics HOWTOs, including a step-by-step from CERT:

http://www.cert.org/tech_tips/root_compromise.html

The security references thread also has an extensive number of links to forensic tools and utils that should help you out. If you have any specific questions or need advice on doing the analysis, feel free to post them.

Tnx Capt_caveman.. I will try to look at it.

ran across these the other day in the news. might help you out.

'Forensic Analysis of a Live Linux System, Part One'
http://www.securityfocus.com/infocus/1769

'Forensic Analysis of a Live Linux System, Part Two'
http://www.securityfocus.com/infocus/1773

Mcalizo, I think forensics could be easier when you tap into the collective power of LQ, so could you at least tell us the symptoms the box displays or alerts you've gotten?

unSpawn i got a RH9.0 web server, last week i run chkrootkit (as i always do daily), i found that su is infected. I also run rkhunter to verify the result and the same output comes out. But the worse case is there's a lot of spoof IP trying to connect to that server when i run iptstate and iptraf. So i isolated that server and replace another "hardened" server, but still i can monitor a lot of attempted connection using port 80. Those connection ( though not sucessfull) make our bandwitdh utilization to maximum.

last week i run chkrootkit (as i always do daily), i found that su is infected.
If you check Chkrootkit it'll do a grep. Could you please post the output of running:
strings -an1 | egrep -ie "(satori|vejeta|conf)"
This should rule out false positives.


I also run rkhunter to verify the result and the same output comes out.
If you check Rootkit Hunter it'll do a md5sum check, and that may not match your su version. Could you please post the output of running this, it's a quick way of checking md5sum with rpm contents (sh-utils from running "rpm -q --whatprovides /bin/su"):
rpm -q --dump sh-utils | grep "n/s"
md5sum /bin/su


So i isolated that server
How did you isolate it (yank/reconnect network cable, power down, other)? Where is it located (isolated subnet?)? Is it still up without reboot? Any chance of getting account, process and network details? Logs? If you powered it down I hope you didn't reboot it.
[EDIT]
I mean a dead server should be left dead until the coroner has done it's job. Resurrection could result in all sorts of weirdness (or nothing, but are you willing to take the chance...).
[/EDIT]


but still i can monitor a lot of attempted connection using port 80. Those connection ( though not sucessfull) make our bandwitdh utilization to maximum.
Do you run an IDS (like Snort or Prelude)? Or Ethereal or tcpdump? Any chance of getting tcpdump output?


BTW, did you read the stuff CC told you about?