LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Enterprise Linux Forums > Linux - Enterprise
User Name
Password
Linux - Enterprise This forum is for all items relating to using Linux in the Enterprise.

Notices


Reply
  Search this Thread
Old 02-27-2007, 03:02 AM   #1
jhanx
LQ Newbie
 
Registered: Feb 2007
Posts: 2

Rep: Reputation: 0
All files are lost!


Hello guys! I am having this problem with one of our servers.

When I execute ls -l in /tmp and /var/log directories, no files are
displayed, not even a single one. Even (.) and (..) are not displayed.
But if I change to other directories, there is no problem..
I also tried to execute crontab -e as an ordinary user, it displayed
an error(Input error).

Maybe somebody knows what could be the probable cause of this problem.
Please help!
 
Old 02-27-2007, 06:53 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
I am having this problem with one of our servers.
What is the purpose of the server, who has access to it, what's the distro+release?
When did this start happening, any known events leading up to this and what do the logs say?


When I execute ls -l in /tmp and /var/log directories, no files are
displayed, not even a single one. Even (.) and (..) are not displayed.

What filesystem, mounted how and did you fsck the filesystems?


I also tried to execute crontab -e as an ordinary user, it displayed
an error(Input error).

Please inspect the crontab as root user and post exact command and output for the unprivileged username?
 
Old 02-27-2007, 07:45 PM   #3
jhanx
LQ Newbie
 
Registered: Feb 2007
Posts: 2

Original Poster
Rep: Reputation: 0
distro+release : Red Hat Advanced Server release 3(Taroon Update 5)
Kernel 2.4.21.32.ELsmp on an i686.
This server is part of a management system.
Only 3 engineers(including me) have access to this machine.
There are three harddisks installed: sda, sdb and sdc which contain the OS installation,
java application, and database, respectively.
sda has root,swap and /boot partions only,
sda and sdb have 1 partition each(all fs are ext3).

I can't view the system logs since there are no files under /var/log.
Actually some problem started last year, around November, when suddenly
executing ls under /home/${USER} will result in I/O Error message.
We were also unable to create a file under this directory.
But this did not affect our java application.
We just restarted the server recently and everything seemed fine,
until we found this current issue. We did not do anything yet because the server
is currently in use. I am unable to log in to the customer network as of the
moment. I will post speicific informations next time.

Last edited by jhanx; 02-27-2007 at 07:48 PM.
 
Old 02-28-2007, 04:49 PM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
I will post speicific informations next time.
OK. Do a few things *at least*:
1. Remote syslog. Do this first. If the customers network allows outbound and you can set up a single purpose expendable DMZ box configured as a listening syslogd on your side, do so. At least then you have a remote (heh) chance of logging. Make sure it logs everything. If you can't syslog out of the customers network see if any of the other boxen can serve as remote syslogd but verify the integrity of that box first. Now test and watch syslogd output for a bit. If there's too many unexplainable errors skip below and proceed with the Intruder Detection Checklist (CERT): http://www.cert.org/tech_tips/intrud...checklist.html.
2. Execute "cp /dev/sdN /dev/null" where N is the int of the disk the /home partition is on. Inspect the log. If there's no errors you don't have device but filesystem errors.
3. Verify the system with "rpm --verify". If there's too many unexplainable errors skip the rest and proceed with the Intruder Detection Checklist.

Whatever you do try to start by leaving an audit trail when you access root there. You choice of using (trusting or placing) stuff on the system like running the shell through "script" or "screen" with logging on (commandkey-shift-h by default) or "sudosh". If unsure always pipe command output through "2>&1 | tee -a /dev/shm/log.tee" to capture output.

Dismissing errors w/o investigation and mitigation is a serious situation. I would never tolerate an excuse like "the server is currently in use" for doing nothing. And letting it linger over a three month period is, well, I'm skip posting what I think of that.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
lost files herjoc Linux - General 6 09-29-2005 09:28 PM
Lost files gauge73 Linux - General 1 07-18-2005 03:57 PM
Lost files? ParadiseNZ Linux - Software 2 04-19-2005 10:11 PM
lost files when mv ewlnxnewB Linux - Newbie 3 11-20-2003 03:14 PM
lost files still there... acid_kewpie Linux - General 1 08-07-2001 07:54 PM

LinuxQuestions.org > Forums > Enterprise Linux Forums > Linux - Enterprise

All times are GMT -5. The time now is 05:20 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration