Having problem with /tmp "No space left on device" Error

jim.thornton · 05-22-2009, 09:28 PM

Hello:

When I login to root (via su -), I'm getting the following error:

Code:

Null message body; hope that's ok
/tmp/Rsoph96A: No space left on device

I am also getting other errors relating to the /tmp directory:
1. Uploading files via html/php fail because of this problem.
2. MySQL errors (I think 28) indicating a problem with the disk being full.

The think is that the partitions are not full. Here is the output from # df -h:

Code:

Filesystem            Size  Used Avail Use% Mounted on
/dev/simfs             40G   11G   30G  26% /
tmpfs                 3.9G  4.0M  3.9G   1% /tmp

As you can see, there is A LOT of available room.

I thought I had this fixed last week. I deleted everything in the /tmp directory and created two directories:

/tmp/sessions (chmod 777)
/tmp/uploads (chmod 777)

Then I edited the php.ini file and made sure to set these directories for save.session_path and upload_tmp_dir

It seemed to work but after a few days I'm in the same mess.

A little while ago my host for my VPS decided to run updates on all the VPS' and lock down the /tmp directories. However, in this process they messed up a LOT of programs that I had running. I spend 2 or three days completely offline, then I was finally able to get PHP re-compiled again, but was unable to get eAccelerator working correctly (I think it isn't compatible with PHP 5.2.9).

Anyway... I have no clue what they did to the /tmp directory, and I have no clue how to debug or fix this problem.

Can anyone help me with this please?

janhe · 05-23-2009, 10:34 AM

Your /tmp is a tmpfs, that means it isn't on your hard disk, but it is using your RAM to store its contents.

This means everything inside is lost when you reboot. This usually isn't a big problem, since it is /tmp

Only, you said you needed to create a few directories for your system to work.

My guess is that /tmp is completely empty, and that you cannot create anything inside it without being root.

Please post your /etc/fstab. If you recreate the directories, does everything work again?

jamescondron · 05-23-2009, 10:39 AM

I've had exactly the same problem in the past, it occured for me when I was writing data that over stepped the bounds of the file I was trying to write to, so it had to be written to ramdisk. Reboot to clear it out, and fix whatever process failed when it gave you that error, that- or whatever you've been running which writes huge swathes of data.

jim.thornton · 05-23-2009, 11:19 PM

Quote:

Originally Posted by jamescondron

I've had exactly the same problem in the past, it occured for me when I was writing data that over stepped the bounds of the file I was trying to write to, so it had to be written to ramdisk. Reboot to clear it out, and fix whatever process failed when it gave you that error, that- or whatever you've been running which writes huge swathes of data.

I'm sorry, I don't follow.

jim.thornton · 05-23-2009, 11:20 PM

Quote:

Originally Posted by janhe

Your /tmp is a tmpfs, that means it isn't on your hard disk, but it is using your RAM to store its contents.

This means everything inside is lost when you reboot. This usually isn't a big problem, since it is /tmp

Only, you said you needed to create a few directories for your system to work.

My guess is that /tmp is completely empty, and that you cannot create anything inside it without being root.

Please post your /etc/fstab. If you recreate the directories, does everything work again?

Code:

# cat /etc/fstab
none    /dev/pts        devpts  rw      0 0

janhe · 05-24-2009, 03:48 AM

Now I cannot follow anymore. On my system, all partitions that are mounted when the system boots have an entry in /etc/fstab

I have no idea how your setup works.

The only shot I can take at it is: did you look what is inside /tmp . Do the problems remain when you recreate the directories you mentioned in your first post?

If that doesn't work, I cannot help you.

Maybe jamescondron can give a bit more detail?

kstan · 05-24-2009, 04:30 AM

type 'ls -l /', /tmp entry shall like this:
drwxrwxrwt 14 root root 4096 2009-05-24 13:49 tmp

jamescondron · 05-24-2009, 09:19 AM

Your /tmp/ is mounted on ramdisk because it is being used to hold some kind of overflow. You could just empty it as root. You could just reboot. You could just leave it- pick a poison

jim.thornton · 05-25-2009, 07:22 AM

Hi James,

The problem is that I can't keep going in and rebooting everytime this happens.

It seems as though my host has fixed it for now by changing the way /tmp is mounted. I'll keep my eye on it and let you know if I have any more problems.

Thank you for the help.

jamescondron · 05-25-2009, 07:24 AM

Okay, no worries Jim, If you have root, what happens if you do just do an rm on the contents?

onebuck · 05-25-2009, 07:52 AM

Hi,

Quote:

Originally Posted by jim.thornton

Code:

# cat /etc/fstab
none    /dev/pts        devpts  rw      0 0

From the cli as root do a 'mount' then post the output. Your '/etc/fstab' is not complete.

veerain · 05-25-2009, 07:57 AM

Here is the solution.
The /tmp directory is mounted as a tmpfs filesystem.
do as following

umount /tmp

then see if the error occurs

JZL240I-U · 05-25-2009, 08:25 AM

I seem to have read somewhere that you don't chmod 777 /tmp-files because then you can have executables in the /tmp space which could compromise security. chmod 666 should be better AFAIR. Perhaps you should read up on this when the main problem is solved.

jim.thornton · 05-26-2009, 07:12 PM

I remember reading the same thing but then other posts that I was reading said 777.

As for the mount/umount. Will that work even if I'm on a VPS?