Ubuntu takes too much memory even if all processes killed

10110111 · 01-09-2010, 09:16 AM

Here's what i have with Ubuntu 9.10:

Code:

ruslan@hplaptop:~$ ps -Ao%mem,pid,cmd
%MEM   PID CMD
 0.5     1 /sbin/init
 0.0     2 [kthreadd]
 0.0     3 [migration/0]
 0.0     4 [ksoftirqd/0]
 0.0     5 [watchdog/0]
 0.0     6 [events/0]
 0.0     7 [cpuset]
 0.0     8 [khelper]
 0.0     9 [netns]
 0.0    10 [async/mgr]
 0.0    11 [kintegrityd/0]
 0.0    12 [kblockd/0]
 0.0    13 [kacpid]
 0.0    14 [kacpi_notify]
 0.0    15 [kacpi_hotplug]
 0.0    16 [ata/0]
 0.0    17 [ata_aux]
 0.0    18 [ksuspend_usbd]
 0.0    19 [khubd]
 0.0    20 [kseriod]
 0.0    21 [kmmcd]
 0.0    22 [bluetooth]
 0.0    23 [khungtaskd]
 0.0    24 [pdflush]
 0.0    25 [pdflush]
 0.0    26 [kswapd0]
 0.0    27 [aio/0]
 0.0    28 [ecryptfs-kthrea]
 0.0    29 [crypto/0]
 0.0    33 [scsi_eh_0]
 0.0    34 [scsi_eh_1]
 0.0    37 [kstriped]
 0.0    38 [kmpathd/0]
 0.0    39 [kmpath_handlerd]
 0.0    40 [ksnapd]
 0.0    41 [kondemand/0]
 0.0    42 [kconservative/0]
 0.0    43 [krfcommd]
 0.0   297 [kjournald2]
 0.0   939 /sbin/getty -8 38400 tty4
 0.0   942 /sbin/getty -8 38400 tty5
 0.0   951 /sbin/getty -8 38400 tty2
 0.0   952 /sbin/getty -8 38400 tty3
 0.0   954 /sbin/getty -8 38400 tty6
 0.2   992 /usr/sbin/sshd
 0.8  2028 /bin/login --       
 1.8  2132 -bash
 1.5  2179 sshd: ruslan [priv]
 0.9  2236 sshd: ruslan@pts/0
 1.8  2237 -bash
 0.4  2328 ps -Ao%mem,pid,cmd
ruslan@hplaptop:~$ lsmod
Module                  Size  Used by
tulip                  48320  0 
binfmt_misc             8356  1 
intel_agp              27484  1 
agpgart                34988  1 intel_agp
ruslan@hplaptop:~$ mount
/dev/sda3 on / type ext4 (rw,errors=remount-ro)
udev on /dev type tmpfs (rw,mode=0755)
none on /dev/pts type devpts (rw,noexec,nosuid,gid=5,mode=0620)
none on /proc type proc (rw)
ruslan@hplaptop:~$ cat /proc/mounts
rootfs / rootfs rw 0 0
udev /dev tmpfs rw,relatime,mode=755 0 0
/dev/disk/by-uuid/19ef22cb-3f57-486d-ad1f-5eccebeac1a8 / ext4 rw,relatime,errors=remount-ro,barrier=1,data=ordered 0 0
none /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
none /proc proc rw,relatime 0 0
ruslan@hplaptop:~$ free
             total       used       free     shared    buffers     cached
Mem:        186008     182600       3408          0       9032      25244
-/+ buffers/cache:     148324      37684
Swap:      4200988        880    4200108

If i kill sshd, unload tulip, free memory remains almost the same.
So, almost no processes, nothing mounted, no kernel modules and still 145M of RAM in use! What could be using so much memory? AFAIK linux kernel should work on 32M (maybe even less) machine...

pixellany · 01-09-2010, 10:07 AM

Arch32, KDE 4.3.4, nothing open except Firefox and a terminal:

Code:

[root@Ath mherring]#   free
total       used       free     shared    buffers     cached
Mem:       1033812     763128     270684          0      46160     431276
-/+ buffers/cache:     285692     748120
Swap:      3903784      85308    3818476

As you can see, I'm using most of my 1GB of RAM just watching LQ.

I'll go take a quick look with XFCE....

10110111 · 01-09-2010, 10:14 AM

Quote:

-/+ buffers/cache: 285692 748120

This is what you should look at.

pixellany · 01-09-2010, 10:21 AM

Running XFCE4, with only FF and a terminal:

Code:

[mherring@Ath Desktop]$ free
             total       used       free     shared    buffers     cached
Mem:       1033812     911332     122480          0      97200     671392
-/+ buffers/cache:     142740     891072
Swap:      3903784       6968    3896816

More than with KDE!!

10110111 · 01-09-2010, 10:29 AM

Quote:

More than with KDE!!

Really less

KDE:

Quote:

-/+ buffers/cache: 285692 748120

XFCE:

Quote:

-/+ buffers/cache: 142740 891072

285692 with KDE vs 142740 with XFCE

pixellany · 01-09-2010, 10:39 AM

Hmmmm---for those of us who struggle with the most basic things, could you explain??

10110111 · 01-09-2010, 10:45 AM

Well, what you thought of being memory usage is just actual usage by apps + buffers + cache. Buffers & cache are enlarged (as needed) when the memory gets free and shrinked when an app needs more memory than is available.
+/- buffers/cache line gives information with buffers & cache subtracted from full memory utilization, so it's what you actually need to estimate memory use by apps.

syg00 · 01-09-2010, 05:13 PM

Memory management is a can of worms. In Linux it's exacerbated by the fact that (de-)allocation is lazy - it's only done when needed, or maybe in a "quiet time". It's too expensive to continually run the queues moving page frames around between free and allocated. Hence storage can appear (still) allocated even when the task has terminated.
Then there are the shared libraries ... and dentry and inode caches ... and things that aren't apparently accounted for anywhere.

There are a pile of sysctls for managing the points at which decisions can be affected - have a look at /proc/sys/vm/
I'd suggest you only play with such on a test system.
/proc/meminfo for overall system metrics - for specific tasks see /proc/<pid>/smaps

10110111 · 01-10-2010, 06:00 AM

Quote:

(de-)allocation is lazy

If i start programs/load modules etc., memory usage gets higher, and at some point linux begins to swap. Is it enough to think memory IS really used?

syg00 · 01-10-2010, 07:12 AM

You want to run something it uses memory - you just don't (generally) get any say in what memory it uses.

DragonSlayer48DX · 01-14-2010, 08:27 AM

Quote:

Originally Posted by syg00

You want to run something it uses memory - you just don't (generally) get any say in what memory it uses.

Isn't RAM and CPU usage what makes Linux so much faster and smoother?

I'm not the most technically minded here, but in my own personal experience with several older versions of Windows on quite a few different PCs, it seemingly prefers the swap file over RAM; Whenever RAM dropped below 74% free, regardless of how much your system actually had, it really slowed to a crawl, as it had to access swap just to open a menu! And that's why Windows has always needed far more than the 'minimum requirement' to actually work, but CPU usage still hardly ever exceeded 19% as the system was still at the mercy of the HDD. Windows won't not swap. Ever.

10110111 · 01-15-2010, 07:34 AM

Quote:

Windows won't not swap. Ever.

Disable swap and it will

DragonSlayer48DX · 01-15-2010, 07:50 AM

Quote:

Originally Posted by 10110111

Disable swap and it will

Guess you got me there, hehehe...

I did try that once though, and it crashed. I had to hard-reset the PC and boot into 'safe' mode to re-enable swap. That was with Win98 on a PII w/128Mb RAM. It would work if you had a lot more, I suppose.

Cheers

Skaperen · 01-15-2010, 10:33 AM

Just because a program/process is (or even all of them are) killed, does not mean the kernel will remove latent cached copies of blocks of disk data. If you restart one of these programs, it will start faster by virtue of not having to physically read those block of data from disk the first time. That's what cache is about.

If you want to see how much memory would be free, you could modify the startup scripts to display the sizes and pause. That way you can see before very much gets started. Or write/find a program that just allocates and uses up as much RAM as it can, which will eventually force cached data to go away. After that program runs a while, kill it, and then see how much is free (data a program allocates for itself will go away when the program process exits).

10110111 · 01-16-2010, 06:23 AM

Well, it's seen that the memory is really used not for cache because when i start more and more programs, memory usage is increased, and when about 90% is shown as used, swap starts to be used instead of unallocation of cache.

Hmm... i restarted the machine and now it's using 33M without gdm... seems not very reproducible...