Mayor Memory LEAK

jpbag · 02-05-2004, 04:23 PM

my fedora core 1 is up to date right now.
just runned yum and the system is ok.

i saw the glibc-2.3.2-101.1 memory leak problem, but my sistem is still having it with glibc-2.3.2-101.4.

i dont know if is glibc or another component.
so i only start samba (its a emulated PDC and fileserver of my office network) and twice a day RSYNC.
i can manually run rsync and the script called via cron.
no problems.
i had to cron 3 restarts a day to keep the server healthy.

its a p II 266 256 ram.
when fresh booted the server runs in < 100mb mem, a couple of hours the server has ~256 mb in use.
top does not show any process with more than 30 % of mem.
may be 1 with 25 and the rest less than 1%

Code:

free -m
             total       used       free     shared    buffers     cached
Mem:           245        222         23          0         76         55
-/+ buffers/cache:         90        154
Swap:          251          1        249


top

 19:25:49  up 20:24,  1 user,  load average: 0.04, 0.01, 0.00
39 processes: 38 sleeping, 1 running, 0 zombie, 0 stopped
CPU states:  cpu    user    nice  system    irq  softirq  iowait    idle
           total    0.3%    0.0%    0.5%   0.0%     0.0%    0.0%   99.0%
Mem:   251508k av,  227804k used,   23704k free,       0k shrd,   78152k buff
       101004k active,              42224k inactive
Swap:  257032k av,    1272k used,  255760k free                   56596k cached
Sort by memory usage
  PID USER     PRI  NI  SIZE  RSS SHARE STAT %CPU %MEM   TIME CPU COMMAND
 1090 ntp       16   0  3744 3744  2204 S     0.0  1.4   0:00   0 ntpd
 1224 xfs       16   0  3232 3228   880 S     0.0  1.2   0:00   0 xfs
 1110 root      16   0  2664 2664  1992 S     0.0  1.0   0:00   0 sendmail
 1121 smmsp     16   0  2344 2116  1808 S     0.0  0.8   0:00   0 sendmail
 2860 root      16   0  2060 2056  1788 S     0.0  0.8   0:00   0 sshd
 1006 root      16   0  1932 1932  1516 S     0.0  0.7   0:00   0 cupsd
 1139 root      16   0  2024 1564  1368 S     0.0  0.6   0:01   0 nmbd
 1060 root      19   0  1424 1424  1300 S     0.0  0.5   0:01   0 sshd
 2863 root      16   0  1380 1380  1136 S     0.0  0.5   0:00   0 bash
 1132 root      16   0  1784 1260  1044 S     0.0  0.5   0:00   0 smbd
 2994 root      17   0  1068 1068   900 R     0.9  0.4   0:01   0 top
 1076 root      16   0   916  916   804 S     0.0  0.3   0:00   0 xinetd
  982 root      17   0   680  680   496 S     0.0  0.2   0:00   0 smartd
 1270 dbus      18   0   692  664   600 S     0.0  0.2   0:00   0 dbus-daemon-1
 1200 root      16   0   624  624   540 S     0.0  0.2   0:00   0 crond
  870 root      16   0   572  572   496 S     0.0  0.2   0:00   0 syslogd
 1259 daemon    16   0   564  556   504 S     0.0  0.2   0:00   0 atd
 1285 root      17   0   492  488   436 S     0.0  0.1   0:00   0 rhnsd
  942 root      15   0   432  432   384 S     0.0  0.1   0:00   0 apmd
    1 root      16   0   428  428   372 S     0.0  0.1   0:07   0 init
 1166 root      16   0   404  400   348 S     0.0  0.1   0:00   0 gpm
  874 root      16   0   392  392   332 S     0.0  0.1   0:00   0 klogd
 1291 root      18   0   356  356   304 S     0.0  0.1   0:00   0 mingetty
 1292 root      17   0   356  356   304 S     0.0  0.1   0:00   0 mingetty
 1293 root      18   0   356  356   304 S     0.0  0.1   0:00   0 mingetty
 1295 root      19   0   356  356   304 S     0.0  0.1   0:00   0 mingetty
 1294 root      18   0   352  352   304 S     0.0  0.1   0:00   0 mingetty
 1296 root      20   0   352  352   304 S     0.0  0.1   0:00   0 mingetty
    2 root      15   0     0    0     0 SW    0.0  0.0   0:00   0 keventd
    3 root      15   0     0    0     0 SW    0.0  0.0   0:00   0 kapmd
    4 root      34  19     0    0     0 SWN   0.0  0.0   0:00   0 ksoftirqd/0
    6 root      25   0     0    0     0 SW    0.0  0.0   0:00   0 bdflush
    5 root      16   0     0    0     0 SW    0.0  0.0   0:06   0 kswapd
    7 root      15   0     0    0     0 SW    0.0  0.0   0:00   0 kupdated
    8 root      21   0     0    0     0 SW    0.0  0.0   0:00   0 mdrecoveryd
   14 root      15   0     0    0     0 SW    0.0  0.0   0:00   0 ahc_dv_0
   15 root      25   0     0    0     0 SW    0.0  0.0   0:00   0 scsi_eh_0
   18 root      15   0     0    0     0 SW    0.0  0.0   0:02   0 kjournald
  495 root      15   0     0    0     0 SW    0.0  0.0   0:00   0 kjournald

WTH is going on ?

(

wapcaplet · 02-05-2004, 06:04 PM

Looks fine to me. Do a search here on "memory leak" - this question has been asked frequently

If you are not experiencing lots of slowdown and swapfile thrashing, you do not have any problems with memory. Looks like your swap space is barely being used at all, and as your 'free -m' command shows, you have 154M free. No worries!

snacky · 02-05-2004, 06:10 PM

Yup, so far this looks like misinterpreting the output of "free"

Hint: the most relevant line is the one labelled" -/+ buffers/cache: "

jpbag · 02-05-2004, 10:09 PM

guys. the system HUNGS.

it keeps swapping until its death.

theres no way the system can eat 256 MB doing NOTHING !

seems to be threads that use certain amount of memory that is never being released ...

i have told you guys what i know so far.
but is not a "reading problem". its a "out of memory" problem.

Code:

Feb  2 12:51:02 fileserver kernel: Out of Memory: Killed process 1147 (xfs).
Feb  2 12:53:33 fileserver kernel: Out of Memory: Killed process 1087 (sendmail).
Feb  2 12:55:02 fileserver kernel: Out of Memory: Killed process 1184 (dbus-daemon-1).
Feb  2 12:57:17 fileserver kernel: Out of Memory: Killed process 1058 (ntpd).
Feb  2 12:59:13 fileserver kernel: Out of Memory: Killed process 13614 (mrtg).
Feb  2 12:59:41 fileserver kernel: Out of Memory: Killed process 13629 (rhn_check).
Feb  2 12:59:55 fileserver kernel: Out of Memory: Killed process 13788 (mrtg).
Feb  2 13:00:27 fileserver kernel: Out of Memory: Killed process 13678 (rsync).
Feb  2 13:01:33 fileserver kernel: Out of Memory: Killed process 13687 (rsync).
Feb  2 13:02:24 fileserver kernel: Out of Memory: Killed process 13716 (rsync).
Feb  2 13:03:02 fileserver kernel: Out of Memory: Killed process 13733 (rsync).
Feb  2 13:04:08 fileserver kernel: Out of Memory: Killed process 13767 (rsync).

wapcaplet · 02-05-2004, 10:41 PM

OK, you didn't mention that the first time

And there are a lot of posts here from people wondering why all their memory is being used up when nothing's actually wrong - it's perfectly normal for Linux to use a lot of memory if it's available; it's not normal for it to cause problems, though.

Anyway, according to the output of 'top' and 'free' that you posted, everything looks fine. 'top' really ought to show what is using memory; is the one you posted earlier from when it's running normally, or when it's about to start having problems? Does your system log show anything unusual before you start getting out of memory errors? Maybe there's a cron job or something that is starting up periodically that is causing problems. I guess my only other suggestion would be to try disabling some things temporarily to see if that helps; it might help you narrow down which one is causing trouble.

jpbag · 02-05-2004, 10:54 PM

i was launching a script via cron.
i used kcron to set it up.
the script calls several times to rsync using ssh to synchronize diff dirs on 2 diff servers.

this script has one line at a time to process sequentially each dir.

well. i dont know why, each time it starts the system get out of memory and dies.
i was wondering that was a problem with some lib or component within FEDORA.
but it think it was cron running several times my script.
so i changed with kcron this job from root to another user: (System crontab).
it seems to be working ok now.

wapcaplet · 02-05-2004, 11:21 PM

Sounds like it's definitely a problem with your script then. Is this a script you wrote yourself, or something else?