LinuxQuestions.org
Page 1 of 2 1 2
Show 50 post(s) from this thread on one page

LinuxQuestions.org (/questions/)
-   Linux - Enterprise (https://www.linuxquestions.org/questions/linux-enterprise-47/)
-   -   RHEL eating RAM space in production environment (top command) (https://www.linuxquestions.org/questions/linux-enterprise-47/rhel-eating-ram-space-in-production-environment-top-command-4175643496/)

fawaz25 12-02-2018 05:29 AM
RHEL eating RAM space in production environment (top command)
 
Dear Members,

I have attached the top command output and free command output with the message. I see all RAM is being consumed. What can be the issue for this?

would be glad for your help.

smallpond 12-02-2018 06:11 AM
https://www.linuxatemyram.com

berndbausch 12-02-2018 08:33 AM
Quote:
Originally Posted by fawaz25 (Post 5932444)
the server becomes slow and goes down
What do you mean by “goes down”?

Quote:
I have attached the top command output and free command output with the message
Try again, nothing is attached.

scasey 12-02-2018 10:00 AM
I believe one has to have more than 1 post before they can attach files...
That said, if this a RHEL server, have you contacted RedHat Support?

berndbausch 12-02-2018 10:09 AM
Quote:
Originally Posted by scasey (Post 5932517)
I believe one has to have more than 1 post before they can attach files...
The output of top and free can easily be added to the body of a post.

scasey 12-02-2018 10:14 AM
Quote:
Originally Posted by berndbausch (Post 5932519)
The output of top and free can easily be added to the body of a post.
Absolutely! Adding the output in the body in [code] tags would be preferred to an attachment, too.

fawaz25 12-02-2018 11:49 PM
Using RHEL 5.7.
The output of top command (SHIFT+M) is as follows:

Code:
top - 08:40:17 up 3 days, 20:26,  1 user,  load average: 18.22, 18.67, 18.38
Tasks: 627 total,  1 running, 626 sleeping,  0 stopped,  0 zombie
Cpu(s): 19.4%us,  8.0%sy,  0.0%ni, 68.0%id,  0.0%wa,  0.0%hi,  4.6%si,  0.0%st
Mem:  128994532k total, 128674952k used,  319580k free,  556464k buffers
Swap:        0k total,        0k used,        0k free, 79488444k cached

# free -m command gives the following output:
Code:
            total      used      free    shared    buffers    cached
Mem:        125971    125661        309          0        543      77611
-/+ buffers/cache:      47506      78464
Swap:            0          0          0

berndbausch 12-03-2018 12:25 AM
Please use code tags. See my signature if you don't know how.

Code:
08:40:17 up 3 days, 20:26,  1 user,  load average: 18.22, 18.67, 18.38
Tasks: 627 total,  1 running, 626 sleeping,  0 stopped,  0 zombie
Cpu(s): 19.4%us,  8.0%sy,  0.0%ni, 68.0%id,  0.0%wa,  0.0%hi,  4.6%si,  0.0%st
Mem:  128994532k total, 128674952k used,  319580k free,  556464k buffers
Swap:        0k total,        0k used,        0k free, 79488444k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU  %MEM    TIME+  COMMAND
11936 hcmpsoft  24  0 1312m 1.2g 1780 S 316.8  1.0  4399:28 java
12126 hcmpsoft  22  0 1271m 1.2g 1764 S 290.2  1.0  4124:44 java
11974 hcmpsoft  25  0 1269m 1.2g 1764 S 298.5  1.0  4219:08 java
12050 hcmpsoft  24  0 1265m 1.2g 1764 S 228.8  1.0  3343:02 java
10684 hcmpsoft  15  0 31.9g 1.2g  47m S  0.0  0.9  2:57.62 PSAPPSRV
10242 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  2:58.41 PSAPPSRV
10394 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  3:10.13 PSAPPSRV
10722 hcmpsoft  15  0 31.9g 1.1g  47m S  0.0  0.9  3:16.06 PSAPPSRV
10646 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  3:17.67 PSAPPSRV
10166 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  3:09.95 PSAPPSRV
10798 hcmpsoft  15  0 31.8g 1.1g  48m S  0.0  0.9  3:00.12 PSAPPSRV
11684 hcmpsoft  15  0 31.9g 1.1g  47m S  0.0  0.9  3:58.00 PSAPPSRV
10836 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  3:09.22 PSAPPSRV
10356 hcmpsoft  15  0 31.8g 1.1g  47m S  0.0  0.9  3:03.21 PSAPPSRV
10912 hcmpsoft  15  0 32.0g 1.1g  48m S  0.0  0.9  2:58.89 PSAPPSRV
11393 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  2:47.18 PSAPPSRV
10128 hcmpsoft  15  0 31.8g 1.1g  48m S  0.0  0.9  3:06.08 PSAPPSRV
 9399 hcmpsoft  15  0 31.8g 1.1g  48m S  1.7  0.9  3:06.92 PSAPPSRV
10432 hcmpsoft  15  0 31.8g 1.1g  48m S  0.0  0.9  3:03.47 PSAPPSRV
10950 hcmpsoft  16  0 31.9g 1.1g  48m S  0.0  0.9  3:03.04 PSAPPSRV
10280 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  2:52.59 PSAPPSRV
10470 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  3:04.24 PSAPPSRV
10988 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  3:02.70 PSAPPSRV
 9195 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  3:00.40 PSAPPSRV
10318 hcmpsoft  15  0 31.8g 1.1g  48m S  0.0  0.9  3:15.51 PSAPPSRV
10760 hcmpsoft  15  0 31.9g 1.1g  48m S  0.0  0.9  3:03.67 PSAPPSRV
11203 hcmpsoft  16  0 31.9g 1.1g  47m S  0.0  0.9  2:21.44 PSAPPSRV
11469 hcmpsoft  15  0 31.8g 1.1g  48m S  0.0  0.9  2:38.83 PSAPPSRV
10204 hcmpsoft  15  0 31.9g 1.0g  47m S  0.0  0.8  2:55.65 PSAPPSRV
 8424 hcmpsoft  15  0 31.8g 1.0g  48m S  0.0  0.8  3:06.12 PSAPPSRV
10874 hcmpsoft  15  0 31.8g 1.0g  48m S  0.0  0.8  2:53.60 PSAPPSRV
11241 hcmpsoft  15  0 31.8g 1.0g  47m S  0.0  0.8  2:31.95 PSAPPSRV
11355 hcmpsoft  16  0 31.8g 1.0g  48m S  0.0  0.8  2:37.33 PSAPPSRV
11507 hcmpsoft  15  0 31.8g 1.0g  48m S  0.0  0.8  2:30.55 PSAPPSRV
11317 hcmpsoft  16  0 31.8g 1.0g  48m S  0.0  0.8  2:31.78 PSAPPSRV
11279 hcmpsoft  15  0 31.8g 1.0g  48m S  0.0  0.8  2:39.42 PSAPPSRV
11431 hcmpsoft  15  0 31.8g 1.0g  48m S  0.0  0.8  2:36.71 PSAPPSRV
 9258 hcmpsoft  15  0 31.8g 1.0g  47m S  0.0  0.8  3:00.52 PSAPPSRV
11165 hcmpsoft  15  0 31.8g 1.0g  48m S  0.0  0.8  2:32.34 PSAPPSRV
 9338 hcmpsoft  15  0 31.8g 1.0g  48m S  0.0  0.8  3:16.69 PSAPPSRV
 9453 hcmpsoft  15  0 31.7g 1.0g  47m S  0.0  0.8  2:56.46 PSAPPSRV
 9201 hcmpsoft  15  0 31.7g 1.0g  47m S  0.0  0.8  3:01.95 PSAPPSRV

# The output of cat proc/meminfo is as follows:

MemTotal:    128994532 kB
MemFree:        318376 kB
Buffers:        556384 kB
Cached:      79474344 kB
SwapCached:          0 kB
Active:      99364828 kB
Inactive:    27979524 kB
HighTotal:          0 kB
HighFree:            0 kB
LowTotal:    128994532 kB
LowFree:        318376 kB
SwapTotal:          0 kB
SwapFree:            0 kB
Dirty:            1472 kB
Writeback:          0 kB
AnonPages:    47313640 kB
Mapped:        112360 kB
Slab:          1078740 kB
PageTables:    153380 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:  64497264 kB
Committed_AS: 132027656 kB
VmallocTotal: 34359738367 kB
VmallocUsed:    262880 kB
VmallocChunk: 34359474939 kB
HugePages_Total:    0
HugePages_Free:      0
HugePages_Rsvd:      0
Hugepagesize:    2048 kB

# free -m command gives the following output:
            total      used      free    shared    buffers    cached
Mem:        125971    125661        309          0        543      77611
-/+ buffers/cache:      47506      78464
Swap:            0          0          0

berndbausch 12-03-2018 12:38 AM
Your CPU is currently 68% idle. Memory is practically depleted; almost nothing is available or used for the buffer cache. I conclude that your application uses up memory. You say
Quote:
I am sure the application won't use this much RAM as it is serving few users only
How did you determine this?

I see 42 processes running, each of them using 1GB or a bit more RAM, totaling 45GB. I wonder how many PSAPPSRV are running altogether? Couldn't they add up to 128GB? Hint: you can scroll top with pgup/pgdown, or run ps -el | grep PSAPPSRV.

syg00 12-03-2018 12:58 AM
What about "uname -a".

And yes, why all the (PeopleSoft ?) instances if only for a few users ?.

fawaz25 12-03-2018 01:15 AM
Quote:
Originally Posted by berndbausch (Post 5932708)
Your CPU is currently 68% idle. Memory is practically depleted; almost nothing is available or used for the buffer cache. I conclude that your application uses up memory. You say

How did you determine this?

I see 42 processes running, each of them using 1GB or a bit more RAM, totaling 45GB. I wonder how many PSAPPSRV are running altogether? Couldn't they add up to 128GB? Hint: you can scroll top with pgup/pgdown, or run ps -el | grep PSAPPSRV.
Yes I do see many instances running. The total used RAM computed from top command (37.9%) is same as that shown by the free command.

berndbausch 12-03-2018 03:10 AM
Quote:
Originally Posted by fawaz25 (Post 5932721)

I wonder why with just 1 user logged in, PSAPPSRV is consuming around 35% of Memory with JAVA just consuming around 4% of memory. Can this be an application related issue??
It is still far from the 128GB you have. It's puzzling that the difference is so big.
EDIT: I misread the free output. It makes sense now.

Regarding your question, the kernel doesn't just launch PSAPPSRV processes because it feels like it. This is most definitely an application issue. It could be caused by a condition outside of the application, but still, the application should be able to handle external conditions correctly and not start tens of processes if they are not needed.

EDIT: I just saw that PSAPPSRV is probably a Peoplesoft process. I would ask the question in an Oracle or Peoplesoft forum.

fawaz25 12-03-2018 03:25 AM
Quote:
Originally Posted by syg00 (Post 5932713)
What about "uname -a".

And yes, why all the (PeopleSoft ?) instances if only for a few users ?.
I too am trying to figure that out.

fawaz25 12-03-2018 03:43 AM
Quote:
Originally Posted by berndbausch (Post 5932744)
It is still far from the 128GB you have. It's puzzling that the difference is so big.
EDIT: I misread the free output. It makes sense now.

Regarding your question, the kernel doesn't just launch PSAPPSRV processes because it feels like it. This is most definitely an application issue. It could be caused by a condition outside of the application, but still, the application should be able to handle external conditions correctly and not start tens of processes if they are not needed.

EDIT: I just saw that PSAPPSRV is probably a Peoplesoft process. I would ask the question in an Oracle or Peoplesoft forum.
Thank you for your comments. I really appreciate.

pan64 12-03-2018 05:44 AM
just a question:
is this only one app with several threads or a lot of apps? (see -H in top)


All times are GMT -5. The time now is 08:36 AM.
Page 1 of 2 1 2
Show 50 post(s) from this thread on one page