LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Embedded & Single-board computer (https://www.linuxquestions.org/questions/linux-embedded-and-single-board-computer-78/)
-   -   Oom killer killed my python3 process while there was free memory (https://www.linuxquestions.org/questions/linux-embedded-and-single-board-computer-78/oom-killer-killed-my-python3-process-while-there-was-free-memory-4175717595/)

selcukgk29 10-09-2022 01:34 PM
Oom killer killed my python3 process while there was free memory
 
I am working on webserver on a linux embedded board without swap memory. Sometimes my webserver crashed when there was avaliable memory.

I set those configurations to prevent this but it didn't work.
Code:
overcommit_memory=2 overcommit_ratio=400 min_free_kbytes=1000,
Meminfo:

MemTotal:        241868 kB
MemFree:          157532 kB
MemAvailable:    129380 kB
Buffers:              0 kB
Cached:            22616 kB
SwapCached:            0 kB
Active:            53448 kB
Inactive:          8852 kB
Active(anon):      45644 kB
Inactive(anon):    8448 kB
Active(file):      7804 kB
Inactive(file):      404 kB
Unevictable:          0 kB
Mlocked:              0 kB
HighTotal:            0 kB
HighFree:              0 kB
LowTotal:        241868 kB
LowFree:          157532 kB
SwapTotal:            0 kB
SwapFree:              0 kB
Dirty:                12 kB
Writeback:            0 kB
AnonPages:        39712 kB
Mapped:            8228 kB
Shmem:            14392 kB
KReclaimable:      4012 kB
Slab:              13148 kB
SReclaimable:      4012 kB
SUnreclaim:        9136 kB
KernelStack:        712 kB
PageTables:          872 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      967472 kB
Committed_AS:    211544 kB
VmallocTotal:    1818624 kB
VmallocUsed:        3292 kB
VmallocChunk:          0 kB
Percpu:              84 kB
CmaTotal:        131072 kB
CmaFree:          121228 kB
Logs:
Code:
kworker/u2:1 invoked oom-killer: gfp_mask=0x2dc2(GFP_KERNEL|__GFP_HIGHMEM|__GFP_NOWARN|__GFP_ZERO), order=0, oom_score_adj=0
Code:
                [  pid  ] uid    tgid total_vm    rs pgtables_bytes swapents  oom_score_adj  name
[256776.750536] [    141]    0  141    2973      39    14336        0            0 rngd
[256776.759213] [    143]    0  143    9620      938    24576        0          -250 systemd-journal
[256776.768543] [    163]    0  163    2858      128    14336        0        -1000 systemd-udevd
[256776.777677] [    179]  992  179    6008      77    18432        0            0 systemd-timesyn
[256776.787171] [    435]    0  435      366      18    8192        0            0 atd
[256776.795496] [    436]    0  436      596      64    8192        0            0 crond
[256776.804098] [    437]  999  437      841      95    10240        0          -900 dbus-daemon
[256776.813047] [    444]    0  444    1455      112    12288        0            0 ofonod
[256776.821765] [    454]    0  454      376      30    10240        0            0 syslogd
[256776.830476] [    461]  998  461      684      44    12288        0            0 rpcbind
[256776.839183] [    468]    0  468    1256      932    12288        0            0 klogd
[256776.847852] [    469]    0  469    1299      81    12288        0            0 systemd-logind
[256776.857158] [    470]  993  470    1342      79    14336        0            0 systemd-resolve
[256776.866536] [    471]  996  471      947      81    12288        0            0 avahi-daemon
[256776.875745] [    472]  997  472      613      123    12288        0            0 rpc.statd
[256776.884678] [    477]  996  477      916      51    10240        0            0 avahi-daemon
[256776.893703] [    478]    0  478    3231      106    16384        0            0 tcf-agent
[256776.902526] [    482]    0  482      886      19    10240        0            0 agetty
[256776.911093] [    488]    0  488    4208    1681    26624        0            0 mydaemon
[256776.919901] [    490]    0  490    4834    2334    26624        0            0 python3
[256776.928492] [    597]    0  597    14636    5348    49152        0            0 python3
[256776.937084] [  12534]    0 12534      655      77    8192        0            0 sh
[256776.945600] [  19169]    0 19169    16437      309    30720        0            0 core
[256776.954004] [  19177]    0 19177    2154      142    14336        0            0 wpa_supplicant
[256776.963406] [  19179]  994 19179    1454      102    14336        0            0 systemd-network
[256776.972869] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),task=python3,pid=597,uid=0
[256776.981576] Out of memory: Killed process 597 (python3) total-vm:58544kB, anon-rss:21392kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:49152kB oom_score_adj:0
[256777.041482] oom_reaper: reaped process 597 (python3), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

Emerson 10-09-2022 09:03 PM
The only way to stop it is to add more RAM (which you probably can't) or create a small swap file.

pan64 10-10-2022 12:28 AM
Some background info:
https://chrisdown.name/2018/01/02/in...e-of-swap.html

syg00 10-10-2022 12:54 AM
Quote:
Originally Posted by selcukgk29 (Post 6385398)
Sometimes my webserver crashed when there was avaliable memory.
Prove it - that meminfo needs to be immediately before the oom killer strikes.
Quote:
[256776.981576] Out of memory: Killed process 597 (python3) total-vm:58544kB, anon-rss:21392kB, file-rss:0kB, shmem-rss:0kB, UID:0 pgtables:49152kB oom_score_adj:0
I'll believe the kernel until proven otherwise.

With those sysctls you need a (large) swap.

selcukgk29 10-11-2022 06:17 AM
The board has 256 Mb ram. When I tested ram allocation test with
Quote:
/dev/zero head -c 1000m |tail
, oom killed tail process at total-vm:159600kB.

Quote:
[12781.161711] Out of memory: Killed process 14482 (tail) total-vm:159600kB, anon-rss:158240kB, file-rss:4kB, shmem-rss:0kB, UID:0 pgtables:165888kB oom_score_adj:0
But oom killed python3 at total-vm 58544kB before ?

pan64 10-11-2022 06:38 AM
python itself needs a lot of ram.


All times are GMT -5. The time now is 06:24 PM.