#!/bin/bash for proc in $(find /proc -maxdepth 1 -regex '/proc/[0-9]+'); do printf"%2d %5d %s\n" \ "$(cat $proc/oom_score)" \ "$(basename $proc)" \ "$(cat $proc/cmdline | tr '\0' ' ' | head -c 50)" done 2>/dev/null | sort -nr | head -n 10
保护措施
设置OverCommit
只有在OverCommit的时候才会触发OOM, 默认是许可一定程度的OverCommit的。
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
https://docs.kernel.org/vm/overcommit-accounting.html vm.overcommit_memory 作用是控制OverCommit是否被许可。 0 Heuristic overcommit handling. Obvious overcommits of address space are refused. Used for a typical system. It ensures a seriously wild allocation fails while allowing overcommit to reduce swap usage. root is allowed to allocate slightly more memory in this mode. This is the default. 1 Always overcommit. Appropriate for some scientific applications. Classic example is code using sparse arrays and just relying on the virtual memory consisting almost entirely of zero pages. 2 Don’t overcommit. The total address space commit for the system is not permitted to exceed swap + a configurable amount (default is 50%) of physical RAM. Depending on the amount you use, in most situations this means a process will not be killed while accessing pages but will receive errors on memory allocation as appropriate.
Useful for applications that want to guarantee their memory allocations will be available in the future without having to initialize every page.
# 手动触发一次OOM规则, Kill符合要求的进程 echo f > /proc/sysrq-trigger
调整服务的OOM Score
对于服务本身的保护方式, 可以采用使用Systemd Unit file里面进行 OOMADJSCORE=*** 的方式来指定,例如保护MySQL的进程不会在OOM Killer的列表中。
1 2 3 4 5 6 7 8 9 10 11 12 13 14
cat /usr/lib/systemd/system/mariadb.service
[Service] Type=simple User=mysql Group=mysql
ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n ExecStart=/usr/bin/mysqld_safe --basedir=/usr ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID # Setting Here. and setting in the /proc/$PID/oom_score_adj. OOMScoreAdjust=-1000