Created attachment 2342 [details]
Make oom_adj reset before re-execing sshd

Currently, on linux sshd attempts to remove itself from the influence of oom-killer by modifying the oom_adj parameter for itself in proc to -17.

This is controlled via two functions; oom_adjust_setup() and oom_adjust_restore().

Setup saves the old score (typically zero on initialization) and sets sshd to -17 whilst oom_adjust_restore places the saved value from initialization back into the running sshd process.

The principle here being that whilst we want the main sshd listener process never to be oom-killed, we want any subsequent children that sshd spawns to be oom-killed if necessary.

The problem that occurs however is on a SIGHUP, sshd re-execs itself and inherits the oom_adj score from its protege. When sshd goes through its initialization steps it calls oom_adjust_setup() and saves the now -17 value. On oom_adjust_restore() sshd then restores -17 to all child processes sshd spawns.

This has the effect of after performing a reload causing all sshd processes (and by the nature of inheritance) all descendant pids having -17 as their oom_adj value which causes all sshd sessions after the reload to never be candidates for oom-killer.

Clearly this is not desirable as it can lead to unrecoverable out of memory conditions on linux.

This is very simple to test on linux. Restart sshd on a linux host. Login to it via ssh and then run cat /proc/self/oom_adj. How reload ssh, relogin again via ssh and recheck cat /proc/self/oom_adj.

I have created a patch, which I believe is honourable to the design philosophy being utilized by openssh.

This patch adds an additional function to platform.h called platform_pre_restart() that (if defined) will perform oom_adjust_restore(). This is then called inside of sighup_restart() prior to performing the re-exec of ssh.