High memory and "can't fork" on heavy long-lived cron daemon

[Ryan Finnie]

Ubuntu 14.04.5 LTS, cron 3.0pl1-124ubuntu2

On a server with heavy cron use (a few thousand entries across cron.d / users, several runs per second, a few dozen forks running at any given time), cron will creep up in memory usage. Last I saw was 6GB RSS, at which point it will start doing:

Jul 6 07:01:27 host cron[21699]: (CRON) error (can't fork)

Not all runs will fail, but this trend will continue until cron is restarted.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jamon Camisso (jamon)]

I ran into this as well, with the same host's cron daemon reaching ~8GB RSS memory after a month.

[Alexis Wilke (alexis-m2osw)]

I found a bug in the code as described in this issue: https://bugs.launchpad.net/ubuntu/+source/cron/+bug/1779583

The bug would not prevent the error you're seeing, however, there is a second fork() which, if it fails, will block that part of cron, which would therefore leak. So if it happens "on the wrong fork()", (1) you won't see any logs and (2) you get a cron "process" which gets stuck waiting for a second child that was never created.

I included a patch too.

[Alexis Wilke (alexis-m2osw)]

Actually, an immediate fix for you guys may be to make sure that there is some *input_data. I have no clue what that means in terms of administrative setup, though. But maybe someone else can help in that area. The really bad statement looks like this:

 if (*input_data && fork() == 0) {
                ... // handle input_data
                exit(0);
        }

        children++;

The children variable counts the number of successful fork()'s. Only in this case (1) the fork() is never checked for failure and (2) if *input_data is false, then the fork() does not even happen. Yet, either way, children is always increment! Ouch! As an administrator, it may not be possible to always make sure there is something in input_data. But it is not unlikely that this is what makes many cron fork() wait forever and thus increase memory. I have not had that problem on my end, I was just reading the code for fun...