Leaking file descriptors after a while
| Affects | Status | Importance | Assigned to | Milestone | |
|---|---|---|---|---|---|
| mortoray.com |
Fix Released
|
Undecided
|
Unassigned | ||
Bug Description
I'm using https:/
The project code lives here:
https:/
In particular, this is the logic that kicks off jobs:
https:/
Looking at the system (Raspberry PI) after a while clearly shows tons of open FDs for "pipe":
I suspect I'm either not using shelljob right, or there is a bug on how it handles closing Group.
For any other info I can provide, please let me know!
root@oclock:
● mqtt2cmd.service - MQTT based command dispatcher
Loaded: loaded (/lib/systemd/
Active: active (running) since Sun 2020-04-12 21:52:14 EDT; 1 day 20h ago
Main PID: 16101 (start_mqtt2cmd.)
CGroup: /system.
├─16101 /bin/bash /home/pi/
├─16106 python ./main.py /home/pi/
├─16110 python ./main.py /home/pi/
└─16111 python ./main.py /home/pi/
Apr 12 21:52:14 oclock systemd[1]: Started MQTT based command dispatcher.
root@oclock:
1060
root@oclock:
COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
python 16111 pi cwd DIR 179,2 4096 252943 /home/pi/
python 16111 pi rtd DIR 179,2 4096 2 /
python 16111 pi txt REG 179,2 3821024 253051 /home/pi/
python 16111 pi mem REG 179,2 14440 139818 /usr/lib/
python 16111 pi mem REG 179,2 76296 139829 /usr/lib/
python 16111 pi mem REG 179,2 124592 1920 /lib/arm-
python 16111 pi mem REG 179,2 26648 139816 /usr/lib/
python 16111 pi mem REG 179,2 13996 1974 /lib/arm-
python 16111 pi mem REG 179,2 66008 3747 /lib/arm-
...
root@oclock:
python 16111 pi 1010w FIFO 0,10 0t0 3436193 pipe
python 16111 pi 1011w FIFO 0,10 0t0 3437223 pipe
python 16111 pi 1012w FIFO 0,10 0t0 3438246 pipe
python 16111 pi 1013w FIFO 0,10 0t0 3439286 pipe
python 16111 pi 1014w FIFO 0,10 0t0 3440319 pipe
python 16111 pi 1015w FIFO 0,10 0t0 3441347 pipe
python 16111 pi 1016w FIFO 0,10 0t0 3442390 pipe
python 16111 pi 1017w FIFO 0,10 0t0 3443426 pipe
python 16111 pi 1018w FIFO 0,10 0t0 3444457 pipe
python 16111 pi 1019w FIFO 0,10 0t0 3445501 pipe
root@oclock:
root@oclock:
1013
| Flavio Fernandes (ffernand) wrote | #1 |
| edA-qa (eda-qa) wrote | #2 |
| edA-qa (eda-qa) wrote | #3 |
| edA-qa (eda-qa) wrote | #4 |
| Flavio Fernandes (ffernand) wrote Re: [Bug 1872849] Re: Leaking file descriptors after a while | #5 |
| edA-qa (eda-qa) wrote | #6 |
| Changed in mortoray.com: | |
| status: | New → Fix Released |
| Flavio Fernandes (ffernand) wrote | #7 |
| Flavio Fernandes (ffernand) wrote | #8 |
I was able to narrow down to a change that makes this issue go away.
In subprocess.Popen:
file proc.py:
def _run_impl(self, cmd, shell):
cmd = _expand_cmd(cmd)
handle = subprocess.
The close_fds is also a good idea to use. See:
https:/
popen2 closes all file descriptors by default, but you have to specify close_fds=True with Popen to guarantee this behavior on all platforms or past Python versions.