| 2017-01-29 04:58:41 |
Nate Eldredge |
description |
On a vps with 512MB RAM and a 2 GB swapfile, I am experiencing 100% reproducible OOM when running duplicity. duplicity uses a fair amount of memory, about 300 MB at the time of OOM, but there should be plenty of swap to accommodate it.
I have attached a syslog excerpt. Note "Free swap = 2004684kB".
If I am reading the log message correctly, "order=2" at the top means this was an attempt to allocate 4 pages, i.e. 16KB. It certainly seems that it should be possible to satisfy this by swapping. I know fragmentation is a possible issue but it seems implausible that we couldn't find 4 pages together.
This doesn't seem to happen when running other memory-hogging processes. I wrote a small program that malloc's 1 GB and writes to all of it. It runs fine, even as root. It is slow because of swapping but the oom-killer does not get invoked. So I am not sure what is special about duplicity. It's an ordinary user-space backup utility written in Python; it reads files, encrypts data, and sends it out over ssh. The only thing I can think of is that it runs gpg to do the encryption, and I believe gpg will mlock() some memory to keep keys secure, but that should be a really small amount.
Happy to supply more information or test things.
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.4.0-59-generic 4.4.0-59.80
ProcVersionSignature: Ubuntu 4.4.0-59.80-generic 4.4.35
Uname: Linux 4.4.0-59-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Jan 21 19:31 seq
crw-rw---- 1 root audio 116, 33 Jan 21 19:31 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.1-0ubuntu2.5
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
Date: Sun Jan 29 03:37:31 2017
HibernationDevice: RESUME=UUID=7c1226a1-9254-49c2-b4e6-64d204affd15
IwConfig:
lo no wireless extensions.
eth1 no wireless extensions.
eth0 no wireless extensions.
Lspci:
Lsusb: Error: command ['lsusb'] failed with exit code 1:
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
XDG_RUNTIME_DIR=<set>
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB:
ProcKernelCmdLine: root=/dev/xvda1 ro quiet splash console=hvc0
RelatedPackageVersions:
linux-restricted-modules-4.4.0-59-generic N/A
linux-backports-modules-4.4.0-59-generic N/A
linux-firmware 1.157.6
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: Upgraded to xenial on 2016-09-19 (132 days ago) |
On a vps with 512MB RAM and a 2 GB swapfile, I am experiencing 100% reproducible OOM when running duplicity. duplicity uses a fair amount of memory, about 300 MB at the time of OOM, but there should be plenty of swap to accommodate it.
I have attached a syslog excerpt. Note "Free swap = 2004684kB".
If I am reading the log message correctly, "order=2" at the top means this was an attempt to allocate 4 pages, i.e. 16KB. It certainly seems that it should be possible to satisfy this by swapping. I know fragmentation is a possible issue but it seems implausible that we couldn't find 4 pages together.
This doesn't seem to happen when running other memory-hogging processes. I wrote a small program that malloc's 1 GB and writes to all of it. It runs fine, even as root. It is slow because of swapping but the oom-killer does not get invoked. So I am not sure what is special about duplicity. It's an ordinary user-space backup utility written in Python; it reads files, encrypts data, and sends it out over ssh. The only thing I can think of is that it runs gpg to do the encryption, and I believe gpg will mlock() some memory to keep keys secure, but that should be a really small amount.
Happy to supply more information or test things.
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.4.0-59-generic 4.4.0-59.80
ProcVersionSignature: Ubuntu 4.4.0-59.80-generic 4.4.35
Uname: Linux 4.4.0-59-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Jan 21 19:31 seq
crw-rw---- 1 root audio 116, 33 Jan 21 19:31 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.1-0ubuntu2.5
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
Date: Sun Jan 29 03:37:31 2017
HibernationDevice: RESUME=UUID=7c1226a1-9254-49c2-b4e6-64d204affd15
IwConfig:
lo no wireless extensions.
eth1 no wireless extensions.
eth0 no wireless extensions.
Lspci:
Lsusb: Error: command ['lsusb'] failed with exit code 1:
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
XDG_RUNTIME_DIR=<set>
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB:
ProcKernelCmdLine: root=/dev/xvda1 ro quiet splash console=hvc0
RelatedPackageVersions:
linux-restricted-modules-4.4.0-59-generic N/A
linux-backports-modules-4.4.0-59-generic N/A
linux-firmware 1.157.6
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: Upgraded to xenial on 2016-09-19 (132 days ago)
---
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Jan 21 19:31 seq
crw-rw---- 1 root audio 116, 33 Jan 21 19:31 timer
AplayDevices: Error: [Errno 2] No such file or directory
ApportVersion: 2.20.1-0ubuntu2.5
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
DistroRelease: Ubuntu 16.04
HibernationDevice: RESUME=UUID=7c1226a1-9254-49c2-b4e6-64d204affd15
IwConfig:
lo no wireless extensions.
eth1 no wireless extensions.
eth0 no wireless extensions.
Lspci:
Lsusb: Error: command ['lsusb'] failed with exit code 1:
Package: linux (not installed)
PciMultimedia:
ProcEnviron:
TERM=xterm
PATH=(custom, no user)
LANG=en_US.UTF-8
SHELL=/bin/bash
ProcFB:
ProcKernelCmdLine: root=/dev/xvda1 ro quiet splash console=hvc0
ProcVersionSignature: Ubuntu 4.4.0-59.80-generic 4.4.35
RelatedPackageVersions:
linux-restricted-modules-4.4.0-59-generic N/A
linux-backports-modules-4.4.0-59-generic N/A
linux-firmware 1.157.6
RfKill: Error: [Errno 2] No such file or directory
Tags: xenial
Uname: Linux 4.4.0-59-generic x86_64
UpgradeStatus: Upgraded to xenial on 2016-09-19 (132 days ago)
UserGroups:
_MarkForUpload: True |
|
