Same as #1655842 - Started seeing oom-killer on multiple servers upgraded to 4.4.0-59.
Unlike #1655842, also seeing the same oom-killer on multiple servers updated to 4.8.0-34.
First upgraded them all 4.8 servers 4.8.0-36, then downgraded a few to 4.4.0-63. I am seeing no change in the memory usage, so I can only assume that 4.4.0-63 is also bugged with the same problem as 4.4.0-59 and 4.8.0-34. Either because #1655842 is not fixed, or it is only fixed for certain kinds of workloads.
However, on closer inspection, there is definitely a notable difference between 4.4.0-59 and 4.4.0-63/4.8.0-34.
The symptoms I'm seeing are:
Upgrading 4.4.0-57 -> 4.4.0-59:
- /proc/meminfo:Buffers: Up from 9GB to 15GB
- /proc/meminfo:Cached: Up from 5GB to 10GB
- /proc/meminfo:SReclaimable: Down from 15GB to 5GB.
- /proc/meminfo:SUnreclaim: No notable change
Upgrading 4.4.0-57 -> 4.8.0-34:
- /proc/meminfo:Buffers: Up from 9GB to 14GB
- /proc/meminfo:Cached: Down from 5GB to 2GB
- /proc/meminfo:SReclaimable: Down from 15GB to 14GB.
- /proc/meminfo:SUnreclaim: No notable change
Downgrading 4.8.0-34 -> 4.4.0-63: (however uptime is only 1 hour)
- /proc/meminfo:Buffers: No notable change
- /proc/meminfo:Cached: No notable change
- /proc/meminfo:SReclaimable: No notable change
- /proc/meminfo:SUnreclaim: No notable change
Setting vm.vfs_cache_pressure = 300 seems to have a positive effect of not causing OOMs.
Downgrading to 4.4.0-57 also works.
ProblemType: Bug
DistroRelease: Ubuntu 16.04
Package: linux-image-4.4.0-63-generic 4.4.0-63.84
ProcVersionSignature: Ubuntu 4.4.0-63.84-generic 4.4.44
Uname: Linux 4.4.0-63-generic x86_64
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access '/dev/snd/': No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.1-0ubuntu2.5
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
Date: Mon Feb 20 16:15:56 2017
InstallationDate: Installed on 2012-06-04 (1721 days ago)
InstallationMedia:
IwConfig:
lo no wireless extensions.
eth0 no wireless extensions.
Lsusb: Error: [Errno 2] No such file or directory: 'lsusb'
MachineType: System manufacturer System Product Name
PciMultimedia:
ProcFB: 0 VESA VGA
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-4.4.0-63-generic root=UUID=b790930f-ad81-4b27-a353-a4b3d6a29007 ro nomodeset nomdmonddf nomdmonisw
RelatedPackageVersions:
linux-restricted-modules-4.4.0-63-generic N/A
linux-backports-modules-4.4.0-63-generic N/A
linux-firmware 1.157.8
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: Upgraded to xenial on 2017-02-16 (4 days ago)
dmi.bios.date: 10/17/2011
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 1106
dmi.board.asset.tag: To be filled by O.E.M.
dmi.board.name: P8H67-M PRO
dmi.board.vendor: ASUSTeK Computer INC.
dmi.board.version: Rev 1.xx
dmi.chassis.asset.tag: Asset-1234567890
dmi.chassis.type: 3
dmi.chassis.vendor: Chassis Manufacture
dmi.chassis.version: Chassis Version
dmi.modalias: dmi:bvnAmericanMegatrendsInc.:bvr1106:bd10/17/2011:svnSystemmanufacturer:pnSystemProductName:pvrSystemVersion:rvnASUSTeKComputerINC.:rnP8H67-MPRO:rvrRev1.xx:cvnChassisManufacture:ct3:cvrChassisVersion:
dmi.product.name: System Product Name
dmi.product.version: System Version
dmi.sys.vendor: System manufacturer
Same as #1655842 - Started seeing oom-killer on multiple servers upgraded to 4.4.0-59.
Unlike #1655842, also seeing the same oom-killer on multiple servers updated to 4.8.0-34.
First upgraded them all 4.8 servers 4.8.0-36, then downgraded a few to 4.4.0-63. I am seeing no change in the memory usage, so I can only assume that 4.4.0-63 is also bugged with the same problem as 4.4.0-59 and 4.8.0-34. Either because #1655842 is not fixed, or it is only fixed for certain kinds of workloads.
However, on closer inspection, there is definitely a notable difference between 4.4.0-59 and 4.4.0-63/4.8.0-34.
The symptoms I'm seeing are:
Upgrading 4.4.0-57 -> 4.4.0-59: Buffers: Up from 9GB to 15GB Cached: Up from 5GB to 10GB SReclaimable: Down from 15GB to 5GB. SUnreclaim: No notable change
- /proc/meminfo:
- /proc/meminfo:
- /proc/meminfo:
- /proc/meminfo:
Upgrading 4.4.0-57 -> 4.8.0-34: Buffers: Up from 9GB to 14GB Cached: Down from 5GB to 2GB SReclaimable: Down from 15GB to 14GB. SUnreclaim: No notable change
- /proc/meminfo:
- /proc/meminfo:
- /proc/meminfo:
- /proc/meminfo:
Downgrading 4.8.0-34 -> 4.4.0-63: (however uptime is only 1 hour) Buffers: No notable change Cached: No notable change SReclaimable: No notable change SUnreclaim: No notable change
- /proc/meminfo:
- /proc/meminfo:
- /proc/meminfo:
- /proc/meminfo:
Setting vm.vfs_ cache_pressure = 300 seems to have a positive effect of not causing OOMs.
Downgrading to 4.4.0-57 also works.
ProblemType: Bug 4.4.0-63- generic 4.4.0-63.84 ature: Ubuntu 4.4.0-63.84-generic 4.4.44
DistroRelease: Ubuntu 16.04
Package: linux-image-
ProcVersionSign
Uname: Linux 4.4.0-63-generic x86_64
AlsaDevices: Error: command ['ls', '-l', '/dev/snd/'] failed with exit code 2: ls: cannot access '/dev/snd/': No such file or directory
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.1-0ubuntu2.5
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
Date: Mon Feb 20 16:15:56 2017
InstallationDate: Installed on 2012-06-04 (1721 days ago)
InstallationMedia:
IwConfig:
lo no wireless extensions.
eth0 no wireless extensions.
Lsusb: Error: [Errno 2] No such file or directory: 'lsusb'
MachineType: System manufacturer System Product Name
PciMultimedia:
ProcFB: 0 VESA VGA /vmlinuz- 4.4.0-63- generic root=UUID= b790930f- ad81-4b27- a353-a4b3d6a290 07 ro nomodeset nomdmonddf nomdmonisw ersions: restricted- modules- 4.4.0-63- generic N/A backports- modules- 4.4.0-63- generic N/A asset.tag: To be filled by O.E.M. asset.tag: Asset-1234567890 version: Chassis Version MegatrendsInc. :bvr1106: bd10/17/ 2011:svnSystemm anufacturer: pnSystemProduct Name:pvrSystemV ersion: rvnASUSTeKCompu terINC. :rnP8H67- MPRO:rvrRev1. xx:cvnChassisMa nufacture: ct3:cvrChassisV ersion: version: System Version
ProcKernelCmdLine: BOOT_IMAGE=
RelatedPackageV
linux-
linux-
linux-firmware 1.157.8
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: Upgraded to xenial on 2017-02-16 (4 days ago)
dmi.bios.date: 10/17/2011
dmi.bios.vendor: American Megatrends Inc.
dmi.bios.version: 1106
dmi.board.
dmi.board.name: P8H67-M PRO
dmi.board.vendor: ASUSTeK Computer INC.
dmi.board.version: Rev 1.xx
dmi.chassis.
dmi.chassis.type: 3
dmi.chassis.vendor: Chassis Manufacture
dmi.chassis.
dmi.modalias: dmi:bvnAmerican
dmi.product.name: System Product Name
dmi.product.
dmi.sys.vendor: System manufacturer