memcg_move_charge_at_immigrate_test from ubuntu_ltp_controllers failed (shmget() failed: Invalid argument)

[Po-Hsu Lin]

Similar to bug 1837035

This memcg_move_charge_at_immigrate_test test failed on an i386 node "pepe" with Disco kernel.

It failed with:
memcg_move_charge_at_immigrate_test 5 TINFO: Warming up pid: 2192
memcg_process: shmget() failed: Invalid argument
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 168: kill: No such process

memcg_move_charge_at_immigrate_test 5 TFAIL: Process 2192 exited with 1 after warm up

<<<test_start>>>
tag=memcg_move_charge_at_immigrate stime=1563448078
cmdline="memcg_move_charge_at_immigrate_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
memcg_move_charge_at_immigrate_test 1 TINFO: Starting test 1
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 522: echo: echo: I/O error
memcg_move_charge_at_immigrate_test 1 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
memcg_move_charge_at_immigrate_test 1 TINFO: Running memcg_process --mmap-anon -s 135168
memcg_move_charge_at_immigrate_test 1 TINFO: Warming up pid: 2162
memcg_move_charge_at_immigrate_test 1 TINFO: Process is still here after warm up: 2162
memcg_move_charge_at_immigrate_test 1 TPASS: rss is 0 as expected
memcg_move_charge_at_immigrate_test 2 TPASS: cache is 0 as expected
memcg_move_charge_at_immigrate_test 3 TPASS: rss is 135168 as expected
memcg_move_charge_at_immigrate_test 4 TPASS: cache is 0 as expected
memcg_move_charge_at_immigrate_test 5 TINFO: Starting test 2
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 522: echo: echo: I/O error
memcg_move_charge_at_immigrate_test 5 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
memcg_move_charge_at_immigrate_test 5 TINFO: Running memcg_process --mmap-anon --shm --mmap-file -s 135168
memcg_move_charge_at_immigrate_test 5 TINFO: Warming up pid: 2192
memcg_process: shmget() failed: Invalid argument
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 168: kill: No such process

memcg_move_charge_at_immigrate_test 5 TFAIL: Process 2192 exited with 1 after warm up
memcg_move_charge_at_immigrate_test 6 TINFO: Starting test 3
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 522: echo: echo: I/O error
memcg_move_charge_at_immigrate_test 6 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
memcg_move_charge_at_immigrate_test 6 TINFO: Running memcg_process --mmap-anon --shm --mmap-file -s 135168
memcg_move_charge_at_immigrate_test 6 TINFO: Warming up pid: 2207
memcg_process: shmget() failed: Invalid argument
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 168: kill: No such process

memcg_move_charge_at_immigrate_test 6 TFAIL: Process 2207 exited with 1 after warm up
memcg_move_charge_at_immigrate_test 7 TINFO: Starting test 4
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 522: echo: echo: I/O error
memcg_move_charge_at_immigrate_test 7 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
memcg_move_charge_at_immigrate_test 7 TINFO: Running memcg_process --mmap-anon --shm -s 135168
memcg_move_charge_at_immigrate_test 7 TINFO: Warming up pid: 2222
memcg_process: shmget() failed: Invalid argument
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 168: kill: No such process

memcg_move_charge_at_immigrate_test 7 TFAIL: Process 2222 exited with 1 after warm up
<<<execution_status>>>
initiation_status="ok"
duration=4 termination_type=exited termination_id=1 corefile=no
cutime=9 cstime=2
<<<test_end>>>

Steps to run this test:
  git clone --depth=1 https://github.com/linux-test-project/ltp.git
  cd ltp; make autotools; ./configure; make; sudo make install
  echo "memcg_move_charge_at_immigrate memcg_move_charge_at_immigrate_test.sh" > /tmp/jobs
  sudo /opt/ltp/runltp -f /tmp/jobs

ProblemType: Bug
DistroRelease: Ubuntu 19.04
Package: linux-image-5.0.0-21-generic 5.0.0-21.22
ProcVersionSignature: User Name 5.0.0-21.22-generic 5.0.15
Uname: Linux 5.0.0-21-generic i686
AlsaDevices:
 total 0
 crw-rw---- 1 root audio 116, 1 Jul 18 10:01 seq
 crw-rw---- 1 root audio 116, 33 Jul 18 10:01 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay': 'aplay'
ApportVersion: 2.20.10-0ubuntu27.1
Architecture: i386
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord': 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
Date: Thu Jul 18 11:16:36 2019
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig': 'iwconfig'
Lsusb:
 Bus 002 Device 002: ID 8087:0020 Intel Corp. Integrated Rate Matching Hub
 Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
 Bus 001 Device 003: ID 0424:2514 Standard Microsystems Corp. USB 2.0 Hub
 Bus 001 Device 002: ID 8087:0020 Intel Corp. Integrated Rate Matching Hub
 Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
MachineType: Dell Inc. PowerEdge R310
PciMultimedia:

ProcFB: 0 mgadrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.0.0-21-generic root=UUID=7b91a2b8-2e02-407e-a51d-766f6d969020 ro
RelatedPackageVersions:
 linux-restricted-modules-5.0.0-21-generic N/A
 linux-backports-modules-5.0.0-21-generic N/A
 linux-firmware 1.178.3
RfKill: Error: [Errno 2] No such file or directory: 'rfkill': 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 08/17/2011
dmi.bios.vendor: Dell Inc.
dmi.bios.version: 1.8.2
dmi.board.name: 05XKKK
dmi.board.vendor: Dell Inc.
dmi.board.version: A05
dmi.chassis.type: 23
dmi.chassis.vendor: Dell Inc.
dmi.modalias: dmi:bvnDellInc.:bvr1.8.2:bd08/17/2011:svnDellInc.:pnPowerEdgeR310:pvr:rvnDellInc.:rn05XKKK:rvrA05:cvnDellInc.:ct23:cvr:
dmi.product.name: PowerEdge R310
dmi.sys.vendor: Dell Inc.

[Po-Hsu Lin (cypressyew)]

If you run this test again on the same node, it will pass:
<<<test_start>>>
tag=memcg_move_charge_at_immigrate stime=1563449267
cmdline="memcg_move_charge_at_immigrate_test.sh"
contacts=""
analysis=exit
<<<test_output>>>
memcg_move_charge_at_immigrate_test 1 TINFO: Starting test 1
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 522: echo: echo: I/O error
memcg_move_charge_at_immigrate_test 1 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
memcg_move_charge_at_immigrate_test 1 TINFO: Running memcg_process --mmap-anon -s 135168
memcg_move_charge_at_immigrate_test 1 TINFO: Warming up pid: 3263
memcg_move_charge_at_immigrate_test 1 TINFO: Process is still here after warm up: 3263
memcg_move_charge_at_immigrate_test 1 TPASS: rss is 0 as expected
memcg_move_charge_at_immigrate_test 2 TPASS: cache is 0 as expected
memcg_move_charge_at_immigrate_test 3 TPASS: rss is 135168 as expected
memcg_move_charge_at_immigrate_test 4 TPASS: cache is 0 as expected
memcg_move_charge_at_immigrate_test 5 TINFO: Starting test 2
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 522: echo: echo: I/O error
memcg_move_charge_at_immigrate_test 5 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
memcg_move_charge_at_immigrate_test 5 TINFO: Running memcg_process --mmap-anon --shm --mmap-file -s 135168
memcg_move_charge_at_immigrate_test 5 TINFO: Warming up pid: 3293
memcg_move_charge_at_immigrate_test 5 TINFO: Process is still here after warm up: 3293
memcg_move_charge_at_immigrate_test 5 TPASS: rss is 135168 as expected
memcg_move_charge_at_immigrate_test 6 TPASS: cache is 0 as expected
memcg_move_charge_at_immigrate_test 7 TPASS: rss is 0 as expected
memcg_move_charge_at_immigrate_test 8 TPASS: cache is 270336 as expected
memcg_move_charge_at_immigrate_test 9 TINFO: Starting test 3
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 522: echo: echo: I/O error
memcg_move_charge_at_immigrate_test 9 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
memcg_move_charge_at_immigrate_test 9 TINFO: Running memcg_process --mmap-anon --shm --mmap-file -s 135168
memcg_move_charge_at_immigrate_test 9 TINFO: Warming up pid: 3323
memcg_move_charge_at_immigrate_test 9 TINFO: Process is still here after warm up: 3323
memcg_move_charge_at_immigrate_test 9 TPASS: rss is 0 as expected
memcg_move_charge_at_immigrate_test 10 TPASS: cache is 270336 as expected
memcg_move_charge_at_immigrate_test 11 TPASS: rss is 135168 as expected
memcg_move_charge_at_immigrate_test 12 TPASS: cache is 0 as expected
memcg_move_charge_at_immigrate_test 13 TINFO: Starting test 4
/opt/ltp/testcases/bin/memcg_move_charge_at_immigrate_test.sh: 522: echo: echo: I/O error
memcg_move_charge_at_immigrate_test 13 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
memcg_move_charge_at_immigrate_test 13 TINFO: Running memcg_process --mmap-anon --shm -s 135168
memcg_move_charge_at_immigrate_test 13 TINFO: Warming up pid: 3353
memcg_move_charge_at_immigrate_test 13 TINFO: Process is still here after warm up: 3353
memcg_move_charge_at_immigrate_test 13 TPASS: rss is 135168 as expected
memcg_move_charge_at_immigrate_test 14 TPASS: cache is 135168 as expected
memcg_move...

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This change was made by a bot.

[Po-Hsu Lin (cypressyew)]

This issue can be found on B-AWS-5.0 ARM64 (5.0.0-1022.25~18.04.1) instance a1.large

 startup='Wed Nov 20 11:54:52 2019'
 memcg_move_charge_at_immigrate_test 1 TINFO: Starting test 1
 sh: echo: I/O error
 memcg_move_charge_at_immigrate_test 1 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
 memcg_move_charge_at_immigrate_test 1 TINFO: Running memcg_process --mmap-anon -s 135168
 memcg_move_charge_at_immigrate_test 1 TINFO: Warming up pid: 28342
 memcg_move_charge_at_immigrate_test 1 TINFO: Process is still here after warm up: 28342
 memcg_move_charge_at_immigrate_test 1 TPASS: rss is 0 as expected
 memcg_move_charge_at_immigrate_test 2 TPASS: cache is 0 as expected
 memcg_move_charge_at_immigrate_test 3 TPASS: rss is 135168 as expected
 memcg_move_charge_at_immigrate_test 4 TPASS: cache is 0 as expected
 memcg_move_charge_at_immigrate_test 5 TINFO: Starting test 2
 sh: echo: I/O error
 memcg_move_charge_at_immigrate_test 5 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
 memcg_move_charge_at_immigrate_test 5 TINFO: Running memcg_process --mmap-anon --shm --mmap-file -s 135168
 memcg_move_charge_at_immigrate_test 5 TINFO: Warming up pid: 28377
 memcg_move_charge_at_immigrate_test 5 TINFO: Process is still here after warm up: 28377
 memcg_move_charge_at_immigrate_test 5 TPASS: rss is 135168 as expected
 memcg_move_charge_at_immigrate_test 6 TPASS: cache is 0 as expected
 memcg_move_charge_at_immigrate_test 7 TPASS: rss is 0 as expected
 memcg_move_charge_at_immigrate_test 8 TFAIL: cache is 135168, 270336 expected
 memcg_move_charge_at_immigrate_test 9 TINFO: Starting test 3
 sh: echo: I/O error
 memcg_move_charge_at_immigrate_test 9 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
 memcg_move_charge_at_immigrate_test 9 TINFO: Running memcg_process --mmap-anon --shm --mmap-file -s 135168
 memcg_move_charge_at_immigrate_test 9 TINFO: Warming up pid: 28418
 memcg_move_charge_at_immigrate_test 9 TINFO: Process is still here after warm up: 28418
 memcg_move_charge_at_immigrate_test 9 TPASS: rss is 0 as expected
 memcg_move_charge_at_immigrate_test 10 TPASS: cache is 270336 as expected
 memcg_move_charge_at_immigrate_test 11 TPASS: rss is 135168 as expected
 memcg_move_charge_at_immigrate_test 12 TPASS: cache is 0 as expected
 memcg_move_charge_at_immigrate_test 13 TINFO: Starting test 4
 sh: echo: I/O error
 memcg_move_charge_at_immigrate_test 13 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
 memcg_move_charge_at_immigrate_test 13 TINFO: Running memcg_process --mmap-anon --shm -s 135168
 memcg_move_charge_at_immigrate_test 13 TINFO: Warming up pid: 28461
 memcg_move_charge_at_immigrate_test 13 TINFO: Process is still here after warm up: 28461
 memcg_move_charge_at_immigrate_test 13 TPASS: rss is 135168 as expected
 memcg_move_charge_at_immigrate_test 14 TPASS: cache is 135168 as expected
 memcg_move_charge_at_immigrate_test 15 TPASS: rss is 0 as expected
 memcg_move_charge_at_immigrate_test 16 TPASS: cache is 0 as expected
 tag=memcg_move_charge_at_immigrate stime=1574250892 dur=4 exit=exited stat=1 core=no cu=13 cs=10

[Po-Hsu Lin (cypressyew)]

This issue can be found on E-oracle as well
5.3.0-1019.21

[Po-Hsu Lin (cypressyew)]

Can be found on B-AWS-5.3 ARM64 a1.large

[Po-Hsu Lin (cypressyew)]

This will timeout on B-4.15 s390x zVM (passed on s390x LPAR):

 memcg_move_charge_at_immigrate_test 5 TINFO: Starting test 2
 sh: echo: I/O error
 memcg_move_charge_at_immigrate_test 5 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
 memcg_move_charge_at_immigrate_test 5 TINFO: Running memcg_process --mmap-anon --shm --mmap-file -s 135168
 memcg_move_charge_at_immigrate_test 5 TINFO: Warming up pid: 34479
 memcg_move_charge_at_immigrate_test 5 TINFO: Process is still here after warm up: 34479
 memcg_move_charge_at_immigrate_test 5 TBROK: timeouted on memory.usage_in_bytes

[Kelsey Steele (kelsey-steele)]

failed on Bionic/azure-4.15 : 4.15.0-1097.107 : amd64

275. 10/09 03:14:32 DEBUG| utils:0153| [stdout] startup='Fri Oct 9 02:17:45 2020'
276. 10/09 03:14:32 DEBUG| utils:0153| [stdout] memcg_move_charge_at_immigrate_test 1 TINFO: Starting test 1
277. 10/09 03:14:32 DEBUG| utils:0153| [stdout] sh: echo: I/O error
278. 10/09 03:14:32 DEBUG| utils:0153| [stdout] memcg_move_charge_at_immigrate_test 1 TINFO: set /dev/memcg/memory.use_hierarchy to 0 failed
279. 10/09 03:14:32 DEBUG| utils:0153| [stdout] memcg_move_charge_at_immigrate_test 1 TINFO: Running memcg_process --mmap-anon -s 135168
280. 10/09 03:14:32 DEBUG| utils:0153| [stdout] memcg_move_charge_at_immigrate_test 1 TINFO: Warming up pid: 7026
281. 10/09 03:14:32 DEBUG| utils:0153| [stdout] memcg_move_charge_at_immigrate_test 1 TINFO: Process is still here after warm up: 7026
282. 10/09 03:14:32 DEBUG| utils:0153| [stdout] memcg_move_charge_at_immigrate_test 1 TBROK: timeouted on memory.usage_in_bytes
283. 10/09 03:14:32 DEBUG| utils:0153| [stdout] tag=memcg_move_charge_at_immigrate stime=1602209865 dur=12 exit=exited stat=2 core=no cu=19 cs=11

[Po-Hsu Lin (cypressyew)]

On B-i386 node pepe 4.15.0-137.141, it's failing with:
memcg_move_charge_at_immigrate_test 2 TFAIL: cache is 135168, 270336 expected

[Kleber Sacilotto de Souza (kleber-souza)]

Found the same cache failure with bionic/linux-hwe-5.4 5.4.0-89.100~18.04.1 on node onza (i386):

memcg_move_charge_at_immigrate_test 2 TFAIL: cache is 135168, 270336 expected

[Krzysztof Kozlowski (krzk)]

Found on: focal/linux-azure/5.4.0-1063.66 (Standard_A2_v2)

[Krzysztof Kozlowski (krzk)]

2022.01.31/bionic/linux-ibm-gt-5.4/5.4.0-1009.11
(as expected since this happens on B/hwe-5.4)
memcg_move_charge_at_immigrate_test 2 TFAIL: cache is 135168, 270336 expected

[Portia Stephens (portias)]

Found on linux-xilinx-zynqmp-5.15.0-1020.24/arm64 on zcu106