Bug #1261338 “lxc-stop and even lxc-stop -k can hang” : Bugs : lxc package : Ubuntu

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2013-12-16:

#1

What exactly is the kernel crash being triggered on the host?

What does 'sudo strace -f lxc-stop -k -n <container>' show?

What does the file outout show after running 'sudo lxc-stop -l info -o outout -n <container>' ?

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2013-12-16:

#2

(I should've explicitly asked: exactly what steps are you taking to reproduce this)

Changed in lxc (Ubuntu):
status:	New → Incomplete
importance:	Undecided → High

Revision history for this message

Stéphane Graber (stgraber) wrote on 2014-01-16:

#3

I vaguely remember discussing this on IRC.

IIRC the case was something in the container getting stuck in D state because of a kernel bug. In such case, the container couldn't be killed as anything attempting to do so would get stuck too.

Anyway, this is pretty unfortunate but it's a kernel bug and there's nothing LXC can do to kill an unkillable process so I think our current behaviour is fine and the kernel is what should get fixed.

Changed in lxc (Ubuntu):
status:	Incomplete → Invalid

Revision history for this message

Vincent Ladeuil (vila) wrote on 2014-01-16:

#4

Sorry for not providing timely feedback there (nor a reproducing env).

Truth is this hasn't recur so marking invalid seems fair.

Thanks for the help nevertheless, if it happens again, I'll make sure to have such a reproducing env before re-opening this bug.

Revision history for this message

ivan (anubis00786) wrote on 2014-08-14:

#6

Download full text (23.6 KiB)

This problem was repeated to me.
executed commands

sudo apt-get install lxc
sudo lxc-create -t ubuntu -n p1
sudo lxc-start -n p1
sudo lxc-stop -n p1 ==execution hangs

sudo strace -f lxc-stop -k -n p1==

execve("/usr/bin/lxc-stop", ["lxc-stop", "-k", "-n", "p1"], [/* 26 vars */]) = 0
brk(0) = 0xb79de000
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7768000
access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=106845, ...}) = 0
mmap2(NULL, 106845, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb774d000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/usr/lib/i386-linux-gnu/liblxc.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200\205\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=429132, ...}) = 0
mmap2(NULL, 427812, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb76e4000
mmap2(0xb774b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x67000) = 0xb774b000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\233\1\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1758972, ...}) = 0
mmap2(NULL, 1768060, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7534000
mprotect(0xb76dd000, 4096, PROT_NONE) = 0
mmap2(0xb76de000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a9000) = 0xb76de000
mmap2(0xb76e1000, 10876, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb76e1000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libcap.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\16\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=18048, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7533000
mmap2(NULL, 20824, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb752d000
mmap2(0xb7531000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0xb7531000
close(3) = 0
access("/etc/ld.so.nohwcap", F_OK) = -1 ENOENT (No such file or directory)
open("/usr/lib/i386-linux-gnu/libapparmor.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\20\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=42820, ...}) = 0
mmap2(NULL, 45564, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7521000
mmap2(0xb752b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9000) = 0xb752b000
close(3) = 0
access(...

This problem was repeated to me.
executed commands

sudo apt-get install lxc
sudo lxc-create -t ubuntu -n p1
sudo lxc-start -n p1 
sudo lxc-stop -n p1 ==execution hangs

sudo strace -f lxc-stop -k -n p1==

execve("/usr/bin/lxc-stop", ["lxc-stop", "-k", "-n", "p1"], [/* 26 vars */]) = 0
brk(0)                                  = 0xb79de000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap2(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7768000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat64(3, {st_mode=S_IFREG|0644, st_size=106845, ...}) = 0
mmap2(NULL, 106845, PROT_READ, MAP_PRIVATE, 3, 0) = 0xb774d000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/i386-linux-gnu/liblxc.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200\205\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=429132, ...}) = 0
mmap2(NULL, 427812, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb76e4000
mmap2(0xb774b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x67000) = 0xb774b000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\340\233\1\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=1758972, ...}) = 0
mmap2(NULL, 1768060, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7534000
mprotect(0xb76dd000, 4096, PROT_NONE)   = 0
mmap2(0xb76de000, 12288, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1a9000) = 0xb76de000
mmap2(0xb76e1000, 10876, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb76e1000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libcap.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\16\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=18048, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7533000
mmap2(NULL, 20824, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb752d000
mmap2(0xb7531000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0xb7531000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/i386-linux-gnu/libapparmor.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300\20\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=42820, ...}) = 0
mmap2(NULL, 45564, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7521000
mmap2(0xb752b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x9000) = 0xb752b000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0pF\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=133072, ...}) = 0
mmap2(NULL, 142348, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb74fe000
mmap2(0xb751e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1f000) = 0xb751e000
mmap2(0xb7520000, 3084, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb7520000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/i386-linux-gnu/libseccomp.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\2208\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=62912, ...}) = 0
mmap2(NULL, 65636, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb74ed000
mmap2(0xb74fa000, 16384, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0xc000) = 0xb74fa000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libcgmanager.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0p \0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=116028, ...}) = 0
mmap2(NULL, 118800, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb74cf000
mmap2(0xb74eb000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b000) = 0xb74eb000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libnih.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\2604\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=95616, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb74ce000
mmap2(NULL, 98612, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb74b5000
mmap2(0xb74cc000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0xb74cc000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libnih-dbus.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\300 \0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=34168, ...}) = 0
mmap2(NULL, 36884, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb74ab000
mmap2(0xb74b3000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x7000) = 0xb74b3000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libdbus-1.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0pO\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=300888, ...}) = 0
mmap2(NULL, 303932, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7460000
mmap2(0xb74a9000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x48000) = 0xb74a9000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\220\n\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=9816, ...}) = 0
mmap2(NULL, 12428, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb745c000
mmap2(0xb745e000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0xb745e000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320]\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0755, st_size=130518, ...}) = 0
mmap2(NULL, 111276, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7440000
mmap2(0xb7458000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0xb7458000
mmap2(0xb745a000, 4780, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0xb745a000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\200\20\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=247196, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb743f000
mmap2(NULL, 249952, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb7401000
mmap2(0xb743d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3b000) = 0xb743d000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\320\n\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=13856, ...}) = 0
mmap2(NULL, 16512, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb73fc000
mmap2(0xb73ff000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0xb73ff000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/i386-linux-gnu/librt.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\1\1\1\0\0\0\0\0\0\0\0\0\3\0\3\0\1\0\0\0\260\31\0\0004\0\0\0"..., 512) = 512
fstat64(3, {st_mode=S_IFREG|0644, st_size=30696, ...}) = 0
mmap2(NULL, 33352, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0xb73f3000
mmap2(0xb73fa000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6000) = 0xb73fa000
close(3)                                = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb73f2000
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb73f1000
set_thread_area({entry_number:-1 -> 6, base_addr:0xb73f17c0, limit:1048575, seg_32bit:1, contents:0, read_exec_only:0, limit_in_pages:1, seg_not_present:0, useable:1}) = 0
mprotect(0xb76de000, 8192, PROT_READ)   = 0
mprotect(0xb7458000, 4096, PROT_READ)   = 0
mprotect(0xb73fa000, 4096, PROT_READ)   = 0
mprotect(0xb73ff000, 4096, PROT_READ)   = 0
mprotect(0xb743d000, 4096, PROT_READ)   = 0
mprotect(0xb745e000, 4096, PROT_READ)   = 0
mprotect(0xb74a9000, 4096, PROT_READ)   = 0
mprotect(0xb74cc000, 4096, PROT_READ)   = 0
mprotect(0xb74b3000, 4096, PROT_READ)   = 0
mprotect(0xb74eb000, 4096, PROT_READ)   = 0
mprotect(0xb74fa000, 12288, PROT_READ)  = 0
mprotect(0xb751e000, 4096, PROT_READ)   = 0
mprotect(0xb752b000, 4096, PROT_READ)   = 0
mprotect(0xb7531000, 4096, PROT_READ)   = 0
mprotect(0xb774b000, 4096, PROT_READ)   = 0
mprotect(0xb778f000, 4096, PROT_READ)   = 0
mprotect(0xb778b000, 4096, PROT_READ)   = 0
munmap(0xb774d000, 106845)              = 0
set_tid_address(0xb73f1828)             = 7860
set_robust_list(0xb73f1830, 12)         = 0
futex(0xbfb240e4, FUTEX_WAIT_BITSET_PRIVATE|FUTEX_CLOCK_REALTIME, 1, NULL, b73f17c0) = -1 EAGAIN (Resource temporarily unavailable)
rt_sigaction(SIGRTMIN, {0xb74457d0, [], SA_SIGINFO}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0xb7445850, [], SA_RESTART|SA_SIGINFO}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM_INFINITY}) = 0
uname({sys="Linux", node="ivan-Satellite-A40", ...}) = 0
statfs64("/sys/fs/selinux", 84, 0xbfb2427c) = -1 ENOENT (No such file or directory)
statfs64("/selinux", 84, 0xbfb2427c)    = -1 ENOENT (No such file or directory)
brk(0)                                  = 0xb79de000
brk(0xb79ff000)                         = 0xb79ff000
open("/proc/filesystems", O_RDONLY|O_LARGEFILE) = 3
fstat64(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7767000
read(3, "nodev\tsysfs\nnodev\trootfs\nnodev\tr"..., 1024) = 343
read(3, "", 1024)                       = 0
close(3)                                = 0
munmap(0xb7767000, 4096)                = 0
open("/proc/self/cgroup", O_RDONLY|O_CLOEXEC) = 3
fcntl64(3, F_GETFL)                     = 0 (flags O_RDONLY)
fstat64(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7767000
_llseek(3, 0, [0], SEEK_CUR)            = 0
read(3, "11:hugetlb:/user/1000.user/c2.se"..., 1024) = 372
read(3, "", 1024)                       = 0
close(3)                                = 0
munmap(0xb7767000, 4096)                = 0
clock_getres(CLOCK_MONOTONIC, {0, 1})   = 0
getresuid32([0], [0], [0])              = 0
getresgid32([0], [0], [0])              = 0
socket(PF_LOCAL, SOCK_STREAM|SOCK_CLOEXEC, 0) = 3
connect(3, {sa_family=AF_LOCAL, sun_path="/sys/fs/cgroup/cgmanager/sock"}, 31) = 0
fcntl64(3, F_GETFL)                     = 0x2 (flags O_RDWR)
fcntl64(3, F_SETFL, O_RDWR|O_NONBLOCK)  = 0
geteuid32()                             = 0
getsockname(3, {sa_family=AF_LOCAL, NULL}, [2]) = 0
poll([{fd=3, events=POLLOUT}], 1, 0)    = 1 ([{fd=3, revents=POLLOUT}])
send(3, "\0", 1, MSG_NOSIGNAL)          = 1
send(3, "AUTH EXTERNAL 30\r\n", 18, MSG_NOSIGNAL) = 18
poll([{fd=3, events=POLLIN}], 1, -1)    = 1 ([{fd=3, revents=POLLIN}])
read(3, "OK 991ae3692e436944b196d8ee53ec7"..., 2048) = 37
poll([{fd=3, events=POLLOUT}], 1, -1)   = 1 ([{fd=3, revents=POLLOUT}])
send(3, "NEGOTIATE_UNIX_FD\r\n", 19, MSG_NOSIGNAL) = 19
poll([{fd=3, events=POLLIN}], 1, -1)    = 1 ([{fd=3, revents=POLLIN}])
read(3, "AGREE_UNIX_FD\r\n", 2048)      = 15
poll([{fd=3, events=POLLOUT}], 1, -1)   = 1 ([{fd=3, revents=POLLOUT}])
send(3, "BEGIN\r\n", 7, MSG_NOSIGNAL)   = 7
poll([{fd=3, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=3, revents=POLLOUT}])
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\0018\0\0\0\1\0\0\0h\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 120}, {" \0\0\0org.linuxcontainers.cgmanage"..., 56}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 176
clock_gettime(CLOCK_MONOTONIC, {10265, 531676835}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\10\0\0\0\1\0\0\0\17\0\0\0\5\1u\0\1\0\0\0\10\1g\0\1v\0\0"..., 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 40
recvmsg(3, 0xbfb23f80, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
geteuid32()                             = 0
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1\30\0\0\0\2\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\7\0\0\0hugetlb\0\1\0\0\0/\0\0\0\264\36\0\0", 24}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 168
clock_gettime(CLOCK_MONOTONIC, {10265, 532679702}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\2\0\0\0\10\0\0\0\5\1u\0\2\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1\34\0\0\0\3\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\n\0\0\0perf_event\0\0\1\0\0\0/\0\0\0\264\36\0\0", 28}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 172
clock_gettime(CLOCK_MONOTONIC, {10265, 550900729}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\3\0\0\0\10\0\0\0\5\1u\0\3\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1\30\0\0\0\4\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\5\0\0\0blkio\0\0\0\1\0\0\0/\0\0\0\264\36\0\0", 24}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 168
clock_gettime(CLOCK_MONOTONIC, {10265, 551586091}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\4\0\0\0\10\0\0\0\5\1u\0\4\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1\30\0\0\0\5\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\7\0\0\0freezer\0\1\0\0\0/\0\0\0\264\36\0\0", 24}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 168
clock_gettime(CLOCK_MONOTONIC, {10265, 552301556}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\5\0\0\0\10\0\0\0\5\1u\0\5\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1\30\0\0\0\6\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\7\0\0\0devices\0\1\0\0\0/\0\0\0\264\36\0\0", 24}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 168
clock_gettime(CLOCK_MONOTONIC, {10265, 553018767}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\6\0\0\0\10\0\0\0\5\1u\0\6\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1\30\0\0\0\7\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\6\0\0\0memory\0\0\1\0\0\0/\0\0\0\264\36\0\0", 24}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 168
clock_gettime(CLOCK_MONOTONIC, {10265, 553693864}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\7\0\0\0\10\0\0\0\5\1u\0\7\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1\30\0\0\0\10\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\7\0\0\0cpuacct\0\1\0\0\0/\0\0\0\264\36\0\0", 24}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 168
clock_gettime(CLOCK_MONOTONIC, {10265, 554415964}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\10\0\0\0\10\0\0\0\5\1u\0\10\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1\24\0\0\0\t\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\3\0\0\0cpu\0\1\0\0\0/\0\0\0\264\36\0\0", 20}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 164
clock_gettime(CLOCK_MONOTONIC, {10265, 555090851}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\t\0\0\0\10\0\0\0\5\1u\0\t\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1\30\0\0\0\n\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\6\0\0\0cpuset\0\0\1\0\0\0/\0\0\0\264\36\0\0", 24}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 168
clock_gettime(CLOCK_MONOTONIC, {10265, 555568223}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\n\0\0\0\10\0\0\0\5\1u\0\n\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
sendmsg(3, {msg_name(0)=NULL, msg_iov(2)=[{"l\1\0\1 \0\0\0\v\0\0\0y\0\0\0\1\1o\0\36\0\0\0/org/lin"..., 144}, {"\f\0\0\0name=systemd\0\0\0\0\1\0\0\0/\0\0\0\264\36\0\0", 32}], msg_controllen=0, msg_flags=0}, MSG_NOSIGNAL) = 176
clock_gettime(CLOCK_MONOTONIC, {10265, 556070739}) = 0
poll([{fd=3, events=POLLIN}], 1, 25000) = 1 ([{fd=3, revents=POLLIN}])
recvmsg(3, {msg_name(0)=NULL, msg_iov(1)=[{"l\2\1\1\0\0\0\0\v\0\0\0\10\0\0\0\5\1u\0\v\0\0\0", 2048}], msg_controllen=0, msg_flags=MSG_CMSG_CLOEXEC}, MSG_CMSG_CLOEXEC) = 24
recvmsg(3, 0xbfb24030, MSG_CMSG_CLOEXEC) = -1 EAGAIN (Resource temporarily unavailable)
close(3)                                = 0
stat64("/sys/kernel/security/apparmor/features/mount/mask", {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
open("/sys/module/apparmor/parameters/enabled", O_RDONLY) = 3
fstat64(3, {st_mode=S_IFREG|0444, st_size=4096, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7767000
read(3, "Y\n", 4096)                    = 2
close(3)                                = 0
munmap(0xb7767000, 4096)                = 0
geteuid32()                             = 0
open("/etc/lxc/lxc.conf", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
geteuid32()                             = 0
geteuid32()                             = 0
access("/var", F_OK)                    = 0
access("/var/log", F_OK)                = 0
access("/var/log/lxc", F_OK)            = 0
getuid32()                              = 0
open("/var/log/lxc/p1.log", O_WRONLY|O_CREAT|O_APPEND|O_CLOEXEC, 0666) = 3
getuid32()                              = 0
geteuid32()                             = 0
mkdir("/", 0755)                        = -1 EEXIST (File exists)
mkdir("/run/", 0755)                    = -1 EEXIST (File exists)
mkdir("/run/lock/", 0755)               = -1 EEXIST (File exists)
mkdir("/run/lock/lxc//", 0755)          = -1 EEXIST (File exists)
mkdir("/run/lock/lxc//var/", 0755)      = -1 EEXIST (File exists)
mkdir("/run/lock/lxc//var/lib/", 0755)  = -1 EEXIST (File exists)
mkdir("/run/lock/lxc//var/lib/lxc", 0755) = -1 EEXIST (File exists)
stat64("/var/lib/lxc/p1/config", {st_mode=S_IFREG|0644, st_size=540, ...}) = 0
open("/run/lock/lxc//var/lib/lxc/p1", O_RDWR|O_CREAT, 0600) = 4
fcntl64(4, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=0}) = 0
access("/var/lib/lxc/p1/config", R_OK)  = 0
open("/var/lib/lxc/p1/config", O_RDONLY) = 5
fstat64(5, {st_mode=S_IFREG|0644, st_size=540, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7767000
read(5, "# Template used to create this c"..., 4096) = 540
access("/usr/share/lxc/config/ubuntu.common.conf", R_OK) = 0
open("/usr/share/lxc/config/ubuntu.common.conf", O_RDONLY) = 6
fstat64(6, {st_mode=S_IFREG|0644, st_size=2661, ...}) = 0
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7766000
read(6, "# Default pivot location\nlxc.piv"..., 4096) = 2661
read(6, "", 4096)                       = 0
close(6)                                = 0
munmap(0xb7766000, 4096)                = 0
time(NULL)                              = 1408017271
open("/dev/urandom", O_RDONLY)          = 6
fstat64(6, {st_mode=S_IFCHR|0666, st_rdev=makedev(1, 9), ...}) = 0
ioctl(6, SNDCTL_TMR_TIMEBASE or SNDRV_TIMER_IOCTL_NEXT_DEVICE or TCGETS, 0xbfb23ee8) = -1 EINVAL (Invalid argument)
mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7766000
read(6, "i\213<\216\340\3\27\254\363\373~J\230\23\374\272\254\244J\300\304\200h-M\303D\332\261\6g\354"..., 4096) = 4096
close(6)                                = 0
munmap(0xb7766000, 4096)                = 0
read(5, "", 4096)                       = 0
close(5)                                = 0
munmap(0xb7767000, 4096)                = 0
fcntl64(4, F_SETLK, {type=F_UNLCK, whence=SEEK_SET, start=0, len=0}) = 0
close(4)                                = 0
stat64("/var/lib/lxc/p1/partial", 0xbfb24150) = -1 ENOENT (No such file or directory)
socket(PF_LOCAL, SOCK_STREAM, 0)        = 4
connect(4, {sa_family=AF_LOCAL, sun_path=@"/var/lib/lxc/p1/command"}, 26) = 0
getuid32()                              = 0
getgid32()                              = 0
sendmsg(4, {msg_name(0)=NULL, msg_iov(1)=[{"\4\0\0\0\0\0\0\0\0\0\0\0", 12}], msg_controllen=24, {cmsg_len=24, cmsg_level=SOL_SOCKET, cmsg_type=SCM_CREDENTIALS{pid=7860, uid=0, gid=0}}, msg_flags=0}, MSG_NOSIGNAL) = 12
recvmsg(4,

ivan@ivan-Satellite-A40:~$ sudo lxc-ls == p1  
ivan@ivan-Satellite-A40:~$ sudo lxc-stop -l info -o outout -n 1p == 1p is not running

Revision history for this message

ivan (anubis00786) wrote on 2014-08-14:

#7

ivan@ivan-Satellite-A40:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.1 LTS
Release: 14.04
Codename: trusty

Revision history for this message

Serge Hallyn (serge-hallyn) wrote on 2014-08-14:

#8

Thanks for the information. Indeed, lxc-stop -k still works by talking to the lxc monitor. What appears to have happened here is that the monitor itself is frozen.

Could you please show full output of:

ps -ef

Then do 'ps -ef | grep lxc-start' to find the lxc-start process for the hung container. In one terminal do

sudo strace -f -odebug.out -p $pid

where $pid is the process id of the lxc-start. Then in another terminal again do

lxc-stop -n $container -k

and attach the file debug.out to this report.

Changed in lxc (Ubuntu):
status:	Invalid → Incomplete

Revision history for this message

ivan (anubis00786) wrote on 2014-08-16:

#9

post Edit (11.5 KiB, text/plain)

all post in file post

Revision history for this message

ivan (anubis00786) wrote on 2014-08-16:

#10

debug.out Edit (16.6 KiB, text/plain)

debug file

Revision history for this message

ivan (anubis00786) wrote on 2014-08-16:

#11

post Edit (12.4 KiB, text/plain)

The second option, if you do not use the -d
all post in file post

Revision history for this message

ivan (anubis00786) wrote on 2014-08-16:

#12

ivan.zip Edit (343 bytes, application/zip)

debug file

Serge Hallyn (serge-hallyn) on 2014-08-21

Changed in lxc (Ubuntu):
status:	Incomplete → New

Revision history for this message

Launchpad Janitor (janitor) wrote on 2015-08-13:

#13

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in lxc (Ubuntu):
status:	New → Confirmed

Revision history for this message

Stéphane Graber (stgraber) wrote on 2015-11-09:

#14

Not seeing anything obviously wrong here...

To rule out a few more things, when you get the hanging "lxc-stop -n container -k", can you do and paste the output of:
- ps faux
- dmesg

Thanks!

Changed in lxc (Ubuntu):
status:	Confirmed → Incomplete
importance:	High → Undecided

Revision history for this message

Launchpad Janitor (janitor) wrote on 2016-01-09:

#15

[Expired for lxc (Ubuntu) because there has been no activity for 60 days.]

Changed in lxc (Ubuntu):
status:	Incomplete → Expired

Revision history for this message

Vincent Ladeuil (vila) wrote on 2016-01-09:

#16

It never happened again (and may have been a misuse involving immortal sudo subprocesses but I'm blurry on the details), marking invalid but fixed released may be correct as well.

Changed in lxc (Ubuntu):
status:	Expired → Invalid

Ubuntu
lxc package

lxc-stop and even lxc-stop -k can hang

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches

Ubuntulxc package

lxc-stop and even lxc-stop -k can hang

Bug Description

Other bug subscribers

Bug attachments

Remote bug watches

Ubuntu
lxc package