container-updater deadlock

[Christopher Bartz]

We oberserved (using swift-2.15.1 on CentOS 7.4) that the container-updater (on all storage nodes) was not performing its task, due to a deadlock situation:

 [root@swift tmp]# ps aux | grep container-updater
 swift 22944 0.0 0.0 260672 23712 ? S Okt27 0:01 /usr/bin/python2 /usr/bin/swift-container-updater /etc/swift/container-server.conf
 swift 30002 0.2 0.0 260416 27276 ? Ss Okt27 40:12 /usr/bin/python2 /usr/bin/swift-container-updater /etc/swift/container-server.conf

strace shows that the child process is doing epoll_wait

 [root@tswift01 tmp]# strace -p 22944
 strace: Process 22944 attached
 epoll_wait(6, [], 1023, 60000) = 0
 epoll_wait(6, [], 1023, 60000) = 0
 epoll_wait(6, [], 1023, 60000) = 0
 epoll_wait(6, [], 1023, 60000) = 0
 epoll_wait(6, [], 1023, 60000) = 0

while the parent process waits for the termination of the child process:

 [root@tswift01 tmp]# strace -p 30002
 strace: Process 30002 attached
 wait4(-1,

Restarting the container-updater solved the problem, but just for a certain amount of time. The deadlock appeared again on all storage nodes.