nova-novncproxy leaving behind zombie processes

[Joe T]

Hello,

As reported in this thread (http://www.gossamer-threads.com/lists/openstack/dev/18862) nova-novncproxy / novnc is leaving behind zombie processes after someone uses the VNC console:

$ pstree -Aapn

  |-python,109347 /usr/bin/nova-novncproxy --flagfile=/etc/nova/nova.conf --web ...
  | |-(python,95613)
  | |-(python,95624)
  | `-(python,95625)

$ ps aux | grep 95613
nova 95613 0.0 0.0 0 0 ? Z Sep08 0:00 [python] <defunct>

Please let me know if any further details or information is needed -- I'm not sure exactly what to collect.

Thanks,
Joe

[Rafi Khardalian (rkhardalian)]

I can confirm this one. We see it occurring in every deployment.

[Vish Ishaya (vishvananda)]

Can someone verify if this happens in devstack? Need to know if this needs to be investigated in folsom or if it is essex only.

[Joe T (joe-topjian-v)]

I was able to reproduce this issue with a recent (as of today) install of devstack:

$ pstree -Aapn

  | |-bash,26889
  | | `-python,30368 ./utils/nova-novncproxy --config-file /etc/nova/nova.conf ...
  | | |-(python,31700)
  | | `-(python,31701)

$ ps aux | grep Z

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
ubuntu 31894 0.0 0.0 0 0 pts/12 Z+ 03:01 0:00 [python] <defunct>
ubuntu 31895 0.0 0.0 0 0 pts/12 Z+ 03:01 0:00 [python] <defunct>

[Joe T (joe-topjian-v)]

Oops, the last output was actually another set of zombie processes from a second test (for verification). For completeness, the output should have been:

$ ps aux | grep Z

ubuntu 31700 0.0 0.0 0 0 pts/12 Z+ 02:59 0:00 [python] <defunct>
ubuntu 31701 0.0 0.0 0 0 pts/12 Z+ 02:59 0:00 [python] <defunct>

[Vish Ishaya (vishvananda)]

Potential fix for this in websockify library:

https://github.com/kanaka/websockify/pull/61

currently devstack is pulling novnc directly so that will need to be changed to see if this fix works.

You could manually test by making the same fix in:

/opt/stack/noVNC/utils/websockify.py

[Joe T (joe-topjian-v)]

Hi Vish,

Your comments sound specific to devstack. Do you think fixing the issue in Essex is possible? I'm more concerned about Essex.

Thanks,
Joe

[Joe T (joe-topjian-v)]

Hello,

This is also a problem in Folsom. The patch that you linked to is included in the websockify package from the Ubuntu 12.04 folsom repo.

Thanks,
Joe

[pippo (claudio-enjoy)]

same problem in grizzly ubuntu 12.04 with
  nova-novncproxy 1:2013.1-0ubuntu1~cloud0 OpenStack Compute - NoVNC proxy

in http://ubuntu-cloud.archive.canonical.com/ubuntu precise-updates/grizzly main

[Matteo Panella (mpanella)]

The problem persists in nova-novncproxy 1:2013.1.1-0ubuntu2~cloud0 from the Ubuntu Cloud Grizzly repo for precise.

[Jason Dillaman (jdillaman)]

Potential fix in the websockify library: https://github.com/kanaka/websockify/pull/85

[Joe T (joe-topjian-v)]

I quickly tested the latest websockify module and, on first glance, it looks like it fixes this bug. Thanks, Jason!

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/94778

[Thomas Goirand (thomas-goirand)]

Hi,

I have updated the Debian package to version 0.6.0+dfsg1-1. I guess Ubuntu users shall rebuild the package from the Debian source package, or use one of my (non-official) Ubuntu repository. Can anyone try this?

For Ubuntu trusty:
http://icehouse.dev-ubuntu.pkgs.enovance.com/debian/pool/trusty-icehouse-backports/main/w/websockify/

For Ubuntu precise:
http://icehouse.dev-precise.pkgs.enovance.com/debian/pool/precise-icehouse-backports/main/w/websockify/

For Debian Wheezy:
http://icehouse.dev-debian.pkgs.enovance.com/debian/pool/icehouse/main/w/websockify/

Note that I have no rights to update the official Ubuntu packages, I am only the poor Debian maintainer from which Ubuntu is just syncing from! :D

[OpenStack Infra (hudson-openstack)]

Change abandoned by Joe Gordon (<email address hidden>) on branch: master
Review: https://review.openstack.org/94778
Reason: Is this still active? Nothing has happened in a few months so marking as abandoned. Feel free to restore this.

[Sean Dague (sdague)]

This is fixed by a requirements bump - https://review.openstack.org/#/c/122702/

[Davanum Srinivas (DIMS) (dims-v)]

The review was abandoned..

[Deliang Fan (vanderliang)]

https://review.openstack.org/#/c/91663/