nova-network leaks memory overtime and eventually stops responding
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Compute (nova) |
Invalid
|
High
|
Johannes Erdfelt |
Bug Description
I have noticed a trend in nova-networking where it seems to leak memory over time and then eventually stops responding or processing messages from RabbitMQ.
Stracing the process doesn't reveal anything enlightening. Please let me know if I can provide any further information?
Stopping and starting nova-network seems to be a work-around for this bug (despite LP#785955).
= Process information =
$ ps axfuwww | grep nova-network
nova 27939 0.0 0.0 45824 580 ? Ss Nov30 0:00 su -c nova-network --flagfile=
nova 27940 9.1 23.1 4730344 1416820 ? Dl Nov30 1566:29 \_ /usr/bin/python /usr/bin/
$ sudo strace -p 27940
Process 27940 attached - interrupt to quit
Swapped (from /proc/27940/smaps):
2788140 kB - 27940 (nova-network)
$ sudo rabbitmqctl list_queues | grep -E '^network'
network.cc 3056
= System Information =
$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_
DISTRIB_
DISTRIB_
$ dpkg-query --show nova-network
nova-network 2011.3-0ubuntu6.2
Changed in nova: | |
importance: | Undecided → High |
status: | New → Confirmed |
Changed in nova: | |
assignee: | nobody → Johannes Erdfelt (johannes.erdfelt) |
tags: | added: canonistack |
Added nova config below:
--dhcpbridge_ flagfile= /etc/nova/ nova.conf /usr/bin/ nova-dhcpbridge /var/log/ nova path=/var/ lib/nova path=/var/ lock/nova log_levels= DEBUG host=172. 16.58.1 n=mysql: //user: pass@172. 16.58.1/ nova api_servers= 172.16. 58.1:9292 manager= nova.network. manager. FlatDHCPManager interface= eth1 interface= eth0 range=172. 16.60.0/ 24 network_ dhcp_start= 172.16. 60.3 range=172. 16.93.64/ 26 d_auth dhcp_release= True helper= tgtadm
--dhcpbridge=
--logdir=
--state_
--lock_
--default_
--my_ip=172.16.58.1
--rabbit_
--sql_connectio
--glance_
--network_
--network_size=256
--public_
--flat_
--bridge=br100
--fixed_
--flat_
--floating_
--use_deprecate
--force_
--iscsi_
--verbose