We have MOS 5.1.1 installed but we still see some behaviour as described here. Here are the relevant packages on our controller:
root@node-1:~# dpkg -l | grep nova-api
ii nova-api 1:2014.1.3-fuel5.1.1~mira3 OpenStack Compute - API frontend
root@node-1:~# dpkg -l | grep oslo
ii python-oslo.config 1:1.2.1-0ubuntu1~cloud0 Common code for Openstack Projects (configuration API)
ii python-oslo.i18n 1.0.0-fuel5.1.1~mira1 Oslo Internationalization Utilities - Python 2.x
ii python-oslo.messaging 1.3.1-fuel5.1.1~mira0 oslo messaging library
ii python-oslo.rootwrap 1.0.0-0ubuntu2 Fine filtering of shell commands as 'root'
ii python-oslo.serialization 1.0.0-fuel5.1.1~mira0 utilities for serialization , especially JSON - Python 2.x
ii python-oslo.utils 1.0.0-fuel5.1.1~mira0 set of utility functions for OpenStack - Python 2.x
ii python-oslo.vmware 0.2-0ubuntu1 oslo vmware library
root@node-1:~# dpkg -l | grep rabbit
ii rabbitmq-server 3.3.5-ubuntu3 AMQP server written in Erlang
We see issues that the rabbit closes connections to many of the processes (it seems many of the
nova-api processes); when there is new request to nova-api, it often has a broken pipe and then
it started a new connection to rabbitmq. So we have this sequence:
- rabbit kills connection to many nova-api processes due to heartbeat timeouts
- nova-api receives request
- nova-api tries to talk to rabbit but experiences broken pipe
- (many) nova-api processes restart communications with rabbit
- nova-api is able to then communicate
It does not have much of an impact in userland, but it's suboptimal from pov of keeping
clean logs.
We have MOS 5.1.1 installed but we still see some behaviour as described here. Here are the relevant packages on our controller:
root@node-1:~# dpkg -l | grep nova-api 1.3-fuel5. 1.1~mira3 OpenStack Compute - API frontend
ii nova-api 1:2014.
root@node-1:~# dpkg -l | grep oslo 0ubuntu1~ cloud0 Common code for Openstack Projects (configuration API) 1.1~mira1 Oslo Internationaliz ation Utilities - Python 2.x oslo.messaging 1.3.1-fuel5. 1.1~mira0 oslo messaging library oslo.rootwrap 1.0.0-0ubuntu2 Fine filtering of shell commands as 'root' oslo.serializat ion 1.0.0-fuel5. 1.1~mira0 utilities for serialization , especially JSON - Python 2.x 1.1~mira0 set of utility functions for OpenStack - Python 2.x
ii python-oslo.config 1:1.2.1-
ii python-oslo.i18n 1.0.0-fuel5.
ii python-
ii python-
ii python-
ii python-oslo.utils 1.0.0-fuel5.
ii python-oslo.vmware 0.2-0ubuntu1 oslo vmware library
root@node-1:~# dpkg -l | grep rabbit
ii rabbitmq-server 3.3.5-ubuntu3 AMQP server written in Erlang
We see issues that the rabbit closes connections to many of the processes (it seems many of the
nova-api processes); when there is new request to nova-api, it often has a broken pipe and then
it started a new connection to rabbitmq. So we have this sequence:
- rabbit kills connection to many nova-api processes due to heartbeat timeouts
- nova-api receives request
- nova-api tries to talk to rabbit but experiences broken pipe
- (many) nova-api processes restart communications with rabbit
- nova-api is able to then communicate
It does not have much of an impact in userland, but it's suboptimal from pov of keeping
clean logs.