Add respawn to upstart for neutron-plugin-openvswitch-agent

[Miroslav Anashkin]

It was discovered, under Ubuntu, upstart does not restart crashed neutron-plugin-openvswitch-agent automatically on the compute nodes.
It leads to network connectivity loss, since there is no other service to restart failed Neutron plugins on failure.

Please modify the package installation script to configure upstart to restart this service automatically.

For controllers we'll be disable upstart with Puppet.

Current package:
http://fuel-repository.mirantis.com/fwm/5.1.1/ubuntu/pool/main/neutron-plugin-openvswitch-agent_2014.1.3-fuel5.1.1~mira1_all.deb

[Sergey Kolekonov (skolekonov)]

I can't reproduce this problem using our packages from 5.1.1 and 6.0. We already have 'respawn' directive in upstart scripts for openvswitch-agent packages and after killing agent via 'kill -9' it respawns successfully.

[Alexander Ignatov (aignatov)]

In addition to Sergey's comment above and since that issue is not reproducible we need more debug logs or snapshot at least for further investigation.

[Miroslav Anashkin (manashkin)]

Will be provided internally. Logs are under NDA.

[Miroslav Anashkin (manashkin)]

BTW, there is working workaround:

In ubuntu you can insert "respawn" stanza into /etc/init/nova-compute.conf
if you wish. For example:
===================
respawn
respawn limit 10 5
===================
should respawn the service ten times in five second.
Please see details here: http://upstart.ubuntu.com/cookbook/#id192

[Fabrizio Soppelsa (fsoppelsa)]

Logs provided internally to the MOS-neutron team.
For any information, please contact me.

[OSCI Robot (oscirobot)]

DEB package nova has been built for project openstack/nova
Package version == 2014.2, package release == fuel6.1~mira20+git.602b9e7.ad49c2e

Changeset: https://review.fuel-infra.org/2204
project: openstack-build/nova-build
branch: openstack-ci/fuel-6.1/2014.2
author: Ivan Udovichenko
committer: Ivan Udovichenko
subject: Respawn Nova services
status: patchset-created

Files placed on repository:
nova-ajax-console-proxy_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api-ec2_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api-metadata_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api-os-compute_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api-os-volume_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-baremetal_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-cells_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-cert_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-common_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-kvm_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-libvirt_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-lxc_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-qemu_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-vmware_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-xen_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-conductor_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-console_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-consoleauth_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-doc_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-network_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-novncproxy_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-objectstore_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-scheduler_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-spiceproxy_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-volume_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-xvpvncproxy_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
python-nova_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb

NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-6.1-stable-2204/ubuntu

[OSCI Robot (oscirobot)]

DEB package nova has been built for project openstack/nova
Package version == 2014.2, package release == fuel6.1~mira20+git.602b9e7.f38868c

Changeset: https://review.fuel-infra.org/2204
project: openstack-build/nova-build
branch: openstack-ci/fuel-6.1/2014.2
author: Ivan Udovichenko
committer: Ivan Udovichenko
subject: Respawn Nova services
status: patchset-created

Files placed on repository:
nova-ajax-console-proxy_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api-ec2_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api-metadata_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api-os-compute_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api-os-volume_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-api_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-baremetal_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-cells_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-cert_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-common_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-kvm_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-libvirt_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-lxc_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-qemu_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-vmware_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute-xen_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-compute_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-conductor_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-console_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-consoleauth_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-doc_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-network_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-novncproxy_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-objectstore_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-scheduler_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-spiceproxy_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-volume_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
nova-xvpvncproxy_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb
python-nova_2014.2-fuel6.1~mira20+git.602b9e7.ad49c2e_all.deb

NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://osci-obs.vm.mirantis.net:82/ubuntu-fuel-6.1-stable-2204/ubuntu

[Bogdan Dobrelya (bogdando)]

MOS-Neutron team please verify the updated packages if the issue is resolved

[Bogdan Dobrelya (bogdando)]

The 5.1 amd 6.0 targets for this issue remained incomplete for about a month, setting to invalid

[Dmitry Mescheryakov (dmitrymex)]

Discussed the bug with Neutron team: nothing was found in Neutron so far, hence setting it as invalid. As for the similar issue in Nova, please file a separate bug for it.

[Fuel Devops McRobotson (fuel-devops-robot)]

Change abandoned by Ivan Udovichenko <email address hidden> on branch: openstack-ci/fuel-6.1/2014.2
Review: https://review.fuel-infra.org/2204
Reason: Fix will be in 7.0