Race condition in FWaaS service setup

Bug #1287554 reported by Mark T. Voelker
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Cisco Openstack
Fix Released
Medium
Mark T. Voelker
Havana
Fix Released
Medium
Mark T. Voelker
puppet-neutron
Fix Released
Undecided
Mark T. Voelker
Havana
Fix Released
Undecided
Unassigned

Bug Description

The neutron::services::fwaas class has a race condition: it may attempt to set up the /etc/neutron/fwaas_drive.ini file before the Neutron L3 agent is installed and before the /etc/neutron directory exists. This leads to errors on the first catalog run of the form:

Error: Could not set 'present' on ensure: No such file or directory - /etc/neutron/fwaas_driver.ini at 42:/usr/share/puppet/modules/neutron/manifests/services/fwaas.pp

As a result, the fwaas_driver.ini file doesn't get written and the FWaaS service doesn't start after the first catalog run. The workaround is to do a second catalog run, which will create the necessary files.

The solution for this is to ensure the l3 agent package is installed before we attempt to do the configuration.

Changed in puppet-neutron:
status: New → In Progress
assignee: nobody → Mark T. Voelker (mvoelker)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/77805

Changed in openstack-cisco:
assignee: nobody → Mark T. Voelker (mvoelker)
milestone: none → h.2
importance: Undecided → Medium
status: New → In Progress
Revision history for this message
Ken Schroeder (kschroed) wrote :

In H1. FWaaS agent did not work unless you updated /etc/init/neutron-l3-agent.conf with something like
exec start-stop-daemon --start --chuid neutron --exec /usr/bin/neutron-l3-agent -- --config-file=/etc/neutron/neutron.conf --config-file=/etc/neutron/l3_agent.ini --config-file /etc/neutron/fwaas_driver.ini --log-file=/var/log/neutron/l3-agent.log
root@svl-cc-net2-0

Not sure if that is changed in H2, but it was easier for us to inject the FWaaS config contents into /etc/neutron.conf since it is minimal as opposed to trying to modify the init service.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to puppet-neutron (stable/havana)

Fix proposed to branch: stable/havana
Review: https://review.openstack.org/78423

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-neutron (master)

Reviewed: https://review.openstack.org/77805
Committed: https://git.openstack.org/cgit/stackforge/puppet-neutron/commit/?id=cfedc81cb60cabdbb31f3809e9d56a54d771458b
Submitter: Jenkins
Branch: master

commit cfedc81cb60cabdbb31f3809e9d56a54d771458b
Author: Mark T. Voelker <email address hidden>
Date: Tue Mar 4 02:07:57 2014 -0500

    Fix FWaaS race condition

    The current FWaaS manifest suffers from a race condition: it may
    attempt to configure the FWaaS service before the Neutron packages
    on which it depends has been installed, and before the
    /etc/neutron directory exists. This leads to errors on the first
    catalog run, and the FWaaS service doesn't start. The errors
    go away on the second catalog run since by then the requisite
    directory exists. This patch adds ordering such that the packages
    on which the FWaaS service relies are installed before the FWaaS
    service is configured.

    Change-Id: I174cf06b9e8355bfe499b23f390d4078dc670e72
    Closes-Bug: #1287554

Changed in puppet-neutron:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to puppet-neutron (stable/havana)

Reviewed: https://review.openstack.org/78423
Committed: https://git.openstack.org/cgit/stackforge/puppet-neutron/commit/?id=a56f2d6b64e2e8f3cdd20192d99024b8d11b5dcf
Submitter: Jenkins
Branch: stable/havana

commit a56f2d6b64e2e8f3cdd20192d99024b8d11b5dcf
Author: Mark T. Voelker <email address hidden>
Date: Tue Mar 4 02:07:57 2014 -0500

    Fix FWaaS race condition

    The current FWaaS manifest suffers from a race condition: it may
    attempt to configure the FWaaS service before the Neutron packages
    on which it depends has been installed, and before the
    /etc/neutron directory exists. This leads to errors on the first
    catalog run, and the FWaaS service doesn't start. The errors
    go away on the second catalog run since by then the requisite
    directory exists. This patch adds ordering such that the packages
    on which the FWaaS service relies are installed before the FWaaS
    service is configured.

    Change-Id: I174cf06b9e8355bfe499b23f390d4078dc670e72
    Closes-Bug: #1287554
    (cherry picked from commit cfedc81cb60cabdbb31f3809e9d56a54d771458b)

tags: added: in-stable-havana
Changed in openstack-cisco:
status: In Progress → Fix Committed
Mathieu Gagné (mgagne)
Changed in puppet-neutron:
milestone: none → 4.0.0
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.