Managing functional job hooks in the infra config repo is error prone

Bug #1360658 reported by Maru Newby
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
neutron
High
Maru Newby
Icehouse
High
Assaf Muller

Bug Description

The hook scripts that support Neutron's functional gate/check job are currently defined in openstack-infra/config (https://github.com/openstack-infra/config/blob/master/modules/openstack_project/files/jenkins_job_builder/config/neutron-functional.yaml). They are proving difficult to maintain there due to the inability to verify the scripts' functionality before merge. This combined with an overloaded infra core team suggests defining the hook scripts in the neutron tree where the job config can call them (this strategy is already employed by other projects like solum and tripleo).

Revision history for this message
Maru Newby (maru) wrote :

The scripts in the neutron tree will have to be backported to icehouse before the infra job config can be updated.

Changed in neutron:
importance: Undecided → High
assignee: nobody → Maru Newby (maru)
milestone: none → juno-3
tags: added: icehouse-backport-potential
Changed in neutron:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/114717
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=31ca6ea1412ca30c708f52ef844f5a2eedb7a56d
Submitter: Jenkins
Branch: master

commit 31ca6ea1412ca30c708f52ef844f5a2eedb7a56d
Author: Maru Newby <email address hidden>
Date: Fri Aug 22 23:21:07 2014 -0700

    Add hook scripts for the functional infra job

    Infra jobs support hooks to customize behavior. This change adds
    scripts that are intended to be called by the neutron-dsvm-functional
    jobs as hooks. Defining hook behavior in-tree instead of directly in
    the job definition has the advantage of allowing changes in hook
    behavior to be tested before merge.

    The gate_hook.sh script added by this change installs a rootwrap
    filter to allow functional tests to destroy python processes that
    they have spawned. The tests use the python executable provided by
    their tox env, and the default rootwrap configuration does not
    support killing processes launched by non-system python executables.

    This change will have to merge to master and stable/icehouse before
    the supporting infra config change can be merged:

    https://review.openstack.org/#/c/116458/

    Partial-bug: #1360658
    Change-Id: Idcc6195f91fac833c14892f27fb793048ef964b9

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/icehouse)

Fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/116554

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/icehouse)

Reviewed: https://review.openstack.org/116554
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=ea43103bc5b79e39914613cdf847f5f69a3caaca
Submitter: Jenkins
Branch: stable/icehouse

commit ea43103bc5b79e39914613cdf847f5f69a3caaca
Author: Maru Newby <email address hidden>
Date: Fri Aug 22 23:21:07 2014 -0700

    Add hook scripts for the functional infra job

    Infra jobs support hooks to customize behavior. This change adds
    scripts that are intended to be called by the neutron-dsvm-functional
    jobs as hooks. Defining hook behavior in-tree instead of directly in
    the job definition has the advantage of allowing changes in hook
    behavior to be tested before merge.

    The gate_hook.sh script added by this change installs a rootwrap
    filter to allow functional tests to destroy python processes that
    they have spawned. The tests use the python executable provided by
    their tox env, and the default rootwrap configuration does not
    support killing processes launched by non-system python executables.

    This change will have to merge to master and stable/icehouse before
    the supporting infra config change can be merged:

    https://review.openstack.org/#/c/116458/

    Partial-bug: #1360658
    Change-Id: Idcc6195f91fac833c14892f27fb793048ef964b9
    (cherry picked from commit 31ca6ea1412ca30c708f52ef844f5a2eedb7a56d)

tags: added: in-stable-icehouse
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/117280

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to neutron (stable/icehouse)

Related fix proposed to branch: stable/icehouse
Review: https://review.openstack.org/117281

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (master)

Reviewed: https://review.openstack.org/117280
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=1cdeb17c04819a6fa0a39c599af818f06b66f145
Submitter: Jenkins
Branch: master

commit 1cdeb17c04819a6fa0a39c599af818f06b66f145
Author: Maru Newby <email address hidden>
Date: Wed Aug 27 17:33:29 2014 +0200

    Fix func job hook script permission problems

    - The hook script is the wrong place to unstack since
      it needs to run as the stack user with sudo privileges and those
      permissions are removed as part of the devstack-vm-gate.sh script.

    - The functional job gate hook script needs to use sudo to
      install the test-only rootwrap filter.

    Change-Id: Ib2e5ec016a5a414dd9675328650efb2b39b0e3fe
    Related-Bug: #1360658

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron (stable/icehouse)

Reviewed: https://review.openstack.org/117281
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=6f545652f8dd94127539da05a178a1a819f9aa36
Submitter: Jenkins
Branch: stable/icehouse

commit 6f545652f8dd94127539da05a178a1a819f9aa36
Author: Maru Newby <email address hidden>
Date: Wed Aug 27 17:33:29 2014 +0200

    Fix func job hook script permission problems

    - The hook script is the wrong place to unstack since
      it needs to run as the stack user with sudo privileges and those
      permissions are removed as part of the devstack-vm-gate.sh script.

    - The functional job gate hook script needs to use sudo to
      install the test-only rootwrap filter.

    Change-Id: Ib2e5ec016a5a414dd9675328650efb2b39b0e3fe
    Related-Bug: #1360658
    (cherry-picked from commit 2f7932055247df3ac5ec4783e029cfeda0466f71)

Maru Newby (maru)
Changed in neutron:
status: In Progress → Fix Committed
Thierry Carrez (ttx)
Changed in neutron:
status: Fix Committed → Fix Released
Thierry Carrez (ttx)
Changed in neutron:
milestone: juno-3 → 2014.2
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers