functional job broken by new os-testr

Bug #1716746 reported by Ihar Hrachyshka
22
This bug affects 5 people
Affects Status Importance Assigned to Milestone
BaGPipe
Undecided
Thomas Morin
networking-bgpvpn
Undecided
Thomas Morin
networking-sfc
Undecided
Bernard Cafarelli
neutron
Critical
Ihar Hrachyshka
tacker
Critical
yong sheng gong

Bug Description

functional job fails with:

2017-09-12 16:09:20.705975 | 2017-09-12 16:09:20.705 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:main:L67: testr_exit_code=0
2017-09-12 16:09:20.707372 | 2017-09-12 16:09:20.706 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:main:L68: set -e
2017-09-12 16:09:20.718005 | 2017-09-12 16:09:20.717 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:main:L71: generate_testr_results
2017-09-12 16:09:20.719619 | 2017-09-12 16:09:20.719 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:generate_testr_results:L12: sudo -H -u stack chmod o+rw .
2017-09-12 16:09:20.720974 | 2017-09-12 16:09:20.720 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:generate_testr_results:L13: sudo -H -u stack chmod o+rw -R .testrepository
2017-09-12 16:09:20.722284 | 2017-09-12 16:09:20.721 | chmod: cannot access '.testrepository': No such file or directory

This is because new os-testr switched to stestr that has a different name for the directory (.stestr).

Changed in neutron:
importance: Undecided → Critical
status: New → Confirmed
tags: added: gate-failure
Changed in neutron:
assignee: nobody → Ihar Hrachyshka (ihar-hrachyshka)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (master)

Fix proposed to branch: master
Review: https://review.openstack.org/503138

Changed in neutron:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (master)

Reviewed: https://review.openstack.org/503138
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3e341a9941607a404ce5219292464c423a3fa5fe
Submitter: Jenkins
Branch: master

commit 3e341a9941607a404ce5219292464c423a3fa5fe
Author: Ihar Hrachyshka <email address hidden>
Date: Tue Sep 12 12:27:28 2017 -0600

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    Change-Id: I82d52bf0ad885bd36d2f0782a7c86ac61df532f2
    Closes-Bug: #1716746

Changed in neutron:
status: In Progress → Fix Released
Changed in networking-sfc:
assignee: nobody → Bernard Cafarelli (bcafarel)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-sfc (master)

Fix proposed to branch: master
Review: https://review.openstack.org/503561

Changed in networking-sfc:
status: New → In Progress
Changed in bgpvpn:
assignee: nobody → Thomas Morin (tmmorin-orange)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to neutron-dynamic-routing (master)

Reviewed: https://review.openstack.org/503765
Committed: https://git.openstack.org/cgit/openstack/neutron-dynamic-routing/commit/?id=d13fdaef4356c75d4cde7dd7934e991400bbd2e6
Submitter: Jenkins
Branch: master

commit d13fdaef4356c75d4cde7dd7934e991400bbd2e6
Author: Brian Haley <email address hidden>
Date: Wed Sep 13 11:42:22 2017 -0600

    Fix unit tests and test configuration

    A recent change in the neutron repository changed the
    import statements for external_process, fixing fallout.

    Fix post gate hook to accommodate for new os-testr.
    New versions now use .stestr instead of previous .testrepository
    directory.

    Related-bug: #1716746
    Change-Id: Iba7579f8de63c6ce87d95167918fc46979702a35

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-bagpipe (master)

Reviewed: https://review.openstack.org/503793
Committed: https://git.openstack.org/cgit/openstack/networking-bagpipe/commit/?id=c6a9bcc34b63c927a37a0596f4bd8c1b0292cff2
Submitter: Jenkins
Branch: master

commit c6a9bcc34b63c927a37a0596f4bd8c1b0292cff2
Author: Thomas Morin <email address hidden>
Date: Wed Sep 13 13:45:56 2017 -0600

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of files
    generated in the directory.

    (shamelessly stolen from I82d52bf0ad885bd36d2f0782a7c86ac61df532f2)
    Co-Authored-By: Ihar Hrachyshka <email address hidden>

    Change-Id: Ieee4bf6e3399b1f496850a23ba2e135b61b03f27
    Closes-Bug: 1716746

Changed in networking-bagpipe:
status: New → Fix Released
Changed in networking-bagpipe:
assignee: nobody → Thomas Morin (tmmorin-orange)
Revision history for this message
Thomas Morin (tmmorin-orange) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-sfc (master)

Reviewed: https://review.openstack.org/503561
Committed: https://git.openstack.org/cgit/openstack/networking-sfc/commit/?id=f9ea384efc86321e90b1bb93f6e1c09ffa868d62
Submitter: Jenkins
Branch: master

commit f9ea384efc86321e90b1bb93f6e1c09ffa868d62
Author: Bernard Cafarelli <email address hidden>
Date: Wed Sep 13 10:28:37 2017 +0200

    Fix unit tests and test configuration

    Fix post gate hook to accommodate for new os-testr
    New versions now use .stestr instead of previous .testrepository
    directory

    Fix ml2 plugin config import
    Commit Ibc5a9ab268578c243ef13f7e0041bacd6c0c410b moved the ml2 plugin
    config file, which breaks unit tests

    Change-Id: Ib7541abb0bee619b87fbcc7fee5d5095255e1ec7
    Closes-Bug: #1716746

Changed in networking-sfc:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-bgpvpn (master)

Reviewed: https://review.openstack.org/503790
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=8f576a2ccb2aa16ff1585b93c4fbb58247d209bd
Submitter: Jenkins
Branch: master

commit 8f576a2ccb2aa16ff1585b93c4fbb58247d209bd
Author: Thomas Morin <email address hidden>
Date: Wed Sep 13 13:28:11 2017 -0600

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    (shamelessly stolen from I82d52bf0ad885bd36d2f0782a7c86ac61df532f2)
    Co-Authored-By: Ihar Hrachyshka <email address hidden>

    This change also wraps up another fix (simultaneous voting test job
    breakage) related to a recent change in the neutron repository of
    the import statements for ml2 config.

    Change-Id: I6bfaca7de99c6ef6834375d4e5b3218a50a60491
    Closes-Bug: 1716746

Changed in bgpvpn:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron-vpnaas (master)

Fix proposed to branch: master
Review: https://review.openstack.org/504294

Changed in tacker:
importance: Undecided → Critical
assignee: nobody → yong sheng gong (gongysh)
milestone: none → queens-1
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tacker (master)

Fix proposed to branch: master
Review: https://review.openstack.org/504791

Changed in tacker:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tacker (master)

Reviewed: https://review.openstack.org/504791
Committed: https://git.openstack.org/cgit/openstack/tacker/commit/?id=e35f07289649b30feee8cee3e712b0de84563226
Submitter: Jenkins
Branch: master

commit e35f07289649b30feee8cee3e712b0de84563226
Author: yong sheng gong <gong.yongsheng@99cloud.net>
Date: Sun Sep 17 23:28:28 2017 +0800

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    Change-Id: I9b7fbba2fe7657bed53f74a22d36ad5116269f69
    Closes-bug: #1716746

Changed in tacker:
status: In Progress → Fix Released
tags: added: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron-fwaas (master)

Fix proposed to branch: master
Review: https://review.openstack.org/513590

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron-vpnaas (master)

Change abandoned by zhichao zhu (<email address hidden>) on branch: master
Review: https://review.openstack.org/504294

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-bagpipe 8.0.0.0b1

This issue was fixed in the openstack/networking-bagpipe 8.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-bgpvpn 8.0.0.0b1

This issue was fixed in the openstack/networking-bgpvpn 8.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-sfc 6.0.0.0b1

This issue was fixed in the openstack/networking-sfc 6.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 12.0.0.0b1

This issue was fixed in the openstack/neutron 12.0.0.0b1 development milestone.

tags: removed: neutron-proactive-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on neutron-fwaas (master)

Change abandoned by XieYingYun (<email address hidden>) on branch: master
Review: https://review.openstack.org/513590

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kuryr-libnetwork 1.0.0

This issue was fixed in the openstack/kuryr-libnetwork 1.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tacker 0.9.0

This issue was fixed in the openstack/tacker 0.9.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to neutron (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/566283

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to neutron (stable/pike)

Reviewed: https://review.openstack.org/566283
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=98bb076905b475bbe1567ff9d8b8771a45216b81
Submitter: Zuul
Branch: stable/pike

commit 98bb076905b475bbe1567ff9d8b8771a45216b81
Author: Ihar Hrachyshka <email address hidden>
Date: Tue Sep 12 12:27:28 2017 -0600

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    Change-Id: I82d52bf0ad885bd36d2f0782a7c86ac61df532f2
    Closes-Bug: #1716746
    (cherry picked from commit 3e341a9941607a404ce5219292464c423a3fa5fe)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/neutron 11.0.5

This issue was fixed in the openstack/neutron 11.0.5 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-bgpvpn (stable/pike)

Reviewed: https://review.openstack.org/595529
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=9c3494339cd5ee22b03f427fd8c2870335e7f900
Submitter: Zuul
Branch: stable/pike

commit 9c3494339cd5ee22b03f427fd8c2870335e7f900
Author: Elod Illes <email address hidden>
Date: Tue Jun 5 14:16:56 2018 +0200

    Fix check, gate and periodic jobs

    This is a combination of 3 commits:

    1. Fix sphinx-docs job for stable branch

    build-openstack-sphinx-docs jobs fail on stable branches due to the new
    pip version 10 introduces some previously Warning as Error: in case of
    calling "pip install" without any package name, the command fails.
    tox_install.sh is called during docs job without any package passed to
    pip.

    Also some tests still needs networking-odl package to be installed, however
    it is removed from the required-projects of networking-bgpvpn, so they
    fails now. To fix this, the easiest solution is to add networking-odl
    to .zuul.yaml on stable/* branch as required project.

    2. Add networking-odl for periodic-stable jobs

    periodic-stable jobs fail: networking-odl is required for jobs on queens
    and older stable branches. From rocky, there is no tox_install.sh any
    more, so this is not needed there.

    3. Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    (shamelessly stolen from I82d52bf0ad885bd36d2f0782a7c86ac61df532f2)
    Co-Authored-By: Ihar Hrachyshka <email address hidden>

    This change also wraps up another fix (simultaneous voting test job
    breakage) related to a recent change in the neutron repository of
    the import statements for ml2 config.

    Closes-Bug: 1716746

    Change-Id: I93aea1800b6403482a7e15116610efeed02c28a1
    (cherry picked from commit ff84e5a105cdc8e70693bcb0a7b3e542d5b4f12d)
    (cherry picked from commit d0c1a6b58267bb1e1346cfe35a61f5b57d9cc672)
    (cherry picked from commit 8f576a2ccb2aa16ff1585b93c4fbb58247d209bd)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-bgpvpn (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/596852

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/608169

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-bgpvpn (stable/ocata)

Change abandoned by Andreas Jaeger (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/608169
Reason: works..

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to networking-bgpvpn (stable/ocata)
Download full text (5.7 KiB)

Reviewed: https://review.openstack.org/596852
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=99ab547770a9662ee63851f9ce13b908376a453a
Submitter: Zuul
Branch: stable/ocata

commit 99ab547770a9662ee63851f9ce13b908376a453a
Author: Elod Illes <email address hidden>
Date: Tue Jun 5 14:16:56 2018 +0200

    Fix check, gate and periodic jobs

    This is a combination of 10 commits that are needed to have successful
    check, gate and periodic stable job runs.

    Note: All the tempest related fixes are due to the fact that we can't
    pin the version of tempest that we are using, and hence need to adjust
    things when tempest evolves.

    1. Fix sphinx-docs job for stable branch

    build-openstack-sphinx-docs jobs fail on stable branches due to the new
    pip version 10 introduces some previously Warning as Error: in case of
    calling "pip install" without any package name, the command fails.
    tox_install.sh is called during docs job without any package passed to
    pip.

    Also some tests still needs networking-odl package to be installed, however
    it is removed from the required-projects of networking-bgpvpn, so they
    fails now. To fix this, the easiest solution is to add networking-odl
    to .zuul.yaml on stable/* branch as required project.

    2. Add networking-odl for periodic-stable jobs

    periodic-stable jobs fail: networking-odl is required for jobs on queens
    and older stable branches. From rocky, there is no tox_install.sh any
    more, so this is not needed there.

    3. (stestr change removed)

    4. use networking-odl from pypi instead of git master

    Two reasons:
    - no point in tracking master, now that development is done
      on odl bgpvpn v2 driver, which does not sit in n8g-bgpvpn tree
    - odl master is sometimes broken [1], which impacts our CI

    [1] http://logs.openstack.org/70/470470/2/check/gate-networking-bgpvpn-dsvm-functional-ubuntu-xenial/febf152/console.html#_2017-06-05_12_56_21_302481

    5. Speed up tox_install.sh

    * When installing git repos into tox venv, there is no need to
      use full clone. Shallow clone like --depth 1 is sufficient.
      It speeds up tox venv creation.
    * Fix import check for networking-bagpipe and networking-odl.
      Python module names use underscores instead of hyphens,
      so the current iport check always returns false.
      As a result, installation of these projects are run multiple times.
    * Allow to specify GIT_BASE for local testing.
      Previously PIP_LOCATION exists, but the tox_install.sh installs
      four repositories so PIP_LOCATION actually cannot be used.

    In my local env, the time to prepare tox env was reduced from 4m52s
    to 2m39s for the first run and from 42s to 19s for the second run
    or later (after applying the first two points above).

    Note(elod.illes): some changes of tox_install.sh didn't seem trivial to
    git so it needed some manual intervention...

    Closes-Bug: 1716746

    6. Change sourcing neutron l2 agent script for devstack

    Source the l2 agent script only in...

Read more...

tags: added: in-stable-ocata
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to networking-sfc (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/639312

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on networking-sfc (stable/pike)

Change abandoned by Manuel Buil (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/639312
Reason: https://review.openstack.org/#/c/639625/

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-bgpvpn 7.0.1

This issue was fixed in the openstack/networking-bgpvpn 7.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/networking-bgpvpn ocata-eol

This issue was fixed in the openstack/networking-bgpvpn ocata-eol release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers