functional job broken by new os-testr

[Ihar Hrachyshka]

functional job fails with:

2017-09-12 16:09:20.705975 | 2017-09-12 16:09:20.705 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:main:L67: testr_exit_code=0
2017-09-12 16:09:20.707372 | 2017-09-12 16:09:20.706 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:main:L68: set -e
2017-09-12 16:09:20.718005 | 2017-09-12 16:09:20.717 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:main:L71: generate_testr_results
2017-09-12 16:09:20.719619 | 2017-09-12 16:09:20.719 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:generate_testr_results:L12: sudo -H -u stack chmod o+rw .
2017-09-12 16:09:20.720974 | 2017-09-12 16:09:20.720 | + /opt/stack/new/neutron/neutron/tests/contrib/post_test_hook.sh:generate_testr_results:L13: sudo -H -u stack chmod o+rw -R .testrepository
2017-09-12 16:09:20.722284 | 2017-09-12 16:09:20.721 | chmod: cannot access '.testrepository': No such file or directory

This is because new os-testr switched to stestr that has a different name for the directory (.stestr).

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/503138

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/503138
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=3e341a9941607a404ce5219292464c423a3fa5fe
Submitter: Jenkins
Branch: master

commit 3e341a9941607a404ce5219292464c423a3fa5fe
Author: Ihar Hrachyshka <email address hidden>
Date: Tue Sep 12 12:27:28 2017 -0600

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    Change-Id: I82d52bf0ad885bd36d2f0782a7c86ac61df532f2
    Closes-Bug: #1716746

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/503561

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/503765
Committed: https://git.openstack.org/cgit/openstack/neutron-dynamic-routing/commit/?id=d13fdaef4356c75d4cde7dd7934e991400bbd2e6
Submitter: Jenkins
Branch: master

commit d13fdaef4356c75d4cde7dd7934e991400bbd2e6
Author: Brian Haley <email address hidden>
Date: Wed Sep 13 11:42:22 2017 -0600

    Fix unit tests and test configuration

    A recent change in the neutron repository changed the
    import statements for external_process, fixing fallout.

    Fix post gate hook to accommodate for new os-testr.
    New versions now use .stestr instead of previous .testrepository
    directory.

    Related-bug: #1716746
    Change-Id: Iba7579f8de63c6ce87d95167918fc46979702a35

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/503793
Committed: https://git.openstack.org/cgit/openstack/networking-bagpipe/commit/?id=c6a9bcc34b63c927a37a0596f4bd8c1b0292cff2
Submitter: Jenkins
Branch: master

commit c6a9bcc34b63c927a37a0596f4bd8c1b0292cff2
Author: Thomas Morin <email address hidden>
Date: Wed Sep 13 13:45:56 2017 -0600

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of files
    generated in the directory.

    (shamelessly stolen from I82d52bf0ad885bd36d2f0782a7c86ac61df532f2)
    Co-Authored-By: Ihar Hrachyshka <email address hidden>

    Change-Id: Ieee4bf6e3399b1f496850a23ba2e135b61b03f27
    Closes-Bug: 1716746

[Thomas Morin (tmmorin-orange)]

neutron-lib is also bitten by this bug:

http://logs.openstack.org/54/503954/2/check/gate-neutron-lib-python27-ubuntu-xenial/20632a2/console.html#_2017-09-14_12_48_34_535487

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/503561
Committed: https://git.openstack.org/cgit/openstack/networking-sfc/commit/?id=f9ea384efc86321e90b1bb93f6e1c09ffa868d62
Submitter: Jenkins
Branch: master

commit f9ea384efc86321e90b1bb93f6e1c09ffa868d62
Author: Bernard Cafarelli <email address hidden>
Date: Wed Sep 13 10:28:37 2017 +0200

    Fix unit tests and test configuration

    Fix post gate hook to accommodate for new os-testr
    New versions now use .stestr instead of previous .testrepository
    directory

    Fix ml2 plugin config import
    Commit Ibc5a9ab268578c243ef13f7e0041bacd6c0c410b moved the ml2 plugin
    config file, which breaks unit tests

    Change-Id: Ib7541abb0bee619b87fbcc7fee5d5095255e1ec7
    Closes-Bug: #1716746

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/503790
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=8f576a2ccb2aa16ff1585b93c4fbb58247d209bd
Submitter: Jenkins
Branch: master

commit 8f576a2ccb2aa16ff1585b93c4fbb58247d209bd
Author: Thomas Morin <email address hidden>
Date: Wed Sep 13 13:28:11 2017 -0600

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    (shamelessly stolen from I82d52bf0ad885bd36d2f0782a7c86ac61df532f2)
    Co-Authored-By: Ihar Hrachyshka <email address hidden>

    This change also wraps up another fix (simultaneous voting test job
    breakage) related to a recent change in the neutron repository of
    the import statements for ml2 config.

    Change-Id: I6bfaca7de99c6ef6834375d4e5b3218a50a60491
    Closes-Bug: 1716746

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/504294

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/504791

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/504791
Committed: https://git.openstack.org/cgit/openstack/tacker/commit/?id=e35f07289649b30feee8cee3e712b0de84563226
Submitter: Jenkins
Branch: master

commit e35f07289649b30feee8cee3e712b0de84563226
Author: yong sheng gong <[e-mail]>
Date: Sun Sep 17 23:28:28 2017 +0800

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    Change-Id: I9b7fbba2fe7657bed53f74a22d36ad5116269f69
    Closes-bug: #1716746

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/513590

[OpenStack Infra (hudson-openstack)]

Change abandoned by zhichao zhu (<email address hidden>) on branch: master
Review: https://review.openstack.org/504294

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-bagpipe 8.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-bgpvpn 8.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-sfc 6.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 12.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

Change abandoned by XieYingYun (<email address hidden>) on branch: master
Review: https://review.openstack.org/513590

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/kuryr-libnetwork 1.0.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tacker 0.9.0 release.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/566283

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/566283
Committed: https://git.openstack.org/cgit/openstack/neutron/commit/?id=98bb076905b475bbe1567ff9d8b8771a45216b81
Submitter: Zuul
Branch: stable/pike

commit 98bb076905b475bbe1567ff9d8b8771a45216b81
Author: Ihar Hrachyshka <email address hidden>
Date: Tue Sep 12 12:27:28 2017 -0600

    Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    Change-Id: I82d52bf0ad885bd36d2f0782a7c86ac61df532f2
    Closes-Bug: #1716746
    (cherry picked from commit 3e341a9941607a404ce5219292464c423a3fa5fe)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/neutron 11.0.5 release.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/595529
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=9c3494339cd5ee22b03f427fd8c2870335e7f900
Submitter: Zuul
Branch: stable/pike

commit 9c3494339cd5ee22b03f427fd8c2870335e7f900
Author: Elod Illes <email address hidden>
Date: Tue Jun 5 14:16:56 2018 +0200

    Fix check, gate and periodic jobs

    This is a combination of 3 commits:

    1. Fix sphinx-docs job for stable branch

    build-openstack-sphinx-docs jobs fail on stable branches due to the new
    pip version 10 introduces some previously Warning as Error: in case of
    calling "pip install" without any package name, the command fails.
    tox_install.sh is called during docs job without any package passed to
    pip.

    Also some tests still needs networking-odl package to be installed, however
    it is removed from the required-projects of networking-bgpvpn, so they
    fails now. To fix this, the easiest solution is to add networking-odl
    to .zuul.yaml on stable/* branch as required project.

    2. Add networking-odl for periodic-stable jobs

    periodic-stable jobs fail: networking-odl is required for jobs on queens
    and older stable branches. From rocky, there is no tox_install.sh any
    more, so this is not needed there.

    3. Fix post gate hook to accommodate for new os-testr

    New os-testr uses stestr under the hood, which creates .stestr but not
    .testrepository directory in the current dir. Other than that, it
    doesn't seem like there is any difference in the format or names of
    files generated in the directory.

    (shamelessly stolen from I82d52bf0ad885bd36d2f0782a7c86ac61df532f2)
    Co-Authored-By: Ihar Hrachyshka <email address hidden>

    This change also wraps up another fix (simultaneous voting test job
    breakage) related to a recent change in the neutron repository of
    the import statements for ml2 config.

    Closes-Bug: 1716746

    Change-Id: I93aea1800b6403482a7e15116610efeed02c28a1
    (cherry picked from commit ff84e5a105cdc8e70693bcb0a7b3e542d5b4f12d)
    (cherry picked from commit d0c1a6b58267bb1e1346cfe35a61f5b57d9cc672)
    (cherry picked from commit 8f576a2ccb2aa16ff1585b93c4fbb58247d209bd)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/596852

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/608169

[OpenStack Infra (hudson-openstack)]

Change abandoned by Andreas Jaeger (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/608169
Reason: works..

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/596852
Committed: https://git.openstack.org/cgit/openstack/networking-bgpvpn/commit/?id=99ab547770a9662ee63851f9ce13b908376a453a
Submitter: Zuul
Branch: stable/ocata

commit 99ab547770a9662ee63851f9ce13b908376a453a
Author: Elod Illes <email address hidden>
Date: Tue Jun 5 14:16:56 2018 +0200

    Fix check, gate and periodic jobs

    This is a combination of 10 commits that are needed to have successful
    check, gate and periodic stable job runs.

    Note: All the tempest related fixes are due to the fact that we can't
    pin the version of tempest that we are using, and hence need to adjust
    things when tempest evolves.

    1. Fix sphinx-docs job for stable branch

    build-openstack-sphinx-docs jobs fail on stable branches due to the new
    pip version 10 introduces some previously Warning as Error: in case of
    calling "pip install" without any package name, the command fails.
    tox_install.sh is called during docs job without any package passed to
    pip.

    Also some tests still needs networking-odl package to be installed, however
    it is removed from the required-projects of networking-bgpvpn, so they
    fails now. To fix this, the easiest solution is to add networking-odl
    to .zuul.yaml on stable/* branch as required project.

    2. Add networking-odl for periodic-stable jobs

    periodic-stable jobs fail: networking-odl is required for jobs on queens
    and older stable branches. From rocky, there is no tox_install.sh any
    more, so this is not needed there.

    3. (stestr change removed)

    4. use networking-odl from pypi instead of git master

    Two reasons:
    - no point in tracking master, now that development is done
      on odl bgpvpn v2 driver, which does not sit in n8g-bgpvpn tree
    - odl master is sometimes broken [1], which impacts our CI

    [1] http://logs.openstack.org/70/470470/2/check/gate-networking-bgpvpn-dsvm-functional-ubuntu-xenial/febf152/console.html#_2017-06-05_12_56_21_302481

    5. Speed up tox_install.sh

    * When installing git repos into tox venv, there is no need to
      use full clone. Shallow clone like --depth 1 is sufficient.
      It speeds up tox venv creation.
    * Fix import check for networking-bagpipe and networking-odl.
      Python module names use underscores instead of hyphens,
      so the current iport check always returns false.
      As a result, installation of these projects are run multiple times.
    * Allow to specify GIT_BASE for local testing.
      Previously PIP_LOCATION exists, but the tox_install.sh installs
      four repositories so PIP_LOCATION actually cannot be used.

    In my local env, the time to prepare tox env was reduced from 4m52s
    to 2m39s for the first run and from 42s to 19s for the second run
    or later (after applying the first two points above).

    Note(elod.illes): some changes of tox_install.sh didn't seem trivial to
    git so it needed some manual intervention...

    Closes-Bug: 1716746

    6. Change sourcing neutron l2 agent script for devstack

    Source the l2 agent script only in...

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/639312

[OpenStack Infra (hudson-openstack)]

Change abandoned by Manuel Buil (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/639312
Reason: https://review.openstack.org/#/c/639625/

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-bgpvpn 7.0.1 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/networking-bgpvpn ocata-eol release.