test_server_basic_ops rename in tempest is breaking cells job

Bug #1562323 reported by Matt Riedemann
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Critical
Matt Riedemann
Liberty
Fix Released
Critical
Matt Riedemann
Mitaka
Fix Released
Critical
Matt Riedemann
tempest
Won't Fix
Undecided
Matt Riedemann

Bug Description

Nova has a blacklist on tempest tests that won't work in the cells v1 job, including:

https://github.com/openstack/nova/blob/master/devstack/tempest-dsvm-cells-rc#L77

With change https://review.openstack.org/#/c/296842/ there was a blacklisted test that was renamed so it's no longer skipped and the cells job is now failing on all nova changes.

We can either revert the tempest change or update the blacklist in nova, but since tempest is branchless we'd have to backport that nova change to stable/mitaka, stable/liberty and stable/kilo.

Matt Riedemann (mriedem)
tags: added: cells security-groups
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tempest (master)

Fix proposed to branch: master
Review: https://review.openstack.org/297951

Changed in tempest:
assignee: nobody → Matt Riedemann (mriedem)
status: New → In Progress
Revision history for this message
Matt Riedemann (mriedem) wrote :
Changed in tempest:
importance: Undecided → Critical
Revision history for this message
Matthew Treinish (treinish) wrote :

This is really a nova bug, and not a tempest bug. Tempest exposes the idempotent ids to enable tracking tests across renames. I don't feel that the fact nova's cell job (which should be using config flags instead of a giant regex anyway) isn't using this to identify tests in its blacklist is justification for a revert. There are multiple options here on the nova side to do this correctly, why does the fact this 1 job doesn't use them warrant a revert?

Changed in tempest:
importance: Critical → Undecided
status: In Progress → Opinion
Revision history for this message
Matt Riedemann (mriedem) wrote :

Presently there is no way to configure tempest to make security group usage optional, there has been a change up for a long time to do that though:

https://review.openstack.org/#/c/225199/

And there has been a change in devstack and nova based on that so we can remove a lot of the regex blacklist in nova. Until that happens though, or nova gets rid of the cells v1 job (not going to happen until cells v2 is up and working), we're going to need to maintain the blacklist regex in tree.

I'm requesting a revert because (1) it's not a majorly important change in tempest, it's just a trivial rename of a test case - that could be done again later with a dependency on the nova changes that use the new name and (2) right now all branches for nova are broken and we don't have control over the stable/mitaka +2 since we're building release candidates, the release team owns those.

Revision history for this message
Matt Riedemann (mriedem) wrote :

FWIW, I agree that nova needs to be using the test uuids for filtering, if possible, I'm just not sure if that's possible with the os-testr --regex that's used in devstack-gate with $DEVSTACK_GATE_TEMPEST_REGEX.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/298065

Changed in nova:
assignee: nobody → Matt Riedemann (mriedem)
status: New → In Progress
Matt Riedemann (mriedem)
Changed in nova:
importance: Undecided → Critical
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tempest (master)

Change abandoned by Matt Riedemann (<email address hidden>) on branch: master
Review: https://review.openstack.org/297951
Reason: We'll just fix nova.

Matt Riedemann (mriedem)
Changed in tempest:
status: Opinion → Won't Fix
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/mitaka)

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/298068

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/298065
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=e86b7fbb99de0c4cff6a9569f748bae1bde693a6
Submitter: Jenkins
Branch: master

commit e86b7fbb99de0c4cff6a9569f748bae1bde693a6
Author: Matt Riedemann <email address hidden>
Date: Sun Mar 27 19:31:32 2016 -0400

    Update cells blacklist regex for test_server_basic_ops

    Tempest change 9bee3b92f1559cb604c8bd74dcca57805a85a97a
    renamed a test in our blacklist so update the filter to
    handle the old and new name.

    The Tempest team is hesitant to revert the change so we
    should handle it ourselves and eventually move to using
    test uuids for our blacklist, but there might need to
    be work in devstack-gate for that first.

    Change-Id: Ibab3958044c21568d7fbbe0a298bb40bbbc20df3
    Closes-Bug: #1562323

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix proposed to nova (master)

Related fix proposed to branch: master
Review: https://review.openstack.org/298437

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/mitaka)

Reviewed: https://review.openstack.org/298068
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=3e9819dab8249ec9993b0b9874e80a78f2ed1754
Submitter: Jenkins
Branch: stable/mitaka

commit 3e9819dab8249ec9993b0b9874e80a78f2ed1754
Author: Matt Riedemann <email address hidden>
Date: Sun Mar 27 19:31:32 2016 -0400

    Update cells blacklist regex for test_server_basic_ops

    Tempest change 9bee3b92f1559cb604c8bd74dcca57805a85a97a
    renamed a test in our blacklist so update the filter to
    handle the old and new name.

    The Tempest team is hesitant to revert the change so we
    should handle it ourselves and eventually move to using
    test uuids for our blacklist, but there might need to
    be work in devstack-gate for that first.

    Change-Id: Ibab3958044c21568d7fbbe0a298bb40bbbc20df3
    Closes-Bug: #1562323
    (cherry picked from commit e86b7fbb99de0c4cff6a9569f748bae1bde693a6)

Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/nova 13.0.0.0rc3

This issue was fixed in the openstack/nova 13.0.0.0rc3 release candidate.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (stable/liberty)

Fix proposed to branch: stable/liberty
Review: https://review.openstack.org/300284

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/liberty)

Reviewed: https://review.openstack.org/300284
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b6e1332705242a3b94140a6dc9ef9bf1e296a5c7
Submitter: Jenkins
Branch: stable/liberty

commit b6e1332705242a3b94140a6dc9ef9bf1e296a5c7
Author: Matt Riedemann <email address hidden>
Date: Sun Mar 27 19:31:32 2016 -0400

    Update cells blacklist regex for test_server_basic_ops

    Tempest change 9bee3b92f1559cb604c8bd74dcca57805a85a97a
    renamed a test in our blacklist so update the filter to
    handle the old and new name.

    The Tempest team is hesitant to revert the change so we
    should handle it ourselves and eventually move to using
    test uuids for our blacklist, but there might need to
    be work in devstack-gate for that first.

    Conflicts:
            devstack/tempest-dsvm-cells-rc

    NOTE(mriedem): The conflict is due to the regex using
    wildcards still in liberty, that was changed in mitaka.

    Change-Id: Ibab3958044c21568d7fbbe0a298bb40bbbc20df3
    Closes-Bug: #1562323
    (cherry picked from commit e86b7fbb99de0c4cff6a9569f748bae1bde693a6)
    (cherry picked from commit 3e9819dab8249ec9993b0b9874e80a78f2ed1754)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Related fix merged to nova (master)

Reviewed: https://review.openstack.org/298437
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=b113cb13e95b4d9305f093342ad4d34378aee8bb
Submitter: Jenkins
Branch: master

commit b113cb13e95b4d9305f093342ad4d34378aee8bb
Author: Chuck Carmack <email address hidden>
Date: Mon Mar 28 19:16:35 2016 +0000

    Change the nova tempest blacklist to use to idempotent ids

    Currently, nova blacklists tests by name. If a name changes,
    the regex will not match, and the test will be attempted.
    If the test idempotent ids are used instead, the test
    will always be skipped.

    Change-Id: Iaf189c42c342b4c2d7c77555980ed49914210bf6
    Related-bug: 1562323
    Closes-bug: 1562945

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/302578

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)
Download full text (11.5 KiB)

Reviewed: https://review.openstack.org/302578
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=a8ebbebd4ee0c3bb1452ea32f92e1588a6b35067
Submitter: Jenkins
Branch: master

commit 7105f888ee1f52d2a462fc0ece3130dc0d3d49f5
Author: OpenStack Proposal Bot <email address hidden>
Date: Thu Mar 31 06:28:06 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: Ibe5d4d38834fbcb99c0332d3375659a21d94154e

commit 5de98cb2de2eca3d061488c55f96e6f7c9bc56a8
Author: OpenStack Proposal Bot <email address hidden>
Date: Wed Mar 30 06:41:25 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: Ia46d661560b1141c1c1522c9477c510d28a0d0e7

commit a9d55427b6e8d2472088e3d40a8a5151ce408283
Author: Moshe Levi <email address hidden>
Date: Wed Mar 23 10:59:04 2016 +0200

    Fix detach SR-IOV when using LibvirtConfigGuestHostdevPCI

    This patch fixes an issue which was introduced by this
    change If3edc1965c01a077eb61984a442e0d778d870d75.
    Usually the vif config is of type LibvirtConfigGuestInterface,
    but some vif use LibvirtConfigGuestHostdevPCI config
    (e.g. the ib_hostdev). The difference is that
    LibvirtConfigGuestInterface keeps the pci address in source_dev
    while LibvirtConfigGuestHostdevPCI has domain, bus, slot and
    function, instead of relying on the vif config type we can take the
    pci address for the neutron port.

    Closes-Bug: #1560860

    Change-Id: I62a7ff16f1c9c5da923451520fbeeabb5cc0c5c6
    (cherry picked from commit f15d9a9693b19393fcde84cf4bc6f044d39ffdca)

commit 5b6ee702df7ad901f68bec2ed8d43b66aa6d98c1
Author: OpenStack Proposal Bot <email address hidden>
Date: Tue Mar 29 06:37:30 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: Iad0e42a18bd3a7dcf216b4df17b9893e13382efe

commit 29042e06f7e570bd13607b62b997a6ae21db80c5
Author: OpenStack Proposal Bot <email address hidden>
Date: Mon Mar 28 06:34:19 2016 +0000

    Imported Translations from Zanata

    For more information about this automatic import see:
    https://wiki.openstack.org/wiki/Translations/Infrastructure

    Change-Id: If159133a2e32c6ef53ba104751a3eb054a95b733

commit 3e9819dab8249ec9993b0b9874e80a78f2ed1754
Author: Matt Riedemann <email address hidden>
Date: Sun Mar 27 19:31:32 2016 -0400

    Update cells blacklist regex for test_server_basic_ops

    Tempest change 9bee3b92f1559cb604c8bd74dcca57805a85a97a
    renamed a test in our blacklist so update the filter to
    handle the old and new name.

    The Tempest team is hesitant to revert the change so we
    should handle it ourselves and eventually move to using
    test uuids for our blacklist, but there might need to
    be work in devstack-gate for that fi...

Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote : Fix included in openstack/nova 12.0.3

This issue was fixed in the openstack/nova 12.0.3 release.

Revision history for this message
Davanum Srinivas (DIMS) (dims-v) wrote : Fix included in openstack/nova 14.0.0.0b1

This issue was fixed in the openstack/nova 14.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.