Some network tests failing with IndexError

[Artom Lifshitz]

Some network tests are failing with traceback ending in:

File "tempest/scenario/manager.py", line 683, in _get_network_by_name
return net_resources.AttributeDict(net[0])
IndexError: list index out of range

For a concrete example: http://logs.openstack.org/91/151391/1/check/check-grenade-dsvm-ironic-sideways/10f5325/console.html

According to logstash, started appearing today as of 14:30: http://logstash.openstack.org/#eyJzZWFyY2giOiJfZ2V0X25ldHdvcmtfYnlfbmFtZSIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiMTcyODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQyNzc0ODA2MzY1OH0=

[Artom Lifshitz (notartom)]

Looks like it only affect nova stable/juno

[Matthew Treinish (treinish)]

This is a configuration bug in juno devstack/grenade for the sideways baremetal->ironic grenade job. That job needs to make sure it's setting the fixed_network_name config option (in the compute group) in the tempest config to be private. This was previously the incorrect default in tempest and was switched to properly be None. (which it should be) However it looks like in this case this juno job required it. The quick fix would be to either add the iniset in juno devstack or grenade to set that option to the correct value.

[Matt Riedemann (mriedem)]

devstack patch: https://review.openstack.org/#/c/169095/

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/169095
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=84ee55b3ee3630945792fa849d793a6d71d2364d
Submitter: Jenkins
Branch: master

commit 84ee55b3ee3630945792fa849d793a6d71d2364d
Author: Adam Gandelman <email address hidden>
Date: Mon Mar 30 14:25:27 2015 -0700

    Set fixed_network_name in Tempest config

    This previously defatuled to 'private' and aligned with devstack's
    defaults but it has since been updated to 'None'. This sets the config
    value according to devstack's.

    Change-Id: I3f480d5480521a93992bedfe602eb20a4999263d
    Closes-bug: #1438415

[Artom Lifshitz (notartom)]

Looks like there may be something else that's needed, as even with the fix in devstack released, the behaviour continues:

http://logs.openstack.org/91/151391/1/check/check-grenade-dsvm-ironic-sideways/5a1014c/

[Artom Lifshitz (notartom)]

Ignore above, forgot devstack, unlike tempest, *does* have branches, and [1] hasn't merged yet.

[1] https://review.openstack.org/#/c/169096

[Adam Gandelman (gandelman-a)]

As it turns out, setting the config via devstack and having tempest specify networks for instances causes us to tickle a bug in nova when we're running the icehouse->juno partial grenade upgrade test (icehouse nova-compute, juno everything else) with no neutron. See https://launchpad.net/bugs/1438415 This is blocking the fix to merge to stable/juno.

The plan is to force merge that devstack patch to stable/juno, at which point we'll have a working ironic sideways job but failing grenade job, which should be fixed by https://review.openstack.org/#/c/169096

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/169096
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=f610ae9e0e35fd9986f7de94e5a6833fe75dc2df
Submitter: Jeremy Stanley (<email address hidden>)
Branch: stable/juno

commit f610ae9e0e35fd9986f7de94e5a6833fe75dc2df
Author: Adam Gandelman <email address hidden>
Date: Mon Mar 30 14:25:27 2015 -0700

    Set fixed_network_name in Tempest config

    This previously defatuled to 'private' and aligned with devstack's
    defaults but it has since been updated to 'None'. This sets the config
    value according to devstack's.

    Change-Id: I3f480d5480521a93992bedfe602eb20a4999263d
    Closes-bug: #1438415
    (cherry picked from commit 84ee55b3ee3630945792fa849d793a6d71d2364d)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/169100
Committed: https://git.openstack.org/cgit/openstack/tempest/commit/?id=878a5fdeb8f9f1bad27934b9ddbdb0c28055ae1f
Submitter: Jenkins
Branch: master

commit 878a5fdeb8f9f1bad27934b9ddbdb0c28055ae1f
Author: Adam Gandelman <email address hidden>
Date: Mon Mar 30 14:33:36 2015 -0700

    Print an error message and fail if network could not be found by name

    When obtaining a network by name, assert that we actually obtain one
    instead of failing on a key error.

    Related-bug: #1438415

    Change-Id: I0e7f487c5d644f178fc770926333cb1e88749b4a