gate-devstack-dsvm-cells fails attaching volume

[Matt Riedemann]

http://logs.openstack.org/07/154607/3/gate/gate-devstack-dsvm-cells/302ef5f/console.html#_2015-02-19_19_20_49_062

2015-02-19 19:20:47.642 | + nova volume-attach d16b33f6-7cf7-4e77-b149-84ba3eedf0e6 e2496f30-dac1-484b-b25c-c925a0766fa6 /dev/vdb
2015-02-19 19:20:49.034 | ERROR (NotFound): Not found (HTTP 404) (Request-ID: req-8ae38d4c-c754-4c45-92bd-0719bc2ecdd5)
2015-02-19 19:20:49.062 | + die 174 'Failure attaching volume ex-vol-f39662b7 to ex-vol-inst'
2015-02-19 19:20:49.062 | + local exitcode=1
2015-02-19 19:20:49.062 | + set +o xtrace
2015-02-19 19:20:49.062 | [Call Trace]
2015-02-19 19:20:49.063 | /opt/stack/new/devstack/exercises/volumes.sh:174:die
2015-02-19 19:20:49.065 | [ERROR] /opt/stack/new/devstack/exercises/volumes.sh:174 Failure attaching volume ex-vol-f39662b7 to ex-vol-inst

Just started today:

http://logstash.openstack.org/#eyJzZWFyY2giOiJtZXNzYWdlOlwiZXhlcmNpc2VzL3ZvbHVtZXMuc2hcIiBBTkQgbWVzc2FnZTpcIkZhaWx1cmUgYXR0YWNoaW5nIHZvbHVtZVwiIEFORCBtZXNzYWdlOlwidG8gZXgtdm9sLWluc3RcIiBBTkQgdGFnczpcImNvbnNvbGVcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiNjA0ODAwIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTQyNDM4MDc1MDI3Nn0=

35 hits in 24 hours, check and gate, all failures.

[Matt Riedemann (mriedem)]

python-novaclient 2.21 was released today so maybe related somehow...

[Joe Gordon (jogo)]

this may be a novaclient bug. Only seeing this failure with the new novaclient

[Matt Riedemann (mriedem)]

novaclient patch: https://review.openstack.org/#/c/157591/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/157614

[Matt Riedemann (mriedem)]

Here is the project-config change to get the devstack exercises job running again on python-novaclient:

https://review.openstack.org/#/c/157661/

Once that's working we can work on reverting https://review.openstack.org/#/c/157591/ and get that working for a 2.22 release.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/157959

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/157974

[Joe Gordon (jogo)]

Sean found the novaclient patch that made volume commands use cinder:

https://github.com/openstack/python-novaclient/commit/03f54c57e13f27324f0ac5aa556bf4ca94e3ed0e

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/157591
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=ac6636a54d72ba76d0adca76e07d1d26d9ea35c3
Submitter: Jenkins
Branch: master

commit ac6636a54d72ba76d0adca76e07d1d26d9ea35c3
Author: Andrew Laski <email address hidden>
Date: Thu Feb 19 18:24:51 2015 -0500

    Revert "Overhaul bash-completion to support non-UUID based IDs"

    This reverts commit 4c8cefb98a425382204df2f38f24e6b5b71520dd.

    The cache completion was causing a bug where 'nova volume-attach' would
    then try to query Nova for volume information using a URL that was not
    valid. This caused it to appear that the attach had failed.

    Additionally the idea of making extra API queries for a bash completion
    cache should be thought through more.

    Conflicts:

     novaclient/client.py
     novaclient/shell.py
     novaclient/v2/client.py
     novaclient/v3/client.py

    Closes-Bug: #1423695
    Change-Id: I6676a11f9b5318a1cda2d03a5d4550bda549d5c2

[OpenStack Infra (hudson-openstack)]

Change abandoned by Dan Smith (<email address hidden>) on branch: master
Review: https://review.openstack.org/157614

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/157959
Committed: https://git.openstack.org/cgit/openstack/python-novaclient/commit/?id=be41ae238d05a2c3ade3822d869d2d199444c52a
Submitter: Jenkins
Branch: master

commit be41ae238d05a2c3ade3822d869d2d199444c52a
Author: Sean Dague <email address hidden>
Date: Fri Feb 20 16:23:24 2015 -0500

    add functional test for nova volume-attach bug

    This is a functional test that boots a server via the cli, creates a
    volume, and tries to attach it via the cli (which causes a failure due
    to completion cache code). Note: the failure actually happens *after*
    the attach command is dispatched, so the volume attach will still
    work, the user is presented an error though.

    Many TODOs remain for future patches. The test also tries to document
    what was learned about the CLI redirection to cinder API, which was
    introduced when Cinder was split out, but was tribal knowledge that
    was lost in the mists of time.

    Related-Bug: #1423695

    Change-Id: Iaf474298be135843bff0114cf211bee19762f3ad

[Matt Riedemann (mriedem)]

Released in 2.22.0.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Joe Gordon (<email address hidden>) on branch: master
Review: https://review.openstack.org/157974
Reason: This review is > 4 weeks without comment, and failed Jenkins the last time it was checked. We are abandoning this for now. Feel free to reactivate the review by pressing the restore button and leaving a 'recheck' comment to get fresh test results.