Grenade failures due to failing to attach a volume

[Michael Still]

http://logstash.openstack.org/#eyJzZWFyY2giOiJmaWxlbmFtZTpjb25zb2xlLmh0bWwgQU5EIG1lc3NhZ2U6XCJub3QgYXR0YWNoZWQgdG8gZXgtdm9sLWluc3RcIiIsImZpZWxkcyI6W10sIm9mZnNldCI6MCwidGltZWZyYW1lIjoiYWxsIiwiZ3JhcGhtb2RlIjoiY291bnQiLCJ0aW1lIjp7InVzZXJfaW50ZXJ2YWwiOjB9LCJzdGFtcCI6MTM4NTQyNzE1MDI0Mn0=

This failure isn't super frequent, but it does happen.

console.html says:

2013-11-25 02:46:40.773 | + start_time=1385347600
2013-11-25 02:46:40.774 | + nova volume-attach 20006080-a4a4-460a-ba2c-3e43558354af 80222c16-3a1b-45ff-a7c8-d6cddfc4e324 /dev/vdb
2013-11-25 02:46:42.281 | +----------+--------------------------------------+
2013-11-25 02:46:42.282 | | Property | Value |
2013-11-25 02:46:42.282 | +----------+--------------------------------------+
2013-11-25 02:46:42.282 | | device | /dev/vdb |
2013-11-25 02:46:42.282 | | id | 80222c16-3a1b-45ff-a7c8-d6cddfc4e324 |
2013-11-25 02:46:42.282 | | serverId | 20006080-a4a4-460a-ba2c-3e43558354af |
2013-11-25 02:46:42.282 | | volumeId | 80222c16-3a1b-45ff-a7c8-d6cddfc4e324 |
2013-11-25 02:46:42.282 | +----------+--------------------------------------+
2013-11-25 02:46:42.314 | + timeout 90 sh -c 'while ! cinder list | grep ex-vol-c6536eef | grep in-use; do sleep 1; done'
2013-11-25 02:48:12.319 | + die 176 'Volume ex-vol-c6536eef not attached to ex-vol-inst'
2013-11-25 02:48:12.320 | + local exitcode=0
2013-11-25 02:48:12.320 | + set +o xtrace
2013-11-25 02:48:12.320 | [Call Trace]
2013-11-25 02:48:12.320 | /opt/stack/new/devstack/exercises/volumes.sh:176:die
2013-11-25 02:48:12.359 | [ERROR] /opt/stack/new/devstack/exercises/volumes.sh:176 Volume ex-vol-c6536eef not attached to ex-vol-inst

[John Griffith (john-griffith)]

seems to be a compat issue as it only shows up in Grenade

[John Griffith (john-griffith)]

So the issue is on the Cinder side is captured in the logs here:

http://logs.openstack.org/44/56544/1/check/check-grenade-devstack-vm/0304a83/logs/new/screen-c-api.txt.gz#_2013-11-25_02_46_45_308

Unfortunately I'm at a loss as to the cause of it so I'll keep digging but this will not be the quick fix I was hoping for. Others who have insight please chime in :)

[John Griffith (john-griffith)]

Haven't seen this in any queries lately, suspect it may have been cleaned up by:
Change-Id: I934a6ea6e9eb510513640870750ea37e6f076df9

Will re-open if we see it again.