grenade-dsvm-neutron Failure in upgrade-swift

[Joe Gordon]

*-grenade-dsvm-neutron jobs are failing in upgrade-swift

http://logs.openstack.org/00/70900/14/check/check-grenade-dsvm-neutron/5a0cfa8/console.html

logstash query:

filename:"console.html" AND message:"mount: wrong fs type, bad option, bad superblock on /dev/loop1," AND build_name:*-grenade-dsvm-neutron

+ sudo chown -R stack:1006 /opt/stack/data/swift
+ sudo mkdir -p /var/cache/swift
+ sudo chown stack /var/cache/swift
+ rm -f /var/cache/swift/cacert.pem /var/cache/swift/revoked.pem /var/cache/swift/signing_cert.pem
+ egrep -q /opt/stack/data/swift/drives/sdb1 /proc/mounts
+ sudo mount -t xfs -o loop,noatime,nodiratime,nobarrier,logbufs=8 /opt/stack/data/swift/drives/images/swift.img /opt/stack/data/swift/drives/sdb1
mount: wrong fs type, bad option, bad superblock on /dev/loop1,
       missing codepage or helper program, or other error
       In some cases useful info is found in syslog - try
       dmesg | tail or so

+ die 257 'Failure in upgrade-swift'
+ local exitcode=32
+ set +o xtrace
[Call Trace]
./grenade.sh:257:die
[ERROR] ./grenade.sh:257 Failure in upgrade-swift

[Sean Dague (sdague)]

This is a 100% fail on the neutron grenade job, and is basically the only reason that neutron grenade job doesn't function as far as I can tell.

[Jakub Libosvar (libosvar)]

There is running process which seems like to be kernel thread [neutron-ns-meta] . I have no idea where does it come from. Based on the name a hot candidate is neutron-ns-metadata-proxy. I'm not able to reproduce it on my own environment.

I added to stop_base in grenade removal of loop device in case it's still present after unmounting swift data successfully with some additional debug information that should provide more details on hanging task.