Volume attachment silently fails in current devstack

[Tres Henry]

With latest devstack when trying to attach a volume to an instance in horizon the volume goes into the attaching state but after ~ 2 minutes it return to the state it was in prior to attempting to attach. Nothing interesting that I could find in nova api log or cinder log.

This is in /var/log/syslog:

Aug 21 18:09:04 EIGHT kernel: [ 373.467923] connection1:0: detected conn error (1019)
Aug 21 18:09:04 EIGHT iscsid: Kernel reported iSCSI connection 1:0 error (1019) state (2)
Aug 21 18:09:21 EIGHT kernel: [ 391.235575] connection1:0: detected conn error (1019)
Aug 21 18:09:22 EIGHT iscsid: Kernel reported iSCSI connection 1:0 error (1019) state (2)
Aug 21 18:09:39 EIGHT kernel: [ 408.498545] connection1:0: detected conn error (1019)
Aug 21 18:09:39 EIGHT iscsid: Kernel reported iSCSI connection 1:0 error (1019) state (2)
Aug 21 18:09:56 EIGHT kernel: [ 425.762905] connection1:0: detected conn error (1019)
Aug 21 18:09:57 EIGHT iscsid: Kernel reported iSCSI connection 1:0 error (1019) state (2)
Aug 21 18:10:13 EIGHT kernel: [ 443.025484] connection1:0: detected conn error (1019)
Aug 21 18:10:14 EIGHT iscsid: Kernel reported iSCSI connection 1:0 error (1019) state (2)
Aug 21 18:10:30 EIGHT kernel: [ 460.287754] connection1:0: detected conn error (1019)
Aug 21 18:10:31 EIGHT iscsid: Kernel reported iSCSI connection 1:0 error (1019) state (2)
Aug 21 18:10:48 EIGHT kernel: [ 477.550922] connection1:0: detected conn error (1019)
Aug 21 18:10:48 EIGHT iscsid: Kernel reported iSCSI connection 1:0 error (1019) state (2)
Aug 21 18:15:53 EIGHT kernel: [ 783.117893] scsi6 : iSCSI Initiator over TCP/IP
Aug 21 18:15:53 EIGHT kernel: [ 783.123425] connection2:0: detected conn error (1019)

[John Griffith (john-griffith)]

I think the persistent targets change has introduced some problems when upgrading. In this particular case I'm wondering if you had a devstack install previously on this machine prior to pulling the latest version of devstack and re-running?

If this was not a fresh OS and devstack install, let me know a bit more about the system and setup.

If this was in fact a fresh install of the OS and of devstack, could you please try simply running the devstack/exercises/volumes.sh test and see if that test passes?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/11899

[John Griffith (john-griffith)]

Oh... well that's funny. I found an issue in the parameters for tgt-admin command and went to fix it in Nova and it was already there.

Turns out this was detected/fixed on Nova already: https://bugs.launchpad.net/nova/+bug/1033670

Anyway, when you get a chance let me know if this in fact resolves the issue for you. My reproduced case had slightly different behavior than your description so I'll need your feedback to be 100% sure.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/11899
Committed: http://github.com/openstack/cinder/commit/3034247a4ec7c9c6e433cad6d13183b001fe2e6d
Submitter: Jenkins
Branch: master

commit 3034247a4ec7c9c6e433cad6d13183b001fe2e6d
Author: John Griffith <email address hidden>
Date: Thu Aug 23 13:23:04 2012 -0600

    Fix incorrect tgt-admin call in create_iscsi_target

      * Fixes bug 1039826
      * Missed one of the execute cmds that was using 'val %s' as args

    Change-Id: I5e6af737ada1c2bcf7a495710e2b10c8c74bb534

[Gabriel Hurley (gabriel-hurley)]

Just tried this on a fresh devstack and the underlying problem seems to be fixed. Closing as invalid for Horizon since no fix was required for Horizon. I believe Cinder can mark this as Fixed.