Passing an incorrect zone name to euca-create-volume results in a volume stuck in the "creating" state

Bug #1016111 reported by Andrew Glen-Young
22
This bug affects 4 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Invalid
High
Chuck Short
nova (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

What happens:

I have two regions and one availability zone (AZ) for my Essex cluster. When creating a volume I accidentally used the region name instead of the AZ name. This resulted in a volume being stuck in the "creating" state which I cannot delete.

What I expect:

I expect two things:
 1. The volume creation should fail (it currently does)
 2. The volume to go into an "error" state (it does not currently do this)

Reasoning:

Having the volume go into an error state informs the user that there was a problem and does not leave them wondering if they should wait longer in case the command hasn't completed.

Steps to reproduce:

 1. user@laptop:~$ euca-create-volume -s 2 # command requires a zone
    These required options are missing: zone

 2. user@laptop:~$ euca-create-volume -s 2 -z lcy-2 # accidentally pass a non-existant zone name
    VOLUME vol-00000001 2 lcy-2 creating 2012-06-21T13:50:10.074Z

 3. user@laptop:~$ euca-describe-volumes vol-00000001
    VOLUME vol-00000001 2 lcy-2 creating 2012-06-21T13:50:10.000Z

 4. The nova-scheduler log file shows the following.
    2012-06-21 13:58:41 WARNING nova.scheduler.manager [req-63af9b42-502d-4ad4-9ae1-e981d053f9fc a9d62e6e73294368b79d21ea2a2e2d86 df473f958e4f47949282696966e58f49] Failed to schedule_create_volume: No valid host was found. Is the appropriate service running?
    2012-06-21 13:58:41 ERROR nova.rpc.amqp [req-63af9b42-502d-4ad4-9ae1-e981d053f9fc a9d62e6e73294368b79d21ea2a2e2d86 df473f958e4f47949282696966e58f49] Exception during message handling
    2012-06-21 13:58:41 TRACE nova.rpc.amqp Traceback (most recent call last):
    2012-06-21 13:58:41 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/rpc/amqp.py", line 252, in _process_data
    2012-06-21 13:58:41 TRACE nova.rpc.amqp rval = node_func(context=ctxt, **node_args)
    2012-06-21 13:58:41 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line 97, in _schedule
    2012-06-21 13:58:41 TRACE nova.rpc.amqp context, ex, *args, **kwargs)
    2012-06-21 13:58:41 TRACE nova.rpc.amqp File "/usr/lib/python2.7/contextlib.py", line 24, in __exit__
    2012-06-21 13:58:41 TRACE nova.rpc.amqp self.gen.next()
    2012-06-21 13:58:41 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/scheduler/manager.py", line 92, in _schedule
    2012-06-21 13:58:41 TRACE nova.rpc.amqp return driver_method(*args, **kwargs)
    2012-06-21 13:58:41 TRACE nova.rpc.amqp File "/usr/lib/python2.7/dist-packages/nova/scheduler/simple.py", line 144, in schedule_create_volume
    2012-06-21 13:58:41 TRACE nova.rpc.amqp raise exception.NoValidHost(reason=msg)
    2012-06-21 13:58:41 TRACE nova.rpc.amqp NoValidHost: No valid host was found. Is the appropriate service running?

Successful volume creation:

 1. user@laptop:~$ euca-describe-availability-zones # find the AZ
    AVAILABILITYZONE nova available

 2. user@laptop:~$ euca-create-volume -s 2 -z nova # create the volume passing in the correct AZ name
    VOLUME vol-00000003 2 nova creating 2012-06-21T14:02:07.842Z

 3. agy@agy-laptop:~$ euca-describe-volumes
    VOLUME vol-00000001 2 lcy-2 creating 2012-06-21T13:50:10.000Z
    VOLUME vol-00000002 2 lcy-2 creating 2012-06-21T13:58:41.000Z
    VOLUME vol-00000003 2 nova available 2012-06-21T14:02:07.000Z

    Note the two volumes that have failed and will not complete and the correctly created volume.

Ideally, I would like the API server to reject the command and return an informative error message to the user before attempting to create the volume.

Operating System Information (all machines):

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04 LTS"

Package Information (API/Controller):

$ dpkg-query --show nova-*
nova-api 2012.1-0ubuntu2.1
nova-cert 2012.1-0ubuntu2.1
nova-common 2012.1-0ubuntu2.1
nova-doc 2012.1-0ubuntu2.1
nova-network 2012.1-0ubuntu2.1
nova-objectstore 2012.1-0ubuntu2.1
nova-scheduler 2012.1-0ubuntu2.1

Package Information (Volume Node):

$ dpkg-query --show nova-*
nova-common 2012.1-0ubuntu2.3
nova-volume 2012.1-0ubuntu2.3

Revision history for this message
Marc Cluet (lynxman) wrote :

Hi Andrew,

Thank you very much for your bug report.

This is indeed an existing problem in nova, could reproduce easily. Don't know if it's an architectural decision or if it's a bug per se.

Handling over to Chuck Short for further analysis.

Changed in nova (Ubuntu):
status: New → Confirmed
Chuck Short (zulcss)
tags: added: ec2
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/8875

Changed in nova:
assignee: nobody → Chuck Short (zulcss)
status: New → In Progress
Thierry Carrez (ttx)
Changed in nova:
importance: Undecided → High
Revision history for this message
Chuck Short (zulcss) wrote :

This seems to be fixed in raring since im not able to reproduce this anymore.

Changed in nova:
status: In Progress → Invalid
Revision history for this message
Chuck Short (zulcss) wrote :

This seems to be fixed in raring.

Changed in nova (Ubuntu):
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.