OpenStack Compute (Nova)

nova-manage floating list crashes if an instance is missing

Reported by Michael Still on 2012-03-26
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Low
Vish Ishaya

Bug Description

Running "nova-manage floating list" on one of our clusters, I get an uncaught InstanceNotFound exception. Now, I guess this might be a DB problem on our side, but I also think that nova-manage should catch it and not crash out.

2012-03-26 03:13:48 CRITICAL nova [req-4cf3eb24-f0da-4afc-a1f2-a942eabb3bdf None None] Instance 6727 could not be found.
(nova): TRACE: Traceback (most recent call last):
(nova): TRACE: File "./nova-manage", line 2202, in <module>
(nova): TRACE: main()
(nova): TRACE: File "./nova-manage", line 2189, in main
(nova): TRACE: fn(*fn_args, **fn_kwargs)
(nova): TRACE: File "./nova-manage", line 683, in list
(nova): TRACE: instance = db.instance_get(ctxt, fixed_ip['instance_id'])
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/db/api.py", line 591, in instance_get
(nova): TRACE: return IMPL.instance_get(context, instance_id)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 119, in wrapper
(nova): TRACE: return f(*args, **kwargs)
(nova): TRACE: File "/usr/lib/python2.7/dist-packages/nova/db/sqlalchemy/api.py", line 1459, in instance_get
(nova): TRACE: raise exception.InstanceNotFound(instance_id=instance_id)
(nova): TRACE: InstanceNotFound: Instance 6727 could not be found.
(nova): TRACE:

Fix proposed to branch: master
Review: https://review.openstack.org/5799

Changed in nova:
assignee: nobody → Michael Still (mikalstill)
status: New → In Progress
Mark McLoughlin (markmc) wrote :

This could be related to bug #939580 - it would cause disassociate_floating_ip() and a dangling reference leave a the instance

I'm not sure the code should handle this - we should be able to make assertions about the consistency of data in the DB

Michael Still (mikalstill) wrote :

I strongly feel that nova-manage should not crash in this instance. It hampers the ability of ops people to manage the cluster if the tools don't work. Sure, flag the error, but keep running.

tags: added: essex-rc-potential
Thierry Carrez (ttx) on 2012-03-29
Changed in nova:
importance: Undecided → Low
Changed in nova:
assignee: Michael Still (mikalstill) → Vish Ishaya (vishvananda)

Reviewed: https://review.openstack.org/5799
Committed: http://github.com/openstack/nova/commit/37dad352a17203102b4b9b828f84db9b75c57071
Submitter: Jenkins
Branch: master

commit 37dad352a17203102b4b9b828f84db9b75c57071
Author: Michael Still <email address hidden>
Date: Mon Mar 26 14:37:48 2012 +1100

    Handle instances being missing while listing floating IPs

    This resolves bug 964950, which is a nova-manage crash while
    listing floating IPs when an instance has gone missing.

    Change-Id: Ib628356608bc0cbb9089029876ab1df7e9f02531

Changed in nova:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/6034
Committed: http://github.com/openstack/nova/commit/8d2a755552d216821f656a01f3ec1cd32cb6a286
Submitter: Jenkins
Branch: milestone-proposed

commit 8d2a755552d216821f656a01f3ec1cd32cb6a286
Author: Michael Still <email address hidden>
Date: Mon Mar 26 14:37:48 2012 +1100

    Handle instances being missing while listing floating IPs

    This resolves bug 964950, which is a nova-manage crash while
    listing floating IPs when an instance has gone missing.

    Change-Id: Ib628356608bc0cbb9089029876ab1df7e9f02531

Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-04-05
Changed in nova:
milestone: none → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers