EC2 get_metadata fails with InstanceNotFound

[Dan Prince]

I'm getting the following failure in nova-api.log when trying to run the nova smoketests w/ nova trunk.

2011-10-12 14:58:49,302 ERROR nova.api.ec2.metadata [-] Failed to get metadata for ip: 192.168.0.4
(nova.api.ec2.metadata): TRACE: Traceback (most recent call last):
(nova.api.ec2.metadata): TRACE: File "/usr/lib/python2.6/dist-packages/nova/api/ec2/metadatarequesthandler.py", line 79, in __call__
(nova.api.ec2.metadata): TRACE: meta_data = self.cc.get_metadata(remote_address)
(nova.api.ec2.metadata): TRACE: File "/usr/lib/python2.6/dist-packages/nova/api/ec2/cloud.py", line 383, in get_metadata
(nova.api.ec2.metadata): TRACE: return_all=False)
(nova.api.ec2.metadata): TRACE: File "/usr/lib/python2.6/dist-packages/nova/api/ec2/cloud.py", line 275, in _get_floaters_for_instance
(nova.api.ec2.metadata): TRACE: fixed_ips = self._get_fixed_ips_for_instance(context, instance)[0]
(nova.api.ec2.metadata): TRACE: File "/usr/lib/python2.6/dist-packages/nova/api/ec2/cloud.py", line 251, in _get_fixed_ips_for_instance
(nova.api.ec2.metadata): TRACE: nw_info = self.network_api.get_instance_nw_info(context, instance)
(nova.api.ec2.metadata): TRACE: File "/usr/lib/python2.6/dist-packages/nova/network/api.py", line 175, in get_instance_nw_info
(nova.api.ec2.metadata): TRACE: raise exception.InstanceNotFound(instance_id=instance['id'])
(nova.api.ec2.metadata): TRACE: InstanceNotFound: Instance 7 could not be found.
(nova.api.ec2.metadata): TRACE:
2011-10-12 14:58:49,304 INFO nova.api [-] 2.135844s 192.168.0.4 GET /2009-04-04/meta-data/local-ipv4 None:None 500 [Wget] text/plain text/plain

---

I've been seeing this failure for almost 2 weeks now although due to a variety of issues (PPA packaging, networking code chaos, etc.) I'm not sure which changeset caused the exact failure.

[Dan Prince (dan-prince)]

+++ b/nova/api/ec2/cloud.py
@@ -285,7 +285,7 @@ class CloudController(object):
This seems to fix it although I'm not entirely sure some of the networking code changes aren't to blame as well (related to fixed_ips not getting unset or something).

     def _get_mpi_data(self, context, project_id):
         result = {}
- search_opts = {'project_id': project_id}
+ search_opts = {'project_id': project_id, 'deleted': False}
         for instance in self.compute_api.get_all(context,
                 search_opts=search_opts):
             # only look at ipv4 addresses
@@ -360,7 +360,7 @@ class CloudController(object):

     def get_metadata(self, address):
         ctxt = context.get_admin_context()
- search_opts = {'fixed_ip': address}
+ search_opts = {'fixed_ip': address, 'deleted': False}

[Dan Prince (dan-prince)]

Fixed in:

https://github.com/openstack/nova/commit/c0cf874acb3a67371ebbd5abbd274f61ffa09396

[Thierry Carrez (ttx)]

Keep bugs fix committed until they are released.