ec2 metadata request fails with: AttributeError: 'NoneType' object has no attribute 'instance'

[Ihar Hrachyshka]

I see the following traceback in nova-api log:

2014-04-22 17:56:40.228 5004 ERROR nova.api.metadata.handler [-] Failed to get metadata for instance id: 9ac5c1ae-c481-4a3e-8b36-a567b146c42f
2014-04-22 17:56:40.229 5004 ERROR nova.api.ec2 [-] FaultWrapper: 'NoneType' object has no attribute 'instance'
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 Traceback (most recent call last):
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/nova/api/ec2/__init__.py", line 87, in __call__
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 return req.get_response(self.application)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/WebOb-1.2.3-py2.6.egg/webob/request.py", line 1296, in send
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 application, catch_exc_info=False)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/WebOb-1.2.3-py2.6.egg/webob/request.py", line 1260, in call_application
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 app_iter = application(self.environ, start_response)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/WebOb-1.2.3-py2.6.egg/webob/dec.py", line 130, in __call__
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 resp = self.call_func(req, *args, **self.kwargs)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/WebOb-1.2.3-py2.6.egg/webob/dec.py", line 195, in call_func
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 return self.func(req, *args, **kwargs)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/nova/api/ec2/__init__.py", line 99, in __call__
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 rv = req.get_response(self.application)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/WebOb-1.2.3-py2.6.egg/webob/request.py", line 1296, in send
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 application, catch_exc_info=False)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/WebOb-1.2.3-py2.6.egg/webob/request.py", line 1260, in call_application
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 app_iter = application(self.environ, start_response)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/WebOb-1.2.3-py2.6.egg/webob/dec.py", line 130, in __call__
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 resp = self.call_func(req, *args, **self.kwargs)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/WebOb-1.2.3-py2.6.egg/webob/dec.py", line 195, in call_func
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 return self.func(req, *args, **kwargs)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/nova/api/metadata/handler.py", line 107, in __call__
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 meta_data = self._handle_instance_id_request(req)
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 File "/usr/lib/python2.6/site-packages/nova/api/metadata/handler.py", line 203, in _handle_instance_id_request
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 if meta_data.instance['project_id'] != tenant_id:
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2 AttributeError: 'NoneType' object has no attribute 'instance'
2014-04-22 17:56:40.229 5004 TRACE nova.api.ec2

[Joe Gordon (jogo)]

Can you provide some more context of how to reproduce this issue.

[Ihar Hrachyshka (ihar-hrachyshka)]

@Joe, I've seen this when e.g. metadata-agent stops to respond due to its own bug.

[Felipe Alfaro Solana (felipe-alfaro-gmail)]

I'm experiencing the same behavior.

[Ihar Hrachyshka (ihar-hrachyshka)]

BTW there is a patch in review for that: https://review.openstack.org/96147

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/96147
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=904fad326b249672908c94daf03c8c35cd4c7365
Submitter: Jenkins
Branch: master

commit 904fad326b249672908c94daf03c8c35cd4c7365
Author: Ihar Hrachyshka <email address hidden>
Date: Wed May 28 14:00:14 2014 +0200

    EC2: fixed AttributeError when metadata is not found

    Change-Id: I91e0141bd89cd18596ccae2ea213f641ba4d7f74
    Closes-Bug: 1324091