Gate jobs failing with InvalidInput exception

Bug #1491262 reported by Marc Koderer
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Shared File Systems Service (Manila)
Invalid
Critical
Valeriy Ponomaryov

Bug Description

See http://logs.openstack.org/64/219264/1/check/gate-manila-tempest-dsvm-neutron/28a136d/logs/screen-m-shr.txt.gz?level=ERROR

Jobs are failing with:

2015-09-02 06:12:09.420 1997 ERROR manila.share.manager [req-f7d19508-463f-48be-8635-6e0f14cbb6a5 ] Share instance 12f5898a-bfe5-489f-ab99-a2750aa3eee8 failed on creation.
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher [req-f7d19508-463f-48be-8635-6e0f14cbb6a5 ] Exception during message handling: Invalid input received: Invalid input for field/attribute device. Value: None. None is not of type 'string' (HTTP 400) (Request-ID: req-bf466a08-346c-4a2a-9c49-42eb57f3b443).
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 142, in _dispatch_and_reply
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher executor_callback))
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 186, in _dispatch
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher executor_callback)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 129, in _do_dispatch
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/manager.py", line 445, in create_share_instance
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher {'status': constants.STATUS_ERROR}
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_utils/excutils.py", line 195, in __exit__
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.type_, self.value, self.tb)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/manager.py", line 416, in create_share_instance
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher context, share_instance, share_server=share_server)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 125, in wrap
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher return f(self, context, *args, **kwargs)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 209, in create_share
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher volume)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 376, in _attach_volume
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher return do_attach(volume)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/oslo_concurrency/lockutils.py", line 254, in inner
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher return f(*args, **kwargs)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 361, in do_attach
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher attach_volume()
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 49, in wrapped_f
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher return Retrying(*dargs, **dkw).call(f, *args, **kw)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 212, in call
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher raise attempt.get()
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 247, in get
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher six.reraise(self.value[0], self.value[1], self.value[2])
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/usr/local/lib/python2.7/dist-packages/retrying.py", line 200, in call
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher attempt = Attempt(fn(*args, **kwargs), attempt_number, False)
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/share/drivers/generic.py", line 359, in attach_volume
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher self.admin_context, instance_id, volume['id'])
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher File "/opt/stack/new/manila/manila/compute/nova.py", line 158, in wrapper
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher raise exception.InvalidInput(reason=six.text_type(e))
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher InvalidInput: Invalid input received: Invalid input for field/attribute device. Value: None. None is not of type 'string' (HTTP 400) (Request-ID: req-bf466a08-346c-4a2a-9c49-42eb57f3b443).
2015-09-02 06:12:09.438 1997 ERROR oslo_messaging.rpc.dispatcher

Revision history for this message
Valeriy Ponomaryov (vponomaryov) wrote :

This bug caused by bump of minimal version for Nova, see commit [1]

Was v2.0, which is legacy version, now set v2.1

v2.1 has validation of input data before their handling - it is main difference. And in case of Manila, without change of Nova, we need start calculating device paths for each volume, it will require sync of all volume device paths for each server.

Another possible variant - have some possibility to make Nova such decision as it was before.

[1] https://github.com/openstack/nova/commit/a9090bb5f50310702408aee340af9d4dad9474cf

Changed in manila:
importance: Undecided → Critical
Revision history for this message
Valeriy Ponomaryov (vponomaryov) wrote :
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to manila (master)

Fix proposed to branch: master
Review: https://review.openstack.org/220028

Changed in manila:
assignee: nobody → Marc Koderer (m-koderer)
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/220204

Changed in manila:
assignee: Marc Koderer (m-koderer) → Valeriy Ponomaryov (vponomaryov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on manila (master)

Change abandoned by Marc Koderer (<email address hidden>) on branch: master
Review: https://review.openstack.org/220028

Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Change abandoned by Valeriy Ponomaryov (<email address hidden>) on branch: master
Review: https://review.openstack.org/220204

Revision history for this message
Valeriy Ponomaryov (vponomaryov) wrote :

Fix went to Nova project. Manila not affected by it anymore.

Changed in manila:
status: In Progress → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.