_move_operation_alloc_request fails with TypeError when using 1.12 version allocation request

[Matt Riedemann]

Seen here in the alternate hosts series:

http://logs.openstack.org/58/511358/43/check/openstack-tox-functional/e642310/job-output.txt.gz#_2017-12-19_00_18_34_585930

2017-12-19 00:18:34.585930 | ubuntu-xenial | Traceback (most recent call last):
2017-12-19 00:18:34.585992 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional/local/lib/python2.7/site-packages/oslo_messaging/rpc/server.py", line 163, in _process_incoming
2017-12-19 00:18:34.586021 | ubuntu-xenial | res = self.dispatcher.dispatch(message)
2017-12-19 00:18:34.586082 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 220, in dispatch
2017-12-19 00:18:34.586114 | ubuntu-xenial | return self._do_dispatch(endpoint, method, ctxt, args)
2017-12-19 00:18:34.586179 | ubuntu-xenial | File "/home/zuul/src/git.openstack.org/openstack/nova/.tox/functional/local/lib/python2.7/site-packages/oslo_messaging/rpc/dispatcher.py", line 190, in _do_dispatch
2017-12-19 00:18:34.586207 | ubuntu-xenial | result = func(ctxt, **new_args)
2017-12-19 00:18:34.586241 | ubuntu-xenial | File "nova/conductor/manager.py", line 603, in build_instances
2017-12-19 00:18:34.586267 | ubuntu-xenial | host.allocation_request_version)
2017-12-19 00:18:34.586300 | ubuntu-xenial | File "nova/scheduler/utils.py", line 800, in claim_resources
2017-12-19 00:18:34.586335 | ubuntu-xenial | user_id, allocation_request_version=allocation_request_version)
2017-12-19 00:18:34.586370 | ubuntu-xenial | File "nova/scheduler/client/__init__.py", line 37, in __run_method
2017-12-19 00:18:34.586402 | ubuntu-xenial | return getattr(self.instance, __name)(*args, **kwargs)
2017-12-19 00:18:34.586435 | ubuntu-xenial | File "nova/scheduler/client/report.py", line 61, in wrapper
2017-12-19 00:18:34.586459 | ubuntu-xenial | return f(self, *a, **k)
2017-12-19 00:18:34.586493 | ubuntu-xenial | File "nova/scheduler/client/report.py", line 110, in wrapper
2017-12-19 00:18:34.586516 | ubuntu-xenial | return f(self, *a, **k)
2017-12-19 00:18:34.586552 | ubuntu-xenial | File "nova/scheduler/client/report.py", line 1126, in claim_resources
2017-12-19 00:18:34.586586 | ubuntu-xenial | payload = _move_operation_alloc_request(current_allocs, ar)
2017-12-19 00:18:34.586625 | ubuntu-xenial | File "nova/scheduler/client/report.py", line 199, in _move_operation_alloc_request
2017-12-19 00:18:34.586657 | ubuntu-xenial | for a in dest_alloc_req['allocations']) - cur_rp_uuids
2017-12-19 00:18:34.586685 | ubuntu-xenial | TypeError: string indices must be integers

This is due to using a 1.12 version allocation candidate request bug _move_operation_alloc_request is expecting the <1.12 format, where allocations is a list instead of a dict.

I don't know if we should change the calling code to format the allocation request to the <1.12 format, or make _move_operation_alloc_request handle both styles (probably better to do the latter).

[Ed Leafe (ed-leafe)]

This *may* be fixed here:

https://review.openstack.org/#/c/511358/43..44/nova/conductor/manager.py@607

The Selection object stores the request as a JSON-encoded blob, and we were passing it directly. The change above now decodes it back to a dict.

[Matt Riedemann (mriedem)]

Hmm, maybe yeah. And we're fixing the move thing another way in change 511358 so we shouldn't even get that far for the move method call, but _move_operation_alloc_request still expects a certain type of allocation request format and that's incompatible with the new 1.14 model, so I think the bug still stands, but it's low priority to fix.

[Chris Dent (cdent)]

Not sure if this is related to this or not, but throughout the nova-side claims-related code we make tweaks on the allocation data structure that we once called "opaque" but actually isn't.

That code is based on the format of allocations and allocation_candidates in microversion < 1.12

It's unclear to me if the "tweaks" are happy with either style of data or not, or if it matters, but since I saw this bug go by, thought I better note it in case it tickles any brain cells on anyone. I don't have any right this minute to do anything thinking about this, right now.

[Chris Dent (cdent)]

The strings in the log don't seem to be showing up in recent logstash, so I'm going to mark this one dead, so it's not lingering.

[Matt Riedemann (mriedem)]

I believe Alex's patch handles both formats which fixes this bug:

https://github.com/openstack/nova/commit/48d1ee1dd0e09b69adbb5e22208d7b995c0f17a1#diff-94f87e728df6465becce5241f3da53c8