'409 Conflict' occurred when adding reserved_host to aggregate

[takahara.kengo]

[Conditions]
* 'reserved_host' is set at 'recovery_method' column of failover_segment.
* Reserved_host has already been added to the aggregate to which the failed host belongs.

[Bug]
Masakari-engine failed to recovery instances because '409 Conflict' occurred when executing add_host API[1]. So recovery process finished with 'error'.
The reason that '409 Conflict' was occurred is that reserved_host had already been added to the aggregate to which the failed host belongs. Therefore, when masakari-engine executes add_host API, it should not add reserved_host if it has already been added to the aggregate or should simply ignore '409 Conflict'.

[1]:
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver Traceback (most recent call last):
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/taskflow/engines/action_engine/executor.py", line 53, in _execute_task
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver result = task.execute(**arguments)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/masakari/engine/drivers/taskflow/host_failure.py", line 294, in execute
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver reserved_host)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/masakari/utils.py", line 284, in inner
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver return f(*args, **kwargs)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/masakari/engine/drivers/taskflow/host_failure.py", line 290, in do_evacuate_with_reserved_host
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver reserved_host=reserved_host)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/masakari/engine/drivers/taskflow/host_failure.py", line 246, in _do_evacuate
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver context, reserved_host.name, aggregate)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/masakari/compute/nova.py", line 78, in wrapper
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver _reraise(exception.Conflict(reason=err_msg))
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/masakari/compute/nova.py", line 50, in _reraise
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver six.reraise(type(desired_exc), desired_exc, sys.exc_info()[2])
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/masakari/compute/nova.py", line 58, in wrapper
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver res = method(self, ctx, *args, **kwargs)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/masakari/compute/nova.py", line 229, in add_host_to_aggregate
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver return nova.aggregates.add_host(aggregate.id, host)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/novaclient/v2/aggregates.py", line 84, in add_host
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver body, "aggregate")
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/novaclient/base.py", line 361, in _create
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver resp, body = self.api.client.post(url, body=body)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/keystoneauth1/adapter.py", line 240, in post
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver return self.request(url, 'POST', **kwargs)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver File "/usr/local/lib/python2.7/dist-packages/novaclient/client.py", line 83, in request
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver raise exceptions.from_response(resp, body, url, method)
2017-12-19 09:39:05.455 9001 ERROR masakari.engine.drivers.taskflow.driver Conflict: Conflict

[takahara.kengo (takahara.kengo)]

I pushed a patch.
https://review.openstack.org/#/c/531310/

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/masakari 5.0.0 release.