cinder replication can leave volume replication_state at an irrecoverable value

[Mudassir Latif]

For example: trying to enabled replication, volume api sets the replication_status to 'enabling_replication'. If an exception is thrown by the driver, the manager fails to set the status to error.

This also happens when a volume goes into an 'error' or 'failed-over' state.

To ensure that an 'error' state is not terminal, we need to allow replication_status to transition 'error' --> 'enabled'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/219900

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/219900
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=519175176032e6c45149b7e234b035ad2f2e9abb
Submitter: Jenkins
Branch: master

commit 519175176032e6c45149b7e234b035ad2f2e9abb
Author: Mudassir Latif <email address hidden>
Date: Wed Sep 2 15:07:04 2015 -0700

    Allow replicated volumes to be recoverable

    Currently, once they are in ‘error’ or ‘failed-over’ states you can’t
    do anything with them. We need to allow enable_replication after the
    admin has resolved any issues that may have prevented the operation.

    Same thing for after a fail-over, you cannot enable replication on it.

    This change allows for more states to be recoverable via enable
    replication. It also more uniformly sets the state to ‘error’ when
    something bad happens.

    'enabled' --> replication is on.
    'disabled' --> replication is off by design.
    'failed-over' --> we have just failed over. Replication is off.
    'error' --> an error occurred during the last operation.

    allowed flow is:
    'enabled' --> 'failed-over'/'disabled'/'error'
    'error' --> 'enabled'/'error'
    'disabled' --> 'enabled'/'error'
    'failed-over' --> 'enabled'/'error'

    Change-Id: Icbc22b8501b048b1755019305430177ba0a9385f
    Partial-Bug: #1491631