Comment 4 for bug 1783613

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (stable/ocata)

Reviewed: https://review.openstack.org/582413
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=456b3d68bc491a6965316e6a4f3fc11dd509237b
Submitter: Zuul
Branch: stable/ocata

commit 456b3d68bc491a6965316e6a4f3fc11dd509237b
Author: melanie witt <email address hidden>
Date: Thu Jul 12 22:42:19 2018 +0000

    [stable only] Handle quota usage during create/delete races

    There is a race during an instance delete while booting where, if the
    build request was found, there is still a chance we can fail to lookup
    the instance record and if so, we need to go ahead and commit the quota
    decrement reservations. We're currently skipping the quotas.commit() if
    we *don't* find the instance record. I believe this is a regression as of
    commit 361b88383130be8da9d474d02a9f62136239d506.

    There are two reasons why we may fail to find the instance record after
    we successfully found the build request:

      a) Conductor didn't create the instance record yet.
      b) Conductor deleted the instance record after finding the build
         request was deleted by us (the API)

    In either case, conductor doesn't do anything to decrement quota usage
    so we need to do it in the compute/api.

    There is a second race where we find the build request and succeed in
    looking up the instance record but then fail to delete the instance
    record (instance.destroy() raises InstanceNotFound). This can happen if

      c) Conductor deleted the instance record after finding the build
         request was deleted by us but after we looked up the instance
      d) If we (the API) are racing with another delete request

    If c) happens, we need to quotas.commit(). If d) happens, we need to
    quotas.rollback(). Since we can't know which case it was when we get
    InstanceNotFound, we'll do a quotas.rollback() to get rid of the
    reservation record and force a refresh of quota usages to make sure
    we end with correct usage in case rollback was not the right choice.

    Closes-Bug: #1783613

    Change-Id: I4d492dcad193ff0b1202dcae97954cc65b75c109