Blazar's reservation resource class and the inventory is not deleted properly at the lease termination.

Bug #1813252 reported by Tetsuro Nakamura
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Blazar
Fix Released
High
Tetsuro Nakamura

Bug Description

Description
===========

Blazar's reservation resource class and the inventory is not deleted properly at the lease termination.

Steps to reproduce
=================

1) Create a lease with an instance reservation.

2) Start the reserved intstance during the lease time.

3) Do nothing before the lease end time comes.

4) Blazar asks placement to delete the reservation inventory on [1]. However, placement says the inventory can't be deleted it since the instance deletion [2] is not finished.

[1] https://github.com/openstack/blazar/blob/98a203b3e763bf0a55dcaf545964f62a0ff59085/blazar/plugins/instances/instance_plugin.py#L493-L497

[2] https://github.com/openstack/blazar/blob/98a203b3e763bf0a55dcaf545964f62a0ff59085/blazar/plugins/instances/instance_plugin.py#L499-L502

Expected result
===============

Reservation resource class and its inventory is deleted properly at the lease termination.

Actual result
=============

It fails with the following log.

Jan 25 07:09:32 ubuntu01 blazar-manager[29323]: 2019-01-25 07:09:32.044 29323 INFO blazar.utils.openstack.placement [-] Resource class CUSTOM_RESERVATION_3328E24F_42CD_4291_844E_66DE1673FC2D doesn't exist or there is no inventory for that resource class on resource provider blazar_ubuntu01. Skipped the deletion
Jan 25 07:09:34 ubuntu01 blazar-manager[29323]: 2019-01-25 07:09:34.666 29323 DEBUG blazar.utils.openstack.nova [-] Removing host 'ubuntu01' from aggregate '4') delete /opt/stack/blazar/blazar/utils/openstack/nova.py:303
Jan 25 07:09:38 ubuntu01 blazar-manager[29323]: 2019-01-25 07:09:38.018 29323 ERROR blazar.utils.openstack.placement [-] Failed to delete resource class with placement API for CUSTOM_RESERVATION_3328E24F_42CD_4291_844E_66DE1673FC2D. Got 409: {"errors": [{"status": 409, "title": "Conflict", "detail": "There was a conflict when trying to complete your request.\n\n Error in delete resource class: Cannot delete resource class CUSTOM_RESERVATION_3328E24F_42CD_4291_844E_66DE1673FC2D. Class is in use in inventory. ", "code": "placement.undefined_code", "request_id": "req-c8c5e300-409c-45f5-9f46-8c16e9141f6b"}]}.
Jan 25 07:09:38 ubuntu01 blazar-manager[29323]: 2019-01-25 07:09:38.018 29323 INFO blazar.utils.openstack.placement [-] Resource class CUSTOM_RESERVATION_3328E24F_42CD_4291_844E_66DE1673FC2D doesn't exist. Skipped the deletion of the resource class
Jan 25 07:09:38 ubuntu01 blazar-manager[29323]: 2019-01-25 07:09:38.086 29323 DEBUG blazar.status [-] Status of lease 49af66ab-2664-423e-9cec-c20e28653b92 changed from TERMINATING to TERMINATED. wrapper /opt/stack/blazar/blazar/status.py:222

description: updated
Changed in blazar:
status: New → Confirmed
assignee: nobody → Tetsuro Nakamura (tetsuro0907)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to blazar (master)

Fix proposed to branch: master
Review: https://review.openstack.org/633203

Changed in blazar:
status: Confirmed → In Progress
Pierre Riteau (priteau)
Changed in blazar:
importance: Undecided → High
milestone: none → stein-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to blazar (master)

Reviewed: https://review.openstack.org/633203
Committed: https://git.openstack.org/cgit/openstack/blazar/commit/?id=1855e059fca6398999ec8c124d6435197132558a
Submitter: Zuul
Branch: master

commit 1855e059fca6398999ec8c124d6435197132558a
Author: Tetsuro Nakamura <email address hidden>
Date: Sun Jan 27 12:35:17 2019 +0000

    Call placement after instance delete

    Blazar called placement to delete reservation inventory on reservation
    resource provider *before* blazar asks nova to delete the instance.
    This caused inventory in use 409 conflict error in placement.

    This patch re-orders it to call placement after instance delete and
    adds a function to ensure the deletion is completed not to cause the
    error.

    Change-Id: I3b4a892a783140b4a25810aeed71c79f6207e155
    Depends-On: https://review.openstack.org/#/c/633408/
    Closes-Bug: #1813252

Changed in blazar:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/blazar 3.0.0.0rc1

This issue was fixed in the openstack/blazar 3.0.0.0rc1 release candidate.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.