Loading stack using stored context calls keystone

[Zane Bitter]

When we load a stack from the database using the stored user context, we make a bunch of calls to Keystone to fetch the roles, user_domain_id and project_domain_id. These calls are fixes for bug 1529354 and bug 1531406.

One of the things we pass stored_context=True for is at startup for loading stacks that were left stuck IN_PROGRESS by some previous dead engine. We are unable to do this if Keystone is not available - which is a fairly likely scenario after e.g. a restart of the box.

If we could lazily load the information we need only when it is required, it would not only be more efficient but we could also be sure to always reset stuck stacks at startup regardless of the order in which services start.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/315772

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/317604

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/315772
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=bcadd55a277b8b8170078f3eca7cfe471456e70b
Submitter: Jenkins
Branch: master

commit bcadd55a277b8b8170078f3eca7cfe471456e70b
Author: Zane Bitter <email address hidden>
Date: Thu May 12 16:20:20 2016 -0400

    Lazy-load context information requiring Keystone calls

    When we load a stored context, don't retrieve additional data from keystone
    unless and until it is needed. This makes stacks that don't require the
    full data more efficient, and may help prevent an inability to contact
    keystone at startup blocking the process of resetting stuck stacks.

    Change-Id: Ifa8f5f22686a2ee5c30b8cdb42d921163467b64a
    Co-Authored-By: Thomas Herve <email address hidden>
    Partial-Bug: #1570569

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/317604
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=026cc94eba48364b35a561fd21e33d6d482ab39d
Submitter: Jenkins
Branch: master

commit 026cc94eba48364b35a561fd21e33d6d482ab39d
Author: Zane Bitter <email address hidden>
Date: Tue May 17 11:08:03 2016 -0400

    Don't use stored context to reset stacks

    The resetting of in-progress stacks at startup is done using an admin
    context. No user actions (e.g. ReST API calls) are performed, so there is
    no need to load the user context that is stored with the stack. (Nor is the
    current context rewritten to the database in the process of resetting the
    state.)

    Since loading the user context from the DB may result in a call to the
    keystone API, eliminating this ensures that stacks can be reset
    successfully even if heat-engine starts before keystone is available.

    Change-Id: I8109c622b7661254a658d78d98f8dc8f756d8755
    Closes-Bug: #1570569

[huangtianhua (huangtianhua)]

I found a bug, seems introduced by this change, please see https://bugs.launchpad.net/heat/+bug/1584724, and I think we should revert this change first.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/320348

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/heat 7.0.0.0b1 development milestone.

[Zane Bitter (zaneb)]

Reopening as the first patch was reverted.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/320348
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=9d239cc80e218dd75571e33e5171f6fee82023b9
Submitter: Jenkins
Branch: master

commit 9d239cc80e218dd75571e33e5171f6fee82023b9
Author: huangtianhua <email address hidden>
Date: Tue May 24 18:36:46 2016 +0800

    Don't use stored context to reset stacks

    The resetting of in-progress stacks at startup is done using an admin
    context. No user actions (e.g. ReST API calls) are performed, so there is
    no need to load the user context that is stored with the stack. (Nor is
    the
    current context rewritten to the database in the process of resetting the
    state.)

    Since loading the user context from the DB may result in a call to the
    keystone API, eliminating this ensures that stacks can be reset
    successfully even if heat-engine starts before keystone is available.

    Also there is no need to resolve outputs data while reset stacks, so set
    resolve_data=False when load stack.

    Change-Id: I9379ef6cc6f1d3bbc2569ff2150795bfaffee430
    Closes-Bug: #1570569

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/heat 7.0.0.0b2 development milestone.