event list REST API call should support nested stacks

[Steve Baker]

Currently heatclient builds the nested stack event list by doing recursive client-side event list calls for each stack. The REST API should support this natively so it can be implemented in a more efficient way.

The GET call to list events will support a nested_depth parameter. The response will have an additional links url with the ref 'root_stack' to indicate that this API supports nested_depth queries.

This has the following consequences for old/new combinations of
 client/server
- new heatclient, new server - nested_depth param is set, server returns nested events
- new heatclient, old server - nested_depth param is set, server returns events with no root_stack, heatclient falls back to recursive event fetch
- old heatclient, new server - nested_depth param is never set, recursive event fetch works as before

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/325034

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/325035

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/327976

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/325034
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=7bfbace79fa0fb3b7551bdf9388237f936c92032
Submitter: Jenkins
Branch: master

commit 7bfbace79fa0fb3b7551bdf9388237f936c92032
Author: Steve Baker <email address hidden>
Date: Fri Jun 3 14:56:43 2016 +1200

    Use Event versioned object for event_list formatting

    This change removes the event.Event.load method, and switches
    api.format_event to using the Event versioned object instead.

    Since the only purpose of loading the stack is to get its identifier,
    this is now also coming from a Stack versioned object.

    Related-Bug: #1588561
    Change-Id: I7b1fc99894818b44cde08cf08b010b1d1fb94e9f

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/325035
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=2a04ea4bb55a8c6f140134ec91bd700e8207dfd2
Submitter: Jenkins
Branch: master

commit 2a04ea4bb55a8c6f140134ec91bd700e8207dfd2
Author: Steve Baker <email address hidden>
Date: Fri Jun 3 15:10:05 2016 +1200

    Do a single query to fetch all event_list stacks

    In preperation for supporting event list --nested-depth, do a single
    query to fetch the stacks that correspond to the event list.

    Related-Bug: #1588561

    Change-Id: I106c21bbb87b9a591f3aba7de03eb130e7fd7b19

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/326229
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=32ade7a24342781dc6887aae78d83d2119f0c8cd
Submitter: Jenkins
Branch: master

commit 32ade7a24342781dc6887aae78d83d2119f0c8cd
Author: Steve Baker <email address hidden>
Date: Fri Jun 3 12:58:30 2016 +1200

    Implement event list nested-depth

    The GET call to list events will support a nested_depth parameter. The
    response will have an additional links url with the ref 'root_stack'
    to indicate that this API supports nested_depth queries.

    This has the following consequences for old/new combinations of
    client/server
    - new heatclient, new server - nested_depth param is set, server
      returns nested events
    - new heatclient, old server - nested_depth param is set, server
      returns events with no root_stack, heatclient falls back to
      recursive event fetch
    - old heatclient, new server - nested_depth param is never set,
      recursive event fetch works as before

    Here are some timings for a TripleO overcloud stack with ~700 events.

    Current heat and python-heatclient master:
    time openstack stack event list --nested-depth 4 overcloud |wc -l
    744
    real 0m17.500s

    This change, with heatclient 31278ff5f77b152b5ef7a4197e15c441c72ff163:
    time openstack stack event list --nested-depth 4 overcloud |wc -l
    608
    real 0m1.725s

    The difference in event count (744 vs 608) is due to the stack events
    being filtered out for stacks with zero resources - these are a source
    of unnecessary noise so their removal should be considered an improvement.

    Closes-Bug: #1588561

    Change-Id: I27e1ffb770e00a7f929c081b2a505e2007f5d584

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/327976
Committed: https://git.openstack.org/cgit/openstack/python-heatclient/commit/?id=31278ff5f77b152b5ef7a4197e15c441c72ff163
Submitter: Jenkins
Branch: master

commit 31278ff5f77b152b5ef7a4197e15c441c72ff163
Author: Steve Baker <email address hidden>
Date: Thu Jun 9 08:47:17 2016 +1200

    Implement client side of event list --nested-depth

    This change does the following:
    - cleans up the usage of get_events so that marker and limit are only
      specified in their dedicated arguments, not also in event_args
      (also, specifying only in event_args still works)
    - first attempts server-side nested_depth support
    - falls back to the old recursive approach if the response data lacks a
      link with the ref root_stack

    Since there is a fallback for old heat APIs, the client change can
    land before or after the heat change
    I27e1ffb770e00a7f929c081b2a505e2007f5d584

    Change-Id: I43d12ec61ec359222184f07c170de3c97481f1ba
    Closes-Bug: #1588561

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/heat 7.0.0.0b2 development milestone.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/python-heatclient 1.3.0 release.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/521651

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/521651
Committed: https://git.openstack.org/cgit/openstack/heat/commit/?id=23e06f66b9e0ccb667c24318764da72aa82d4e18
Submitter: Zuul
Branch: master

commit 23e06f66b9e0ccb667c24318764da72aa82d4e18
Author: Zane Bitter <email address hidden>
Date: Mon Nov 20 15:06:13 2017 -0500

    Speed up event listing with nested stacks

    When handling the command "openstack stack event list --nested-depth=n", we
    obtain the list of nested stacks by querying the database for all resources
    that share the root stack ID. However, since all we're getting is the stack
    IDs, there's no need to query all of the fields and construct a versioned
    object for each resource. Just get the set of stack IDs.

    Change-Id: I12155433e2ac1af919aa4b5e780fb965cd5885d8
    Related-Bug: #1588561