Memory leaks when using "join" in workflows

[Renat Akhmerov]

Memory usage consistently grows if we keep running workflows containing "join" tasks. Even if we run garbage collector explicitly it doesn't get released.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/435404

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/435404
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=086a3d43fa734dae61ed40912cde701912318dc9
Submitter: Jenkins
Branch: master

commit 086a3d43fa734dae61ed40912cde701912318dc9
Author: Renat Akhmerov <email address hidden>
Date: Wed Feb 15 18:16:57 2017 +0700

    Fix memory leak related to cached lookups

    * It turned out that Mistral used a lot of memory because it
      used cached DB lookups for task executions (only task executions
      in a terminal state get cached) and the maximum size of the cache
      was too big, 20000 entries. One task execution in certain cases
      may take a lot of memory (e.g. several megabytes) so 20000 objects
      make memory footprint huge. Additionally, when a workflow completes
      we need to invalidate coresponding task executions in the cache.
      This didn't happen before this patch.
    * This patch fixes the aforementioned issues by using partial
      invalidation of the cache and setting smaller cache size.
    * Fixed "Starting workflow .." log statement to not print the entire
      structure of the workflow definition into the workflow log, only its
      name and input parameters
    * Minor style fixes

    Change-Id: I0ee300f631a4bdfa2f618c2a10048267f27b3345
    Closes-bug: #1664864

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/436575

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/436575
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=060a153922d350dc3f55dc9500c8d621549d670a
Submitter: Jenkins
Branch: stable/ocata

commit 060a153922d350dc3f55dc9500c8d621549d670a
Author: Renat Akhmerov <email address hidden>
Date: Wed Feb 15 18:16:57 2017 +0700

    Fix memory leak related to cached lookups

    * It turned out that Mistral used a lot of memory because it
      used cached DB lookups for task executions (only task executions
      in a terminal state get cached) and the maximum size of the cache
      was too big, 20000 entries. One task execution in certain cases
      may take a lot of memory (e.g. several megabytes) so 20000 objects
      make memory footprint huge. Additionally, when a workflow completes
      we need to invalidate coresponding task executions in the cache.
      This didn't happen before this patch.
    * This patch fixes the aforementioned issues by using partial
      invalidation of the cache and setting smaller cache size.
    * Fixed "Starting workflow .." log statement to not print the entire
      structure of the workflow definition into the workflow log, only its
      name and input parameters
    * Minor style fixes

    Change-Id: I0ee300f631a4bdfa2f618c2a10048267f27b3345
    Closes-bug: #1664864
    (cherry picked from commit 086a3d43fa734dae61ed40912cde701912318dc9)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/453466

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/453466
Committed: https://git.openstack.org/cgit/openstack/mistral/commit/?id=00199fdfe8adb2f07c9d809d845cd46c7f0317a0
Submitter: Jenkins
Branch: stable/newton

commit 00199fdfe8adb2f07c9d809d845cd46c7f0317a0
Author: Renat Akhmerov <email address hidden>
Date: Wed Feb 15 18:16:57 2017 +0700

    Fix memory leak related to cached lookups

    * It turned out that Mistral used a lot of memory because it
      used cached DB lookups for task executions (only task executions
      in a terminal state get cached) and the maximum size of the cache
      was too big, 20000 entries. One task execution in certain cases
      may take a lot of memory (e.g. several megabytes) so 20000 objects
      make memory footprint huge. Additionally, when a workflow completes
      we need to invalidate coresponding task executions in the cache.
      This didn't happen before this patch.
    * This patch fixes the aforementioned issues by using partial
      invalidation of the cache and setting smaller cache size.
    * Fixed "Starting workflow .." log statement to not print the entire
      structure of the workflow definition into the workflow log, only its
      name and input parameters
    * Minor style fixes

    Change-Id: I0ee300f631a4bdfa2f618c2a10048267f27b3345
    Closes-bug: #1664864
    (cherry picked from commit 086a3d43fa734dae61ed40912cde701912318dc9)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/mistral 5.0.0.0b1 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/mistral 4.0.1 release.