glance-cache-cleaner/pruner is executed in multiple containers at the same time

Bug #1979271 reported by Takashi Kajinami
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Fix Released
Medium
Takashi Kajinami

Bug Description

Description
===========
When GlanceCacheEnabled is true, the following glance-cache commands are executed by the glance_api_cron container.

The glance_api_cron container is deployed on every controller node, thus if a deployment has multiple controller nodes, then the cron job can be executed at the exactly same timing in multiple nodes.
This is not ideal and can cause conflict/unsuitability.

Steps to reproduce
==================
* Deploy overcloud node with multiple controller nodes and GlanceCacheEnabled: true
* Check cron jobs in glance_api_cron container

Expected result
===============
* The glance-cache-cleaner/pruner commands are executed at different timings

Actual result
===============
* The glance-cache-cleaner/pruner commands are executed at the exact same timing

Environment
===========
* N/A

Logs & Configs
==============
* N/A

Changed in tripleo:
importance: Undecided → Medium
assignee: nobody → Takashi Kajinami (kajinamit)
Changed in tripleo:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to tripleo-heat-templates (master)

Reviewed: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/847112
Committed: https://opendev.org/openstack/tripleo-heat-templates/commit/376a8a46c3ee898639cab4016ba150342cf83a1a
Submitter: "Zuul (22348)"
Branch: master

commit 376a8a46c3ee898639cab4016ba150342cf83a1a
Author: Takashi Kajinami <email address hidden>
Date: Wed Jun 22 09:31:01 2022 +0900

    Glance: Avoid simultaneous clean/pruner job

    This change introduces randomized delays to cache clean/pruner job
    commands so that these commands are not executed simultaneously in
    multiple nodes.

    Closes-Bug: #1979271
    Depends-on: https://review.opendev.org/846601
    Change-Id: Icf8727779de746fe85e76206291be50b77bfe6eb

Changed in tripleo:
status: In Progress → Fix Released
tags: added: wallaby-backport-potential
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to tripleo-heat-templates (stable/wallaby)

Fix proposed to branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/849827

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/tripleo-heat-templates 17.0.0

This issue was fixed in the openstack/tripleo-heat-templates 17.0.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on tripleo-heat-templates (stable/wallaby)

Change abandoned by "Takashi Kajinami <email address hidden>" on branch: stable/wallaby
Review: https://review.opendev.org/c/openstack/tripleo-heat-templates/+/849827
Reason: tripleo is being retired

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.