Bug #1856573 “Set SyslogIdentifier for container healthchecks” : Bugs : tripleo

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-12-16: Fix proposed to tripleo-ansible (master)

#1

Fix proposed to branch: master
Review: https://review.opendev.org/699215

Changed in tripleo:
status:	Triaged → In Progress

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2019-12-16: Fix proposed to paunch (master)

#2

Fix proposed to branch: master
Review: https://review.opendev.org/699216

Revision history for this message

Cédric Jeanneret (cjeanner) wrote on 2019-12-17:

#3

After some poking, it seems to work. We can get this kind of logs:

Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Compiled against library: libvirt 4.5.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using library: libvirt 4.5.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using API: QEMU 4.5.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running hypervisor: QEMU 2.12.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running against daemon: 4.5.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_compute: 8 ? 00:00:32 nova-compute
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Compiled against library: libvirt 4.5.0
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using library: libvirt 4.5.0
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using API: QEMU 4.5.0
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running hypervisor: QEMU 2.12.0
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running against daemon: 4.5.0
Dec 17 15:05:33 overcloud-0-novacompute-0 healthcheck_nova_compute: 8 ? 00:00:36 nova-compute
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Compiled against library: libvirt 4.5.0
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using library: libvirt 4.5.0
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using API: QEMU 4.5.0
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running hypervisor: QEMU 2.12.0
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running against daemon: 4.5.0
Dec 17 15:07:17 overcloud-0-novacompute-0 healthcheck_nova_compute: Error: exec failed: container_linux.go:345: starting container process caused "exec: \"/openstack/healthcheck\": stat /openstack/healthcheck: no such file or directory": OCI runtime error
Dec 17 15:08:33 overcloud-0-novacompute-0 healthcheck_nova_compute: 8 ? 00:00:40 nova-compute
Dec 17 15:08:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: FAILED
Dec 17 15:08:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Error: non zero exit code: 1: OCI runtime error

The "FAILED" is a tweaked healthcheck that just echoes "FAILED" and exists with 1 - meaning we are able to catch failed healthchecks.

I don't know if this would be enough?

After some poking, it seems to work. We can get this kind of logs:

Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Compiled against library: libvirt 4.5.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using library: libvirt 4.5.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using API: QEMU 4.5.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running hypervisor: QEMU 2.12.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running against daemon: 4.5.0
Dec 17 15:03:13 overcloud-0-novacompute-0 healthcheck_nova_compute: 8 ?        00:00:32 nova-compute
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Compiled against library: libvirt 4.5.0
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using library: libvirt 4.5.0
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using API: QEMU 4.5.0
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running hypervisor: QEMU 2.12.0
Dec 17 15:04:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running against daemon: 4.5.0
Dec 17 15:05:33 overcloud-0-novacompute-0 healthcheck_nova_compute: 8 ?        00:00:36 nova-compute
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Compiled against library: libvirt 4.5.0
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using library: libvirt 4.5.0
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Using API: QEMU 4.5.0
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running hypervisor: QEMU 2.12.0
Dec 17 15:06:07 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Running against daemon: 4.5.0
Dec 17 15:07:17 overcloud-0-novacompute-0 healthcheck_nova_compute: Error: exec failed: container_linux.go:345: starting container process caused "exec: \"/openstack/healthcheck\": stat /openstack/healthcheck: no such file or directory": OCI runtime error
Dec 17 15:08:33 overcloud-0-novacompute-0 healthcheck_nova_compute: 8 ?        00:00:40 nova-compute
Dec 17 15:08:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: FAILED
Dec 17 15:08:33 overcloud-0-novacompute-0 healthcheck_nova_libvirt: Error: non zero exit code: 1: OCI runtime error

The "FAILED" is a tweaked healthcheck that just echoes "FAILED" and exists with 1 - meaning we are able to catch failed healthchecks.

I don't know if this would be enough?

Revision history for this message

Martin Mágr (mmagr) wrote on 2019-12-20:

#4

So I see the error message on the next line. It is not ideal, but I think we can manage to fetch failure _and_ the respective error message for the failure. Thanks a lot for the effort. ... but we still have to do something with https://review.opendev.org/#/c/695859

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-02: Fix merged to tripleo-ansible (master)

#5

Reviewed: https://review.opendev.org/699215
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=49858c5265310c8cdd1694ae389965370fd97abb
Submitter: Zuul
Branch: master

commit 49858c5265310c8cdd1694ae389965370fd97abb
Author: Cédric Jeanneret <email address hidden>
Date: Mon Dec 16 15:27:57 2019 +0100

Add SyslogIdenfier to healthcheck systemd unit

Adding this new field will allow to filter all healthcheck logs using
the Idenfier value.

For instance, using journalctl, you would be able to run this:
`journalctl -t healthcheck_collectd'

It will also allow to get a dedicated file out of (r)syslog if needed.

Change-Id: Icdc5caf4cedc46291a807c39c0a31c74955a4a74
Closes-Bug: #1856573

Changed in tripleo:
status:	In Progress → Fix Released

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-03: Fix merged to paunch (master)

#6

Reviewed: https://review.opendev.org/699216
Committed: https://git.openstack.org/cgit/openstack/paunch/commit/?id=505dc92a63292c763b274e8dd02d852bbe0ace8b
Submitter: Zuul
Branch: master

commit 505dc92a63292c763b274e8dd02d852bbe0ace8b
Author: Cédric Jeanneret <email address hidden>
Date: Mon Dec 16 15:37:57 2019 +0100

Add SyslogIdenfier to healthcheck systemd unit

Adding this new field will allow to filter all healthcheck logs using
the Idenfier value.

For instance, using journalctl, you would be able to run this:
`journalctl -t healthcheck_collectd'

It will also allow to get a dedicated file out of (r)syslog if needed.

This is the reflection of Icdc5caf4cedc46291a807c39c0a31c74955a4a74

Change-Id: I6861baa287f2a8288b87be26aacecbcc061cd96f
Closes-Bug: #1856573

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-06: Fix proposed to paunch (stable/train)

#7

Fix proposed to branch: stable/train
Review: https://review.opendev.org/701186

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-06: Fix merged to paunch (stable/train)

#8

Reviewed: https://review.opendev.org/701186
Committed: https://git.openstack.org/cgit/openstack/paunch/commit/?id=e0566017a52f52f0c41f62ade60cb0bdf1ee1197
Submitter: Zuul
Branch: stable/train

commit e0566017a52f52f0c41f62ade60cb0bdf1ee1197
Author: Cédric Jeanneret <email address hidden>
Date: Mon Dec 16 15:37:57 2019 +0100

Add SyslogIdenfier to healthcheck systemd unit

Adding this new field will allow to filter all healthcheck logs using
the Idenfier value.

For instance, using journalctl, you would be able to run this:
`journalctl -t healthcheck_collectd'

It will also allow to get a dedicated file out of (r)syslog if needed.

This is the reflection of Icdc5caf4cedc46291a807c39c0a31c74955a4a74

    Change-Id: I6861baa287f2a8288b87be26aacecbcc061cd96f
    Closes-Bug: #1856573
    (cherry picked from commit 505dc92a63292c763b274e8dd02d852bbe0ace8b)

tags:

added: in-stable-train

Cédric Jeanneret (cjeanner) on 2020-01-08

tags:

added: stein-backport-potential

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-08: Fix proposed to paunch (stable/stein)

#9

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/701523

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-08: Fix merged to paunch (stable/stein)

#10

Reviewed: https://review.opendev.org/701523
Committed: https://git.openstack.org/cgit/openstack/paunch/commit/?id=3c38fe60011d9156ec71ec4dae78ebc1f1b88a99
Submitter: Zuul
Branch: stable/stein

commit 3c38fe60011d9156ec71ec4dae78ebc1f1b88a99
Author: Cédric Jeanneret <email address hidden>
Date: Mon Dec 16 15:37:57 2019 +0100

Add SyslogIdenfier to healthcheck systemd unit

Adding this new field will allow to filter all healthcheck logs using
the Idenfier value.

For instance, using journalctl, you would be able to run this:
`journalctl -t healthcheck_collectd'

It will also allow to get a dedicated file out of (r)syslog if needed.

This is the reflection of Icdc5caf4cedc46291a807c39c0a31c74955a4a74

    Change-Id: I6861baa287f2a8288b87be26aacecbcc061cd96f
    Closes-Bug: #1856573
    (cherry picked from commit 505dc92a63292c763b274e8dd02d852bbe0ace8b)
    (cherry picked from commit e0566017a52f52f0c41f62ade60cb0bdf1ee1197)

tags:

added: in-stable-stein

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-09: Fix proposed to tripleo-ansible (stable/train)

#11

Fix proposed to branch: stable/train
Review: https://review.opendev.org/701826

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-09:

#12

Fix proposed to branch: stable/train
Review: https://review.opendev.org/701849

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-09: Change abandoned on tripleo-ansible (stable/train)

#13

Change abandoned by Emilien Macchi (<email address hidden>) on branch: stable/train
Review: https://review.opendev.org/701826
Reason: https://review.opendev.org/701849

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-01-30: Fix included in openstack/tripleo-ansible 1.1.0

#14

This issue was fixed in the openstack/tripleo-ansible 1.1.0 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-02-07: Fix merged to tripleo-ansible (stable/train)

#15

Download full text (17.9 KiB)

Reviewed: https://review.opendev.org/701849
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=ec4351c56646122f295a4fbdcba65abecc28df26
Submitter: Zuul
Branch: stable/train

commit ec4351c56646122f295a4fbdcba65abecc28df26
Author: Emilien Macchi <email address hidden>
Date: Wed Oct 2 12:01:28 2019 -0400

[SQUASH] backport tripleo-container-manage to stable/train

This is a squash of 32 commits to facilitate the backport of
tripleo-container-manage and its dependencies.

Introduce tripleo-container-manage role

    This is a first ieration of the role, but there is still a long TODO,
    that will come later in separated patches:
    - Add molecule testing
    - In podman.yaml, add cpuset_cpus with parity of what is in paunch
    - Remove containers that are:
      - managed by tripleo-ansible (using the container_label flag)
      - not in the container-startup-config
    - Print stdout when containers start as it was done with paunch

Story: 2006732
Task: 37165

Co-Authored-By: Kevin Carter <email address hidden>
Co-Authored-By: Alex Schultz <email address hidden>

    Depends-On: https://review.opendev.org/#/c/702144/
    Change-Id: I2f88caa8e1c230dfe846a8a0dd9f939b98992cd5
    (cherry picked from commit a191a2d6001068c77fa6e4a97c12574c59341864)

tripleo-container-manage: set some defaults

Set defaults that are needed to use the role outside of THT more easily.

Change-Id: Id67cf06c85a2a6b50e6494b1a66f534ccb06c4a7
(cherry picked from commit 609d7895a1ff4a216abffb7c46c16c49e8abaf4e)

Move the filters plugin to the core plugins location

    This change is a workaround for a zuul issue which moves the
    nested ansible role plugin to the core plugins directory so
    that it is not creating a gate conflict.

    Change-Id: I9f959803381063502b4d15980b14c3416ffa153f
    Signed-off-by: Kevin Carter <email address hidden>
    (cherry picked from commit e2719131dbdff4fc5fdee32e84ccb4c7655be29b)

Revert "Workaround for ansible-lint installation failure"

    Backport note: this is a second backport of the same patch
    since now it includes the change in tripleo-container-manage
    role that is being backported to stable/train.

    This reverts the disabling of the ansible-lint test from
    commit cffd4fc9d41b15c31610dee4abd8786916f5933b and updates
    ansible-lint to the fixed version.

    Included are fixes for ansible-lint test failures which
    got merged as part of I2f88caa8e1c230dfe846a8a0dd9f939b98992cd5
    while the lint test was disabled.

Change-Id: I37100f5e1764a5cd2cb8df82ae963e673ca0a8da
(cherry picked from commit 28e105c05689f3b8fc046758f9218e9ffab51360)

tripleo-container-manage: few improvements

    - Add and use variables to make the role more flexible:
      tripleo_container_manage_config,
      tripleo_container_manage_config_id
      tripleo_container_manage_debug
      tripleo_container_manage_config_pattern (and rename hashed_files var)

With these vars, the role can pretty much be used outside of TripleO.

- Show logs of config...

Reviewed:  https://review.opendev.org/701849
Committed: https://git.openstack.org/cgit/openstack/tripleo-ansible/commit/?id=ec4351c56646122f295a4fbdcba65abecc28df26
Submitter: Zuul
Branch:    stable/train

commit ec4351c56646122f295a4fbdcba65abecc28df26
Author: Emilien Macchi <emilien@redhat.com>
Date:   Wed Oct 2 12:01:28 2019 -0400

[SQUASH] backport tripleo-container-manage to stable/train
    
    This is a squash of 32 commits to facilitate the backport of
    tripleo-container-manage and its dependencies.
    
    Introduce tripleo-container-manage role
    
    This is a first ieration of the role, but there is still a long TODO,
    that will come later in separated patches:
    - Add molecule testing
    - In podman.yaml, add cpuset_cpus with parity of what is in paunch
    - Remove containers that are:
      - managed by tripleo-ansible (using the container_label flag)
      - not in the container-startup-config
    - Print stdout when containers start as it was done with paunch
    
    Story: 2006732
    Task: 37165
    
    Co-Authored-By: Kevin Carter <kecarter@redhat.com>
    Co-Authored-By: Alex Schultz <aschultz@redhat.com>
    
    Depends-On: https://review.opendev.org/#/c/702144/
    Change-Id: I2f88caa8e1c230dfe846a8a0dd9f939b98992cd5
    (cherry picked from commit a191a2d6001068c77fa6e4a97c12574c59341864)
    
    tripleo-container-manage: set some defaults
    
    Set defaults that are needed to use the role outside of THT more easily.
    
    Change-Id: Id67cf06c85a2a6b50e6494b1a66f534ccb06c4a7
    (cherry picked from commit 609d7895a1ff4a216abffb7c46c16c49e8abaf4e)
    
    Move the filters plugin to the core plugins location
    
    This change is a workaround for a zuul issue which moves the
    nested ansible role plugin to the core plugins directory so
    that it is not creating a gate conflict.
    
    Change-Id: I9f959803381063502b4d15980b14c3416ffa153f
    Signed-off-by: Kevin Carter <kecarter@redhat.com>
    (cherry picked from commit e2719131dbdff4fc5fdee32e84ccb4c7655be29b)
    
    Revert "Workaround for ansible-lint installation failure"
    
    Backport note: this is a second backport of the same patch
    since now it includes the change in tripleo-container-manage
    role that is being backported to stable/train.
    
    This reverts the disabling of the ansible-lint test from
    commit cffd4fc9d41b15c31610dee4abd8786916f5933b and updates
    ansible-lint to the fixed version.
    
    Included are fixes for ansible-lint test failures which
    got merged as part of I2f88caa8e1c230dfe846a8a0dd9f939b98992cd5
    while the lint test was disabled.
    
    Change-Id: I37100f5e1764a5cd2cb8df82ae963e673ca0a8da
    (cherry picked from commit 28e105c05689f3b8fc046758f9218e9ffab51360)
    
    tripleo-container-manage: few improvements
    
    - Add and use variables to make the role more flexible:
      tripleo_container_manage_config,
      tripleo_container_manage_config_id
      tripleo_container_manage_debug
      tripleo_container_manage_config_pattern (and rename hashed_files var)
    
      With these vars, the role can pretty much be used outside of TripleO.
    
    - Show logs of config data generation if debug is enabled
    
    - Do not run the "podman exec" tasks in check mode
    
    - Remove the dependency on the "step" variable
    
    Change-Id: I28ee31b723f27c392f880676aaae9368906cf45f
    (cherry picked from commit 9c69840640261783cf7940c3e66a96608ba0ff77)
    
    Adds new molecule testing for tripleo-container-manage
    
    This test ensure the "create" part is working fine with an easy and
    simple container. The container-create and default scenario have been
    consoldated so its running one complete test for now.
    
    Change-Id: I9139c7b63c15739a1a95d913acb1128af299ce97
    Co-Authored-By: Emilien Macchi <emilien@redhat.com>
    Signed-off-by: Kevin Carter <kecarter@redhat.com>
    (cherry picked from commit 414f47cc321353816b8a80c14c2f4b2b81f5e09e)
    
    tripleo-container-manage: add check tasks to the molecule playbook
    
    Verify that the "fedora" container exists and has the right infos.
    
    Change-Id: If78a254564cff502b49769b0b401d2efdac8cb23
    (cherry picked from commit c20ab4201568cd70f232ac3ed39a073304776919)
    
    Molecule job for testing plugins and modules
    
    and remove ansible_facts from podman_container_info module
    Change-Id: I0c768bc6168363fa3758562f9f053aa9ab85236b
    
    (cherry picked from commit 2d420827378d43d79def26fb254cd33b7a6e89b5)
    
    tripleo-container-manage - first support for idempotency
    
    - Implements a new helper to figure out if the existing containers on
      the hosts need to be removed (and re-created later).
      The helper will remove the container if:
        - the container is managed by tripleo_ansible (+ other conditions later)
        - a running container isn't in the config
        - the container has no config_data Label
        - the config_data changed for the container
    
    - Restart the systemd service for the container if the podman_container
      module reported as changed (note the podman_container isn't yet
      idempotent but we're working on it in a separated patch)
    
    - Fix the healthcheck & timer systemd files to be attached to the right
      service (with tripleo_ prefix)
    
    Story: #2006732
    Task: #37163
    Change-Id: I5081c918b47dcb9f3629a3649fdf33d17668c1ff
    (cherry picked from commit 7e41e0642de46e9c410278afc4ad150ce551fe88)
    
    tripleo-container-manage: include_tasks to speed things up
    
    Change-Id: I79d48fef7552f72c8fc0ebbad35d98cfc618b114
    (cherry picked from commit 57411934b7a1a5486c001699c9ed4d8d486cbd7d)
    
    tripleo-container-manage: introduce concurrency
    
    Co-Authored-By: Alex Schultz <aschultz@redhat.com>
    Change-Id: I1e5e941558c492b050e4db542703e322707dbbbd
    (cherry picked from commit 2f8f0fc027ee2a03eef812d452746986ac7e57be)
    
    tripleo-container-manage: fix log_opt
    
    Put the right variables to have proper logging.
    
    Change-Id: I60a14721ed978dddbebfd28f830fb2c50f326ce0
    (cherry picked from commit e7f71c352e306bceeec97d734cd7228c75b12522)
    
    tripleo-container-manage: some nits fixed
    
    Change-Id: I4006db3f5e3092aefeb8a0e50819dda11931630b
    (cherry picked from commit 73e49888f7daba4a348aefaa0631fc2cb0ff6cbe)
    
    tripleo-container-manage: fix exec check
    
    Fix the task which check if the container where the exec happens is
    actually running.
    
    Also podman_containers needs to be refreshed at every start_order to get
    new container informations status.
    So moving podman_containers from main, and putting it where we need it:
    - before processing the list of containers to delete
    - when start_order playbook starts
    
    Co-Authored-By: Sagi Shnaidman <sshnaidm@redhat.com>
    Change-Id: I2afb71288208c8b97763caa832c94e06e1b9457c
    (cherry picked from commit e68d4b42a222e97f6c0263398ac739866e5969b4)
    
    tripleo-container-manage: port paunch-services
    
    paunch-services used to be useful for container start/stop ordering,
    when on the host some containers are managed by Pacemaker and some
    others by Paunch.
    
    We need to keep that feature so we are now porting these scripts and
    services into tripleo-ansible.
    
    These files where managed by Paunch:
    %{_libexecdir}/paunch-container-shutdown
    %{_libexecdir}/paunch-start-podman-container
    %{_unitdir}/paunch-container-shutdown.service
    %{_presetdir}/91-paunch-container-shutdown.preset
    %{_unitdir}/netns-placeholder.service
    %{_presetdir}/91-netns-placeholder.preset
    
    Now we handle them via Ansible now, and cleanup the Paunch version.
    It creates the exact same files from:
    https://github.com/rdo-packages/paunch-distgit
    
    This feature is disabled by default and will be explicitely enabled by
    THT later. Molecule tests enable it though for testing coverage.
    
    Story: 2006732
    Task: 37382
    Change-Id: I4f79429baab50bc0199fb65fe84641908d83935d
    (cherry picked from commit 6e76f444df00af9ab60de1a5c7a7c8fa1d7eed61)
    
    tripleo-container-manage: use async
    
    - Add a new filter 'haskey' which returns container data with a specific
      config key. The filter takes a list of dicts and returns the dicts
      which have a certain key given in parameter with 'attribute'.
      If 'reverse' is set to True, the returned list won't contains dicts
      which have the 'attribute' value.
      If 'any' is set to True, the returned list will match any value in the
      list of 'value' parameter which has to be a list.
    
    - Make the exec and create playbook using ansible "async".
      (WIP is to re-add the podman_container.changed in systemd playbook).
    
    Note: we don't manage Systemd resources with async, since it exposes
    race conditions at the systemd level.
    
    Change-Id: Ice92cd5f90039e685c171e9035f929349a67ff2c
    (cherry picked from commit e26f817597cf9f5a08d2eb6d559d1f2cc1b7d031)
    
    tripleo-container-manage: fix duplicated loop vars
    
    So we don't have the Ansible warning saying the loop var is already used
    for another loop.
    
    Change-Id: Ie28857cb712d2133aad4a67ae6942fcbbb1a6aee
    (cherry picked from commit 80e0476f78c654f3bd563efb62a2acdc96e6df86)
    
    tripleo-container-manage: skip some tasks in check mode
    
    Change-Id: I5e68fdb2d872c741f00a1a24bac33112ba630f69
    (cherry picked from commit f506dd6994c398669ae0b20d36ecce18d0ffbc4b)
    
    tripleo-container-manage: add no_log to podman create
    
    So we don't leak confidential informations like passwords and others in
    the logs.
    Note that if debug is enabled, the infos are displayed in the ansible
    logs.
    
    Change-Id: I8473c9118dbce2b04eeb2c01bcc1e55232325a67
    (cherry picked from commit ac8cff4ebbbb6964255b2eb45b9cfbfd5394ef65)
    
    Improve molecule tests for tripleo-container-manage (systemd)
    
    Change-Id: Id14ad4876bdf7dcc979d5ecf4fe2b4751c635b88
    (cherry picked from commit 484faa7e8016781afa86c099032b511bf90ef4fb)
    
    fix typo
    
    Change-Id: I0a970ae73f01c6ff181a67367cf668091748fc9c
    Signed-off-by: Kevin Carter <kecarter@redhat.com>
    (cherry picked from commit 3720b41a7f10e937c265a710071ca03fedf2ec70)
    
    tripleo-container-manage: restart systemd service if container changed
    
    If a podman_container resource changed, it'll be added to a fact, that
    later will be used when it comes to figure our if a systemd service
    needs to be restarted or not.
    
    It introduces a new filter: list_of_keys.
    This filter takes in input a list of dictionaries and for each of them
    it will add the key to list_of_keys and returns it.
    
    Change-Id: I0285b006c015f6cd223615ebdad52286f7683a87
    (cherry picked from commit 0a037c65f7804316d4b1c3f1753d4c35bc8fbfbb)
    
    tripleo-container-manage: some improvements
    
    - Improve logging in some tasks, to display what container is managed
    - Use no_log for the tasks which leak all infos about containers
      (config_data). If debug is enabled, the logs will show all the
      config_data.
    - Move the /etc/sysconfig/podman_drop_in tasks into shutdown.yaml so it
      runs once and it's staying close to the other tasks related to why we
      have this file (manage shutdown/start order when reboot).
    
    Change-Id: I56896f92d58db900fd2ea06281d89f75e8d53a17
    (cherry picked from commit 6743f7586343d826c068f82b13388ca170922810)
    
    tripleo-container-manage: create ansible-managed dropin file
    
    If a deployment has containers managed by tripleo-ansible and not
    paunch, and an operator would try to run paunch on the host, we want to
    send a warning because paunch will remove all containers managed by
    tripleo-ansible and then redeploy the containers with managed_by=paunch.
    
    We could change the default of manage_by in Paunch (currently
    managed_by=paunch) to something else e.g. tripleo, which would also be
    the managed_by of tripleo-ansible; so both paunch and tripleo-ansible
    could be used on the same host. However, we decided that when
    tripleo-ansible is used to deploy the containers, paunch could not be
    used anymore on that host.
    
    Therefore, we create a file that will be checked by paunch and if
    present, paunch CLI will show a warning.
    
    Change-Id: I722cb8faa3b7eee81b418da83451bf802351dd79
    (cherry picked from commit db1b13c962fc74ec7be9981d8e6f6af237afd3f5)
    
    tripleo-container-manage: remove no_log for create/exec tasks
    
    1. Remove no_log for the create and exec tasks. It's always useful to
       see what is being run.
    
    2. Enable podman_container debug, so we see what commands are being run.
    
    Change-Id: If728788293dd64622cf95da840b60e271197e9a0
    (cherry picked from commit 89e4adf2ae816244cc9b7b48e86b7ca5511f07d3)
    
    Add missing ExecReload in container service unit file
    
    It may happen that we want to just reload the container. Before this
    patch, it was a "stop and start", while podman has the "podman kill"
    available, accepting the HUP signal.
    
    Doing so allows other automated tools to actually just "reload" the
    container as we would do for a standard service.
    
    Change-Id: I35eff80f7637b013d3a1a831289ec9b1e0f81431
    (cherry picked from commit 2a2bed6f5cb9989d3544918c7e151ba168c7cd6f)
    
    Introduce unit tests structure for tripleo-ansible filters
    
    Change-Id: Ie2fea14d2cbfb2c0b78cdc3064df0a558fa28a4c
    (cherry picked from commit f90a6d42b3f5a55d5269b2d88b30e6455f6b3287)
    
    Fix case in filters when Labels is None
    
    When Labels is None it's not managed by tripleo container,
    then skip it.
    Change-Id: Ib82c4d28c462abb3f1a5ccb7d5137ec6059b2665
    
    (cherry picked from commit 1acc95211d018033858f15e33dd063fca1991d3b)
    
    tripleo-container-manage: include more arguments for container exec
    
    Container execs can be more complex than just a user and a command.
    It can also use --env and --privileged.
    
    - container_exec_cmd is a new filter that will help to build the
      container exec command from the container data.
    - list_or_dict_arg is an utility taken from openstack/paunch which
      allows to build a command and its arguments with list or dict data.
    
    This patch will allow the container exec to work fine when they have
    environment variables, like it's the case for Keystone bootstrap.
    
    Change-Id: I15e0b518936b37e26799dbda9677a248cf17ff3c
    (cherry picked from commit 55d5363b4a1b4d1eceda9522c327f193d133c05b)
    
    Add SyslogIdenfier to healthcheck systemd unit
    
    Adding this new field will allow to filter all healthcheck logs using
    the Idenfier value.
    
    For instance, using journalctl, you would be able to run this:
    `journalctl -t healthcheck_collectd'
    
    It will also allow to get a dedicated file out of (r)syslog if needed.
    
    Change-Id: Icdc5caf4cedc46291a807c39c0a31c74955a4a74
    Closes-Bug: #1856573
    (cherry picked from commit 49858c5265310c8cdd1694ae389965370fd97abb)
    
    Allow to run tripleo-container-manage in check mode
    
    Change-Id: I3350a43805b4a148f64de393716c26b0158fcff4
    (cherry picked from commit 6513a4bed825e73454333fa88ac754af90d01b2d)
    
    tripleo-container-manage: fix config_data in Config/Labels
    
    config_data should not contain the container name, just the actual
    config data which is the value of the container_data dict.
    
    It removes the add of start_order into the compare, this isn't necessary
    and breaks idempotency for containers which don't have start_order in
    their config_data.
    Also adds more unit tests to cover all situations handled by the filter.
    
    Change-Id: I0b64bf34c8f7498128b3b1fceb7c727f8544cec6
    (cherry picked from commit 19c5d7e77e0ed7b3661f8f8663b399394c1e12ec)
    
    tripleo-container-manage: search container configs recursively
    
    If the user doesn't override the default value for
    tripleo_container_manage_config, which is set to
    '/var/lib/tripleo-config/', to something deeper, we want Ansible to
    recursively search for the container config until it finds it.
    
    Change-Id: I7d23fec91ffb813f0ab6f11b85d811ef3897f9e0
    (cherry picked from commit 205f7c9b0c5bc232cac393ab38364acb01d3c40f)
    
    Idempotency for podman_container
    
    Introduce partial idepmpotency for running podman_container
    
    - Replace all options by their defaults it they're not set
    - Force lower case on all podman inspection data
    - Add a class with methods for every parameter to check its value
    and compare it with module argument
    - Add check_mode support and podman_actions
    
    Change-Id: I1ae93dff1e10a1a696bb171996a75a0db6c34fa3
    (cherry picked from commit 1212544a283f99cc4f233c645f92bb805db01e4f)
    
    Fix case where there're no effective caps
    
    When no effective caps in container, it should be list, not None
    Change-Id: I5412edc844ad43223c4b3bda35662a7f7ee43f3a
    
    (cherry picked from commit 3a6690c904a5ed5790f3d1ae8c82134f22d277b4)
    
    Add idempotency for networks parameter in podman_container
    
    Change-Id: Ib7235d1cb64ad0f42e7a0201008536e1a35bd696
    (cherry picked from commit 25f8abbc5a6195d0f72b9fecca8b0728991a4772)
    
    use version parsing from distutils
    
    use LooseVersion for version comparisons
    
    Change-Id: I609920a96c725c49f1623f60f8295d89ae4f3141
    (cherry picked from commit 8c83219fbbc4078bfb8e666bd3186c22e809437a)
    
    Improve idempotency for podman containers module
    
    Fix some pep8 issues
    Change-Id: If4233e57edeec10ccac965d61c78f30688cd5531
    
    (cherry picked from commit 0609c16d10795bee6bb2a48c023d4272b35d7db5)
    
    Improve idempotency for podman container module
    
    Strip all registry values from "image" parameter in input.
    Add this case to test.
    
    Change-Id: I78656e48cf85a1a39f873ee40193765eecf02c56
    (cherry picked from commit 87d9e9d9a6e6966d71bd574466b2128b58ff096c)

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-02-18: Fix included in openstack/paunch 6.0.1

#16

This issue was fixed in the openstack/paunch 6.0.1 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2020-04-09: Fix included in openstack/tripleo-ansible 0.5.0

#17

This issue was fixed in the openstack/tripleo-ansible 0.5.0 release.

Revision history for this message

OpenStack Infra (hudson-openstack) wrote on 2021-07-23: Fix included in openstack/paunch stein-eol

#18

This issue was fixed in the openstack/paunch stein-eol release.

tripleo

Set SyslogIdentifier for container healthchecks

Bug Description

Other bug subscribers

Remote bug watches