Logs collection playbook is stuck on [collect-logs : Archive big collected files] task

[Sagi (Sergey) Shnaidman]

Log collection playbook is stuck on task [collect-logs : Archive big collected files]
https://logserver.rdoproject.org/71/680571/17/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/8a574d2/logs/quickstart_collect_logs.log.txt.gz

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/706797

[Sorin Sbarnea (ssbarnea)]

I do not see any proof that this is really stuck nor that this task takes an unreasonable amount of time.

Before considering other options we need to identify when/how this happens. A first step would be to limit the maximum duration allowed for running this task.

Maybe this task fails for other reasons, like getting out of disk space during compression. In either case we need to acquire data regarding the nature of the issue.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/708290

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.opendev.org/708362

[OpenStack Infra (hudson-openstack)]

Change abandoned by wes hayutin (<email address hidden>) on branch: master
Review: https://review.opendev.org/708290

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/708520

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/706797
Committed: https://git.openstack.org/cgit/openstack/ansible-role-collect-logs/commit/?id=e2bccc1bb12ddebc0b34edf48f5b56ea5533dc5f
Submitter: Zuul
Branch: master

commit e2bccc1bb12ddebc0b34edf48f5b56ea5533dc5f
Author: Sagi Shnaidman <email address hidden>
Date: Mon Feb 10 13:04:47 2020 +0200

    Enable compression option and tar on xfer prior to publish

    When collecting logs of many nodes and storing them on
    the primary nodepool node or slave it makes sense
    to compress the files, tar and send. Pretty standard
    logic there. In upstream the default remains not
    to compress prior to tar or prior to publish. There
    is a sunk time cost in compressing and uncompressing.

    Note: this removes the optional logic to compress
    based on file size as it make more sense to compress
    all the files prior to the tar and xfer to the
    primary node / slave for faster results.

    Signed off by Wes Hayutin.

    Close-Bug: #1862612
    Change-Id: I550e5c5ead7f65fba184f311a3a4b1c64a42a6d2

[OpenStack Infra (hudson-openstack)]

Change abandoned by wes hayutin (<email address hidden>) on branch: master
Review: https://review.opendev.org/708520

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.opendev.org/708362