logging of tripleo-modify-image role during prepare is hard to read

Bug #1813684 reported by Steve Baker on 2019-01-29
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
High
Steve Baker

Bug Description

The tripleo-modify-image role is invoked by a playbook with -vvv, which results in the below output when shell scripts produce a lot of output. This verbose stderr would be much more readable if the yaml stdout_callback was used with a lower level of verbosity (like -v)

Example output:

2019-01-24 01:29:15,616 p=28525 u=root | TASK [tripleo-modify-image : Run yum_update.sh] ********************************
2019-01-24 01:29:15,617 p=28525 u=root | task path: /usr/share/ansible/roles/tripleo-modify-image/tasks/yum_update_buildah.yml:36
2019-01-24 01:29:15,617 p=28525 u=root | Thursday 24 January 2019 01:29:15 +0000 (0:00:01.239) 0:00:11.682 ******
2019-01-24 01:29:15,641 p=28525 u=root | [DEPRECATION WARNING]: Using tests as filters is deprecated. Instead of using
`result|exists` use `result is exists`. This feature will be removed in version
 2.9. Deprecation warnings can be disabled by setting
deprecation_warnings=False in ansible.cfg.
2019-01-24 01:29:15,655 p=28525 u=root | Using module file /usr/lib/python2.7/site-packages/ansible/modules/commands/command.py
2019-01-24 01:29:49,287 p=28525 u=root | changed: [localhost] => {
    "changed": true,
    "cmd": [
        "buildah",
        "run",
        "--volume",
        "/tmp/ansible.pg3Hcf:/tmp/yum_update.sh",
        "--volume",
        "/etc/yum.repos.d:/etc/yum.repos.d",
        "--volume",
        "/opt/gating_repo:/opt/gating_repo",
        "--user",
        "root",
        "--net",
        "host",
        "centos-binary-rsyslog-base-working-container",
        "/tmp/yum_update.sh",
        "gating-repo,delorean-current"
    ],
    "delta": "0:00:33.485473",
    "end": "2019-01-24 01:29:49.248262",
    "invocation": {
        "module_args": {
            "_raw_params": "buildah run\n --volume /tmp/ansible.pg3Hcf:/tmp/yum_update.sh\n --volume /etc/yum.repos.d:/etc/yum.repos.d\n --volume /opt/gating_repo:/opt/g
ating_repo\n --user root\n --net host\n centos-binary-rsyslog-base-working-container\n /tmp/yum_update.sh \"gating-repo,delorean-current\"",
            "_uses_shell": false,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "warn": true
        }
    },
    "rc": 0,
    "start": "2019-01-24 01:29:15.762789",
    "stderr": "+ packages_for_update=\n+ '[' -n gating-repo,delorean-current ']'\n+ command -v repoquery\n++ sort\n++ rpm -qa --qf '%{NAME}\\n'\n+ installed='acl\naudit-libs\naugeas-libs\nbasesystem\nbash\nbind-license\nbinutils\nbzip2-libs\nca-certificates\ncentos-release\ncentos-release-ceph-luminous\ncentos-release-opstools\ncentos-release-qemu-ev\ncentos-release-storage-common\ncentos-release-virt-common\nchkconfig\ncoreutils\ncpio\ncracklib\ncracklib-dicts\ncronie\ncronie-anacron\ncrontabs\ncrudini\ncryptsetup-libs\ncurl\ncyrus-sasl-lib\ndbus\ndbus-glib\ndbus-libs\ndbus-python\ndevice-mapper\ndevice-mapper-event\ndevice-mapper-event-libs\ndevice-mapper-libs\ndevice-mapper-persistent-data\ndiffutils\ndmidecode\ndracut\nelfutils-default-yama-scope\nelfutils-libelf\nelfutils-libs\nexpat\nfacter\nfile-libs\nfilesystem\nfindutils\ngawk\ngdbm\nglib2\nglibc\nglibc-common\ngmp\ngnupg2\ngobject-introspection\ngpg-pubkey\ngpg-pubkey\ngpg-pubkey\ngpg-pubkey\ngpgme\ngrep\ngzip\nhardlink\nhiera\nhostname\nhwdata\ninfo\niproute\niptables\niputils\niscsi-initiator-utils\niscsi-initiator-utils-iscsiuio\njson-c\nkeyutils-libs\nkmod\nkmod-libs\nkpartx\nkrb5-libs\nlibacl\nlibaio\nlibassuan\nlibattr\nlibblkid\nlibcap\nlibcap-ng\nlibcom_err\nlibcurl\nlibdb\nlibdb-utils\nlibestr\nlibfastjson\nlibffi\nlibgcc\nlibgcrypt\nlibgpg-error\nlibidn\nlibmnl\nlibmount\nlibnetfilter_conntrack\nlibnfnetlink\nlibpwquality\nlibselinux\nlibselinux-ruby\nlibselinux-utils\nlibsemanage\nlibsepol\nlibsmartcols\nlibssh2\nlibstdc++\nlibtasn1\nlibuser\nlibutempter\nlibuuid\nlibverto\nlibxml2\nlibxml2-python\nlibyaml\nlogrotate\nlsof\nlua\nlvm2\nlvm2-libs\nlz4\nncurses\nncurses-base\nncurses-libs\nnet-tools\nnspr\nnss\nnss-pem\nnss-softokn\nnss-softokn-freebl\nnss-sysinit\nnss-tools\nnss-util\nopenldap\nopenssl-libs\nopenstack-tripleo-common-container-base\np11-kit\np11-kit-trust\npam\npasswd\npciutils\npciutils-libs\npcre\npinentry\npkgconfig\npopt\nprocps-ng\npth\npuppet\npuppet-aodh\npuppet-apache\npuppet-archive\npuppet-auditd\npuppet-barbican\npuppet-cassandra\npuppet-ceilometer\npuppet-ceph\npuppet-certmonger\npuppet-cinder\npuppet-collectd\npuppet-concat\npuppet-congress\npuppet-contrail\npuppet-corosync\npuppet-datacat\npuppet-designate\npuppet-dns\npuppet-ec2api\npuppet-elasticsearch\npuppet-etcd\npuppet-fdio\npuppet-firewall\npuppet-fluentd\npuppet-git\npuppet-glance\npuppet-gnocchi\npuppet-haproxy\npuppet-heat\npuppet-horizon\npuppet-inifile\npuppet-ipaclient\npuppet-ironic\npuppet-java\npuppet-kafka\npuppet-keepalived\npuppet-keystone\npuppet-kibana3\npuppet-kmod\npuppet-manila\npuppet-memcached\npuppet-midonet\npuppet-mistral\npuppet-module-data\npuppet-mongodb\npuppet-mysql\npuppet-n1k-vsm\npuppet-neutron\npuppet-nova\npuppet-nssdb\npuppet-ntp\npuppet-octavia\npuppet-opendaylight\npuppet-openstack_extras\npuppet-openstacklib\npuppet-oslo\npuppet-ovn\npuppet-pacemaker\npuppet-panko\npuppet-placement\npuppet-powerdns\npuppet-qdr\npuppet-rabbitmq\npuppet-redis\npuppet
...
<snip>
...
    "stderr_lines": [
        "+ packages_for_update=",
        "+ '[' -n gating-repo,delorean-current ']'",
        "+ command -v repoquery",
        "++ sort",
        "++ rpm -qa --qf '%{NAME}\\n'",
        "+ installed='acl",
        "audit-libs",
        "augeas-libs",
        "basesystem",
        "bash",
        "bind-license",
        "binutils",
        "bzip2-libs",
        "ca-certificates",
        "centos-release",
        "centos-release-ceph-luminous",
        "centos-release-opstools",
        "centos-release-qemu-ev",
        "centos-release-storage-common",
        "centos-release-virt-common",
        "chkconfig",
        "coreutils",
        "cpio",
...
etc

Steve Baker (steve-stevebaker) wrote :

The yaml stdout_callback is available in ansible since 2.5, so I believe it is safe to make this change in stein

Fix proposed to branch: master
Review: https://review.openstack.org/633633

Changed in tripleo:
status: Triaged → In Progress

Reviewed: https://review.openstack.org/633633
Committed: https://git.openstack.org/cgit/openstack/tripleo-common/commit/?id=2a6ef6a4f9088436faf092dfee50eca4d37d19f9
Submitter: Zuul
Branch: master

commit 2a6ef6a4f9088436faf092dfee50eca4d37d19f9
Author: Steve Baker <email address hidden>
Date: Tue Jan 29 14:10:24 2019 +1300

    Use the yaml stdout_callback for better prepare logging

    Currently when the tripleo-modify-role runs a shell script which
    produces a lot of output, its logged once as a no-newline-blob, and
    again as one line per array item. Neither output is easy read, which makes
    debugging harder.

    This change switches to the yaml stdout_callback, and a verbosity
    level which only prints the script output once.

    Change-Id: I85fb8ca40e03f077147871fcd85536c0572d526e
    Closes-Bug: #1813684

Changed in tripleo:
status: In Progress → Fix Released

This issue was fixed in the openstack/tripleo-common 10.4.0 release.

Reviewed: https://review.openstack.org/645390
Committed: https://git.openstack.org/cgit/openstack/ansible-role-tripleo-modify-image/commit/?id=c0bcc3c438ef8095b51e18f43a5b7910af18caed
Submitter: Zuul
Branch: master

commit c0bcc3c438ef8095b51e18f43a5b7910af18caed
Author: Steve Baker <email address hidden>
Date: Fri Mar 22 11:30:56 2019 +1300

    Don't print commands during yum update

    The commands contain package lists, so having "set -x" makes the
    logged result very very verbose.

    Change-Id: I8ffda90eb0633b6b46e29c5af7ed2ff3571c14a2
    Related-Bug: #1813684

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers