container run error message is hard to find in the logs

[Alex Schultz]

Trying to find the root cause of a deployment failure when configuring containers is hard to find because the error message is a WARNING and not an ERROR. Error we kick out about failing a specific container is an ERROR, but the root cause message is a warning.

Example:

2019-03-13 13:07:08.697 79966 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ admin] "2019-03-13 13:05:47,450 ERROR: 84000 -- Failed running container for neutron",

But the actual problem is:

2019-03-13 13:07:08.697 79966 WARNING tripleoclient.v1.tripleo_deploy.Deploy [ admin] "2019-03-13 13:05:40,894 WARNING: 84000 -- ['/usr/bin/podman', 'run', '--user', 'root', '--name', 'container-puppet-neutron', '--env', 'PUPPET_TAGS=file,file_line,concat,augeas,cron,neutron_config,ironic_neutron_agent_config,neutron_config,neutron_api_config,neutron_plugin_ml2,neutron_config,neutron_dhcp_agent_config,neutron_config,neutron_l3_agent_config,neutron_config,neutron_agent_ovs,neutron_plugin_ml2', '--env', 'NAME=neutron', '--env', 'HOSTNAME=undercloud-0', '--env', 'NO_ARCHIVE=', '--env', 'STEP=6', '--env', 'NET_HOST=true', '--log-driver', 'json-file', '--volume', '/etc/localtime:/etc/localtime:ro', '--volume', '/tmp/tmpcwn5_jsm:/etc/config.pp:ro', '--volume', '/etc/puppet/:/tmp/puppet-etc/:ro', '--volume', '/etc/pki/ca-trust/extracted:/etc/pki/ca-trust/extracted:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.crt:/etc/pki/tls/certs/ca-bundle.crt:ro', '--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro', '--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro', '--volume', '/var/lib/config-data:/var/lib/config-data/:rw', '--volume', '/dev/log:/dev/log:rw', '--log-opt', 'path=/var/log/containers/stdouts/container-puppet-neutron.log', '--security-opt', 'label=disable', '--volume', '/usr/share/openstack-puppet/modules/:/usr/share/openstack-puppet/modules/:ro', '--volume', '/lib/modules:/lib/modules:ro', '--volume', '/run/openvswitch:/run/openvswitch:shared,z', '--entrypoint', '/var/lib/container-puppet/container-puppet.sh', '--net', 'host', '--volume', '/etc/hosts:/etc/hosts:ro', '--volume', '/var/lib/container-puppet/container-puppet.sh:/var/lib/container-puppet/container-puppet.sh:ro', '192.168.24.1:8787/rhosp15/openstack-neutron-server:latest'] run failed after error checking path \"/run/openvswitch\": stat /run/openvswitch: no such file or directory",

We should increase the logging level for the actual failure messages

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/643071

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/643071
Committed: https://git.openstack.org/cgit/openstack/tripleo-heat-templates/commit/?id=b9d382c514593d105374871a06ee5b1bb6d58b73
Submitter: Zuul
Branch: master

commit b9d382c514593d105374871a06ee5b1bb6d58b73
Author: Alex Schultz <email address hidden>
Date: Wed Mar 13 09:12:46 2019 -0600

    Increase log level for command failures

    It's hard to find the actual root cause of the container configurations
    when looking in the logs because the error messages are just WARNINGs.
    Let's raise the messages where the real error message is to an ERROR
    level.

    Change-Id: Icb7138de7f6065c758cd39cb0e433c6412442b1c
    Closes-Bug: #1819931

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/tripleo-heat-templates 10.5.0 release.