a11y: tripleo output should be easier to read

[Sorin Sbarnea]

Output of tripleo command should be easier to read for the users, whenever this is console log or other logs.

Improving readability of output would include steps like:
- avoiding ansible tasks spam (rsync, cating html or json files to console)
- be verbose on failures but not on successes (no_log: result.rc == 0)
- enabling colored output on CI jobs
- enabling coloring on archived logs (html version in addition to plaintext)
- include bookmarkable line number in html logs (ease sharing links)
- collapsing support for timestamp column (easy to read on smaller screens)
- avoid use of ignore_errors in ansible tasks that are likely to return error codes that are not fatal (they would appear as red in console, even if that would not be a real error)
- print additional debug data on console on *failed* commands (or ansible tasks) instead of returning just a simple error ( cmd || tail pattern), this saves a lot of debugging time.

Change Request related to this bug:
* https://review.openstack.org/#/q/topic:bug/1787912+(status:open+OR+status:merged)
* https://review.rdoproject.org/r/15914

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/595576

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/600039

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/600403

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.openstack.org/590748
Reason: Abandoning since I added these variables to tripleoci repository and at least on Jenkins they work well.

When we will have ANSI support in zuul, we can revive it.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Juan Antonio Osorio Robles (<email address hidden>) on branch: master
Review: https://review.openstack.org/599358
Reason: Purging the gate to free up resources and address the timeout issues

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/599358
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=4bd7ad3187a11f46cf1c80966e6a9a3408fd866c
Submitter: Zuul
Branch: master

commit 4bd7ad3187a11f46cf1c80966e6a9a3408fd866c
Author: Sorin Sbarnea <email address hidden>
Date: Mon Sep 3 12:37:13 2018 +0100

    Make setup repo task output visible when errored

    Improves UX by displaying the error on console and avoiding the
    need to for the user to dig in another file.

    Partial-Bug: #1787912
    Change-Id: I2ae7e972e3a67e47d820030ad3eb23c05fb8b892

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/617118

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/617363

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/617363
Committed: https://git.openstack.org/cgit/openstack/python-tripleoclient/commit/?id=cc9d7247b62d291a209746859c785408146e0445
Submitter: Zuul
Branch: master

commit cc9d7247b62d291a209746859c785408146e0445
Author: Sorin Sbarnea <email address hidden>
Date: Mon Nov 12 17:04:38 2018 +0000

    Avoid printing b'' across logged output

    Converts stdout input from bytes to unicode to avoid printing lines
    as b'<line>' on python3.

    Address runtime warning related to deprecation of iter().

    Stop using bugsize=1 as this is likely to break unicode encodings.

    Usually an implementation using context managers would look much better
    but due to several bugs related to only partial support of context
    managers in py27 we cannot use them here.

    Tested with py27 and py35-py37.

    See: http://logs.openstack.org/37/616937/9/check/tripleo-ci-fedora-28-standalone/4a015f3/logs/undercloud/home/zuul/standalone_deploy.log.txt.gz#_2018-11-12_16_15_51

    Change-Id: I00eaa508a6413c05c99beaca49e70c635a9116cc
    Partial-Bug: #1787912

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/623199

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/623199
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=9c196b00391e5ed5fb0d85913433efc1a2b010c3
Submitter: Zuul
Branch: master

commit 9c196b00391e5ed5fb0d85913433efc1a2b010c3
Author: Sorin Sbarnea <email address hidden>
Date: Thu Dec 6 12:20:14 2018 +0000

    Use ansible debug stdout_callback on overcloud-deploy

    Makes output more human friendly by avoiding collapsing stdett/stdout
    lines as JSON.

    Example: https://logs.rdoproject.org/33/614633/9/openstack-check/tripleo-ci-centos-7-ovb-3ctlr_1comp-featureset001/f391bee/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz#_2018-12-06_09_34_21

    We already adopted debug callback in other areas but it was not used
    on overcloud-deploy.

    Change-Id: Ibfcfa1858fd1326e88844516090c40f40dba21ed
    Partial-Bug: #1787912

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/625621

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/625632

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/625621
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=0c532f715b2a25f072bfa308709fdb784873500e
Submitter: Zuul
Branch: master

commit 0c532f715b2a25f072bfa308709fdb784873500e
Author: Sorin Sbarnea <email address hidden>
Date: Mon Dec 17 15:23:16 2018 +0000

    Makes reproducer output more human friendly (readable)

    Use Ansible debug output callback unless user overrides it with his
    own value. Same output is prefered on CI.

    Change-Id: Ibd284dea867c002698238ec66a328f4267bd49e9
    Related-Bug: 1787912

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/600403
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=55f3bfecac5e480a20bbdd48b462f6de1ca9466e
Submitter: Zuul
Branch: master

commit 55f3bfecac5e480a20bbdd48b462f6de1ca9466e
Author: Sorin Sbarnea <email address hidden>
Date: Thu Sep 6 11:15:25 2018 +0100

    Improve output of Verify Sphinx build task

    Avoid repeated output of HTML content on console and
    marks the task as not changed in ansible.

    A grep failure would be easy to investigate as the file is collected.

    Change-Id: I7520e8f3b5c01f39affeac398aeaeffe6dfdb6cf
    Partial-Bug: #1787912

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/625632
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart-extras/commit/?id=74e8e614f6c3e51bdf0351db99083db7eda61b49
Submitter: Zuul
Branch: master

commit 74e8e614f6c3e51bdf0351db99083db7eda61b49
Author: Sorin Sbarnea <email address hidden>
Date: Mon Dec 17 15:45:13 2018 +0000

    Correct reproducer dependency detection and adds missing git

    Use of exit instead of return in the check dependecy function prevented
    us to list all missing dependencies before exiting.

    Avoid case where user needs to run the script 6 or more times in order
    to install missing dependencies.

    Adds git which was a missing dependency.

    Change-Id: I66f883bbd817ce809261b7408c945e527ab72549
    Partial-Bug: 1787912

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.openstack.org/599594
Reason: failed to quorum

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/645351

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/645166
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=b8c93b903eef4732e60627c018c70a4c7393977c
Submitter: Zuul
Branch: master

commit b8c93b903eef4732e60627c018c70a4c7393977c
Author: Sorin Sbarnea <email address hidden>
Date: Thu Mar 21 12:53:16 2019 +0000

    Improve tempest plugins installation

    Instead of ignoring installation failure, new code would fail if
    tempest plugins cannot be installed and also work if there are
    no plugins specified.

    Change-Id: Ic5ea1bf75cd30f8af6d9a3586a0073ff5c54cb27
    Related-Bug: #1787912

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.opendev.org/645351

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/617118
Committed: https://git.openstack.org/cgit/openstack/tripleo-quickstart/commit/?id=d3ecde97750eb0dd8c231702fc44a1038e4d5a15
Submitter: Zuul
Branch: master

commit d3ecde97750eb0dd8c231702fc44a1038e4d5a15
Author: Sorin Sbarnea <email address hidden>
Date: Sat Nov 10 14:39:44 2018 +0000

    Avoids use of ignore_errors while loading OS specific variables

    Implements pattern described in:
    https://github.com/ansible/ansible/issues/38227

    Example of confusing output before this change:
    https://ci.centos.org/job/tripleo-quickstart-extras-gate-master-delorean-full-featureset052/1604/consoleFull

    Change-Id: I923ab325df699abe2f6d69b5b484b154558e423a
    Partial-Bug: #1787912

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.opendev.org/600039

[OpenStack Infra (hudson-openstack)]

Change abandoned by Sorin Sbarnea (<email address hidden>) on branch: master
Review: https://review.opendev.org/595576