[DIB] Need to fix dib-lint errors

[Sergey Reshetnyak]

Need to fix dib-lint errors in sahara-image-elements

 $ tox -e dib-lint | grep 'ERROR:' | wc -l
228

[Pino Toscano (ptoscano)]

Forgot to add the references when pushing their first versions:
- https://review.openstack.org/#/c/166972/ -- use $DIB_DEBUG_TRACE
- https://review.openstack.org/#/c/166975/ -- fix missing-newline in README.md

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/166975
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=51ec704d3c4ee6e3f4d9142e960f5a300fea6361
Submitter: Jenkins
Branch: master

commit 51ec704d3c4ee6e3f4d9142e960f5a300fea6361
Author: Pino Toscano <email address hidden>
Date: Mon Mar 23 20:09:25 2015 +0100

    Add newline at the end of README.md

    Follow what dib-lint requires, and make sure README.md files ends with
    newline.

    Change-Id: I06b951858580b125b101e4b49199cde46db5d515
    Partial-Bug: #1435306

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/166972
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=55a21cd7460a3fe4e8095980a2e3794632930347
Submitter: Jenkins
Branch: master

commit 55a21cd7460a3fe4e8095980a2e3794632930347
Author: Pino Toscano <email address hidden>
Date: Mon Mar 23 20:11:11 2015 +0100

    Adopt DIB_DEBUG_TRACE for tracing in elements

    Mimic commit 36b59c001c1643217449646b371df46d2cb11b91 in
    diskimage-builder, by adopting the usage of $DIB_DEBUG_TRACE to check
    whether enable tracing in scripts.

    Unlike with the diskimage-builder commit, the default is to not enable
    tracing even in the few scripts that used to unconditionally "set -x".

    Enabling tracing can be done by either:
    - passing -x to disk-image-create
    - exporting DIB_DEBUG_TRACE=N, with N=0/1

    Change-Id: I56ccd6753df31f7ddda641640cdb1985b2d9e856
    Partial-Bug: #1435306

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/170170

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/170196

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/170416

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/170196
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=8bee2fb42572480df0b35d9f5c821b67caddeeb6
Submitter: Jenkins
Branch: master

commit 8bee2fb42572480df0b35d9f5c821b67caddeeb6
Author: Mikhail Ivanov <email address hidden>
Date: Thu Apr 2 19:03:49 2015 +0300

    Sort projects in elements-deps

    Follow dib-lint requirements, sort projects in elements-deps

    Change-Id: I408dbc0aeabb0f2da8e4b8aea89d8247c4fbb693
    Partial-Bug: #1435306

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/169749
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=f3c2954715c54b6c45c5c261dbcb23edfe7aa8cf
Submitter: Jenkins
Branch: master

commit f3c2954715c54b6c45c5c261dbcb23edfe7aa8cf
Author: Pino Toscano <email address hidden>
Date: Wed Apr 1 15:44:38 2015 +0200

    Simplify environment.d scripts

    Just leave the exports there, removing the shebang as they are sourced
    and not executed directly.

    This has also the side-effect of making dib-lint ignore them, and thus
    enforce on them checks which don't make any good in them.

    Change-Id: I2a9506634a9b636ca97bd4a1fab72307c95b5388
    Partial-Bug: #1435306

[OpenStack Infra (hudson-openstack)]

Change abandoned by Mikhail Ivanov (<email address hidden>) on branch: master
Review: https://review.openstack.org/170170

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/171072
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=2fb2422da81f6ece47e626fc4d77bf5931931445
Submitter: Jenkins
Branch: master

commit 2fb2422da81f6ece47e626fc4d77bf5931931445
Author: Pino Toscano <email address hidden>
Date: Tue Apr 7 10:36:42 2015 +0200

    Make almost all the element scripts as e/u/pipefail

    Following the latest dib-lint reporting, make sure almost all the
    scripts are enabling -e, -u, and pipefail. This eases the discovery of
    failing commands, and the usage of unset variables.

    There are few exceptions with flags not set:

    * elements/hadoop-hdp/install.d/40-install-hdp, set -e
      when installing HDP 2.0, it tries to install tez, which does not seem
      to exist in Hortonworks' repositories

    * elements/ssh/install.d/33-ssh, set -e
      the version of augtool (part of augeas 0.10) in older Ubuntu versions
      (like Precise, needed by the cloudera plugin) exits with wrong return
      values in autosave mode

    * elements/storm/install.d/60-storm, set -e
      It tries to change the permission of /etc/supervisord.conf, which does
      not seem to exist

    Change-Id: Ic1314639dfc6a66c48ca87b6820707a2b0cb1dbd
    Partial-Bug: #1435306

[Pino Toscano (ptoscano)]

So, after some patches, the result (at least for me) is the following:

ERROR: elements/mysql/install.d/start-mysql.sh is not executable
ERROR: elements/mysql/install.d/start-mysql.sh is not set -e
ERROR: elements/mysql/install.d/start-mysql.sh is not set -u
ERROR: elements/mysql/install.d/start-mysql.sh is not set -o pipefail
ERROR: elements/mysql/install.d/start-mysql.sh does not follow DIB_DEBUG_TRACE
ERROR: elements/hadoop-hdp/install.d/40-install-hdp is not set -e
ERROR: elements/ssh/install.d/33-ssh is not set -e
ERROR: elements/storm/install.d/60-storm is not set -e
ERROR: elements/hadoop/post-install.d/firstboot is not executable
ERROR: elements/hadoop/post-install.d/firstboot is not set -e
ERROR: elements/hadoop/post-install.d/firstboot is not set -u
ERROR: elements/hadoop/post-install.d/firstboot is not set -o pipefail
ERROR: elements/hadoop/post-install.d/firstboot does not follow DIB_DEBUG_TRACE

* https://review.openstack.org/#/c/170416/ tries to handle elements/mysql/install.d/start-mysql.sh and elements/hadoop/post-install.d/firstboot

* elements/hadoop-hdp/install.d/40-install-hdp, elements/ssh/install.d/33-ssh, and elements/storm/install.d/60-storm need further fixes, see comment #10 about them

[OpenStack Infra (hudson-openstack)]

Change abandoned by Mikhail Ivanov (<email address hidden>) on branch: master
Review: https://review.openstack.org/170416
Reason: New patchset: https://review.openstack.org/#/c/172008/2

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/172008
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=6d3bdbf50a20ca237e310d4d8f66a5e4b5dee3aa
Submitter: Jenkins
Branch: master

commit 6d3bdbf50a20ca237e310d4d8f66a5e4b5dee3aa
Author: Mikhail Ivanov <email address hidden>
Date: Thu Apr 9 13:57:29 2015 +0300

    Adapt not dib elements to dib-lint checks

    This scripts have been adapted to dib-lint, and now they not trigger any dib-lint error, like

    is not set -e
    is not set -o pipefail
    and others

    Partial-Bug: #1435306

    Change-Id: I9abf3e5a543125cfb2ae11bf51d1dc0a03010b3d

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/182710
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=04f282e0e68d97835bd113c1aa3c3d82975a53aa
Submitter: Jenkins
Branch: master

commit 04f282e0e68d97835bd113c1aa3c3d82975a53aa
Author: Mikhail Ivanov <email address hidden>
Date: Wed May 13 18:06:52 2015 +0300

    Change augtool invocation

    - Change autosave mode to configuration file mode, because
      in Precise augtool return wrong value with augtool -s command.
    - Add -e flag, and now it not trigger any dib-lint error

    Change-Id: I239d403d1c20782525766c67b139e40c00907792
    Closes-Bug: #1450510
    Partial-Bug: #1435306

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/180997
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=052c141efe6305e1ef7ff86af72065aa34db5ef1
Submitter: Jenkins
Branch: master

commit 052c141efe6305e1ef7ff86af72065aa34db5ef1
Author: Mikhail Ivanov <email address hidden>
Date: Thu May 7 16:34:32 2015 +0300

    Adapt hadoop-hdp element to dib-lint checks

    - Remove tez package from this script, because it not exist in Hortonworks repository
    - Add -e flag, and now it not trigger any dib-lint error

    Closes-Bug: #1450509
    Partial-Bug: #1435306

    Change-Id: I1a09a727549416bd93f1463ad744b8bd03aa4452

[Pino Toscano (ptoscano)]

Sergey Lukjanov: this has been fixed with I74a386bf7d13203df9e93b6469384648df2ac389, merged as commit 4fd100a03697913e27a482502d974a2d969e319e.
Shall this be closed?

[Sergey Reshetnyak (sreshetniak)]

Pino, I close this bug