[DIB] Image building fails for CDH when sahara-image-create is called with no version

[Luigi Toscano]

When sahara-image-create is called without version for the cloudera plugin (on CentOS7 and RHEL7), the creation of the 5.7 image fails (and 5.9 would fail too) because the following repository is not found:

http://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/.0/repodata/repomd.xml: [Errno 14] HTTP Error 404 - Not Found

The reason is the following code in diskimage-create.sh:

if [ -n "$DIB_CDH_MINOR_VERSION" ]; then
    # cut minor version number, e.g. from 5.7.1 to 5.7
    # this is needed if user specified minor version but didn't specify
    # hadoop version by '-v' parameter
    HADOOP_VERSION=${DIB_CDH_MINOR_VERSION%.*}
fi
export DIB_CDH_MINOR_VERSION=${DIB_CDH_MINOR_VERSION:-$HADOOP_VERSION.0}
[...]
if [ -z "$HADOOP_VERSION" -o "$HADOOP_VERSION" = "5.7" ]; then
   cloudera_5_7_ubuntu_image_name=${cloudera_5_7_ubuntu_image_name:-ubuntu_sahara_cloudera_$DIB_CDH_MINOR_VERSION}

In this case HADOOP_VERSION is empty, and DIB_CDH_MINOR_VERSION is not initialized.

In general, it is not guaranteed that DIB_CDH_MINOR_VERSION is properly initialized whenever it is used. We need at least a quick fix for newton, in addition to a more robust set of check for master.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/422811

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/423236

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/422811
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=3dfcb641c0fd9d26cd9f9d39ad9321fea6b1f4ba
Submitter: Jenkins
Branch: master

commit 3dfcb641c0fd9d26cd9f9d39ad9321fea6b1f4ba
Author: Luigi Toscano <email address hidden>
Date: Thu Jan 19 17:10:57 2017 +0100

    Set DIB_CDH_MINOR_VERSION when required

    Set DIB_CDH_MINOR_VERSION in diskimage-create so that it could not
    be an empty value. The order of the lines where DIB_CDH_VERSION is
    assigned for other CDH versions where DIB_CDH_MINOR_VERSION is not
    used was changed as well for consistency.
    Also, in order to try to prevent future errors, set the value of
    DIB_CDH_MINOR_VERSION based on DIB_CDH_VERSION in all elements where
    it is used (thanks to the ${VAR:-value} syntax which does not fail
    with set -u even if DIB_CDH_MINOR_VERSION is not assigned).

    Closes-Bug: #1657482
    Change-Id: I31b25fd4ba886d051b9b57902cd72349a4a2dbfa

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/423236
Committed: https://git.openstack.org/cgit/openstack/sahara-image-elements/commit/?id=114c3903a2de7f66a9f72b7a0680e9d221bf84d3
Submitter: Jenkins
Branch: stable/newton

commit 114c3903a2de7f66a9f72b7a0680e9d221bf84d3
Author: Luigi Toscano <email address hidden>
Date: Thu Jan 19 17:10:57 2017 +0100

    Set DIB_CDH_MINOR_VERSION when required

    Set DIB_CDH_MINOR_VERSION in diskimage-create so that it could not
    be an empty value. The order of the lines where DIB_CDH_VERSION is
    assigned for other CDH versions where DIB_CDH_MINOR_VERSION is not
    used was changed as well for consistency.
    Also, in order to try to prevent future errors, set the value of
    DIB_CDH_MINOR_VERSION based on DIB_CDH_VERSION in all elements where
    it is used (thanks to the ${VAR:-value} syntax which does not fail
    with set -u even if DIB_CDH_MINOR_VERSION is not assigned).

    Closes-Bug: #1657482
    Change-Id: I31b25fd4ba886d051b9b57902cd72349a4a2dbfa
    (cherry picked from commit 3dfcb641c0fd9d26cd9f9d39ad9321fea6b1f4ba)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/sahara-image-elements 6.0.0.0b3 development milestone.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/sahara-image-elements 5.0.1 release.