Elasticsearch plugin: tzdata persistent issues

[Swann Croiset]

The recurrent issue with tzdata and tzdata-java package versions is problematic and leads to break things regularly

The openjdk-7-jre-headless package has dependency on tzdata-java

These packages are updated upstream several times per year (ubuntu repositories) (and the old versions are removed) and also the fact is that tzdata-java has a stric dependency on tzdata package. This situation introduces 2 issues:
1/ the plugin doesn't build anymore since the last version is removed (HTTP 404 not found). This breaks the CI and delays the plugin validation
2 the released plugin RPM officially available online can be broken

The case #2 is critical and needs to be fixed.
For example the ES plugin 0.8/MOS 8 installed in 2016 since March 20 is broken because the plugin ships an old version of tzdata and tzdata-java BUT the last version of tzdata 2016b is always installed before the plugin installation. When the openjdk package is installed by puppet with its dependencies (tzdata-java) and because the "apt-get install" [0] refuse to DOWNGRADE the tzdata package and then fails.

Why we are shipping tzdata-java: to support the plugin deployment offline butthis package is not embed by fuel and the plugin needs to ship it.

Potential solutions:
* don't embed the tzdata-java package but in this case the plugin won't support the offline deployment and we need to document how to teach the fuel-creamirror command to take the plugin dependencies
* patch the tzdata-java package at plugin build time (pre_build_hook) to update the stric dependency on tzdata -> this is gruik and may introduce a risk
* in long term, the fuel-createmirror must ship plugins dependencies

[0] the command used by puppet to install package: /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install openjdk-7-jre-headless

[Swann Croiset (swann-w)]

The Elasticsearch_kibana plugin 0.8.0 is broken, the manual workaround before deploying the plugin is:

1/ install the plugin as usual (fuel plugins --install *rpm)
2/ on Fuel master node: patch the plugin repository:
  cd /var/www/nailgun/plugins/elasticsearch_kibana-0.8/repositories/ubuntu
  rm tzdata*
  wget http://security.ubuntu.com/ubuntu/pool/main/t/tzdata/tzdata-java_2016b-0ubuntu0.14.04_all.deb
  wget http://security.ubuntu.com/ubuntu/pool/main/t/tzdata/tzdata_2016b-0ubuntu0.14.04_all.deb
  rm Packages.gz
  dpkg-scanpackages . /dev/null | gzip -9c > Packages.gz

[guillaume thouvenin (guillaume-thouvenin)]

I posted the question why a strict dependency is needed between tzdata-java and tzdata. It seems that '>=' should be enough. Here is the link to the question:
https://answers.launchpad.net/ubuntu/+source/tzdata/+question/289092

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/295826

[Patrick Petit (patrick-michel-petit)]

On 22 March 2016 at 14:11:13, guillaume thouvenin (<email address hidden>) wrote:
I posted the question why a strict dependency is needed between tzdata-java and tzdata. It seems that '>=' should be enough. Here is the link to the question:
https://answers.launchpad.net/ubuntu/+source/tzdata/+question/289092

Mais apparement tu n’as pas reçu de réponse….

--
You received this bug notification because you are a member of LMA-
Toolchain Fuel Plugins, which is a bug assignee.
https://bugs.launchpad.net/bugs/1560436

Title:
Elasticsearch plugin: tzdata persistent issues

Status in LMA Toolchain:
Confirmed
Status in LMA Toolchain 0.8 series:
Confirmed

Bug description:
The recurrent issue with tzdata and tzdata-java package versions is
problematic and leads to break things regularly

The openjdk-7-jre-headless package has dependency on tzdata-java

These packages are updated upstream several times per year (ubuntu repositories) (and the old versions are removed) and also the fact is that tzdata-java has a stric dependency on tzdata package. This situation introduces 2 issues:
1/ the plugin doesn't build anymore since the last version is removed (HTTP 404 not found). This breaks the CI and delays the plugin validation
2 the released plugin RPM officially available online can be broken

The case #2 is critical and needs to be fixed.
For example the ES plugin 0.8/MOS 8 installed in 2016 since March 20 is broken because the plugin ships an old version of tzdata and tzdata-java BUT the last version of tzdata 2016b is always installed before the plugin installation. When the openjdk package is installed by puppet with its dependencies (tzdata-java) and because the "apt-get install" [0] refuse to DOWNGRADE the tzdata package and then fails.

Why we are shipping tzdata-java: to support the plugin deployment
offline butthis package is not embed by fuel and the plugin needs to
ship it.

Potential solutions:
* don't embed the tzdata-java package but in this case the plugin won't support the offline deployment and we need to document how to teach the fuel-creamirror command to take the plugin dependencies
* patch the tzdata-java package at plugin build time (pre_build_hook) to update the stric dependency on tzdata -> this is gruik and may introduce a risk
* in long term, the fuel-createmirror must ship plugins dependencies

[0] the command used by puppet to install package: /usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install openjdk-7-jre-headless

To manage notifications about this bug go to:
https://bugs.launchpad.net/lma-toolchain/+bug/1560436/+subscriptions

[guillaume thouvenin (guillaume-thouvenin)]

I confirm that we cannot deploy the LMA 0.8 plugin from Mirantis catalog no MOS 7.0. Installation failed with error:

    The following packages will be DOWNGRADED:
      tzdata
    0 upgraded, 27 newly installed, 1 downgraded, 0 to remove and 2 not upgraded.
    Need to get 43.6 MB of archives.
    After this operation, 73.7 MB of additional disk space will be used.
    E: There are problems and -y was used without --force-yes

[guillaume thouvenin (guillaume-thouvenin)]

Here is the full trace of the puppet error on MOS 7.0 with elasticsearch-kibana 0.8.0:

2016-03-22 14:10:08 +0000 /Stage[main]/Main/Package[openjdk-7-jre-headless]/ensure (err): change from purged to present failed: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install openjdk-7-jre-headless' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
The following packages were automatically installed and are no longer required:
  cloud-guest-utils eatmydata python-cheetah python-configobj
  python-json-pointer python-jsonpatch python-oauth python-serial
  python3-pycurl python3-software-properties software-properties-common
  unattended-upgrades
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  ca-certificates-java fontconfig-config fonts-dejavu-core java-common
  libasyncns0 libavahi-client3 libavahi-common-data libavahi-common3 libcups2
  libflac8 libfontconfig1 libjpeg-turbo8 libjpeg8 liblcms2-2 libnspr4 libnss3
  libnss3-nssdb libogg0 libpcsclite1 libpulse0 libsctp1 libsndfile1
  libvorbis0a libvorbisenc2 lksctp-tools tzdata tzdata-java
Suggested packages:
  default-jre equivs cups-common liblcms2-utils pcscd pulseaudio
  icedtea-7-jre-jamvm libnss-mdns sun-java6-fonts fonts-dejavu-extra
  fonts-ipafont-gothic fonts-ipafont-mincho ttf-wqy-microhei ttf-wqy-zenhei
  ttf-indic-fonts-core ttf-telugu-fonts ttf-oriya-fonts ttf-kannada-fonts
  ttf-bengali-fonts
The following NEW packages will be installed:
  ca-certificates-java fontconfig-config fonts-dejavu-core java-common
  libasyncns0 libavahi-client3 libavahi-common-data libavahi-common3 libcups2
  libflac8 libfontconfig1 libjpeg-turbo8 libjpeg8 liblcms2-2 libnspr4 libnss3
  libnss3-nssdb libogg0 libpcsclite1 libpulse0 libsctp1 libsndfile1
  libvorbis0a libvorbisenc2 lksctp-tools openjdk-7-jre-headless tzdata-java
The following packages will be DOWNGRADED:
  tzdata
0 upgraded, 27 newly installed, 1 downgraded, 0 to remove and 2 not upgraded.
Need to get 43.6 MB of archives.
After this operation, 73.7 MB of additional disk space will be used.
E: There are problems and -y was used without --force-yes

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/295826
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-elasticsearch-kibana/commit/?id=475228797247c8507ce05f833e7dbf23da827521
Submitter: Jenkins
Branch: master

commit 475228797247c8507ce05f833e7dbf23da827521
Author: Swann Croiset <email address hidden>
Date: Tue Mar 22 14:11:04 2016 +0100

    Change dependency of tzdata-java at build time

    Partial-bug: #1560436

    Change-Id: I4d9ec4bd639dfe2b422534ae6fe7318905d359f9

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/0.8
Review: https://review.openstack.org/296651

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/296651
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-elasticsearch-kibana/commit/?id=a81bbc6d4d76336c2a986299bfb2997895837f3d
Submitter: Jenkins
Branch: stable/0.8

commit a81bbc6d4d76336c2a986299bfb2997895837f3d
Author: Swann Croiset <email address hidden>
Date: Tue Mar 22 14:11:04 2016 +0100

    Change dependency of tzdata-java at build time

    Partial-bug: #1560436

    Change-Id: I4d9ec4bd639dfe2b422534ae6fe7318905d359f9
    (cherry picked from commit 475228797247c8507ce05f833e7dbf23da827521)

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/0.9
Review: https://review.openstack.org/297263

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/297263
Committed: https://git.openstack.org/cgit/openstack/fuel-plugin-elasticsearch-kibana/commit/?id=dd69ca50c7cd5622b413de8dbba9f6258ac63a62
Submitter: Jenkins
Branch: stable/0.9

commit dd69ca50c7cd5622b413de8dbba9f6258ac63a62
Author: Swann Croiset <email address hidden>
Date: Tue Mar 22 14:11:04 2016 +0100

    Change dependency of tzdata-java at build time

    Partial-bug: #1560436

    (cherry picked from commit 475228797247c8507ce05f833e7dbf23da827521)

    Conflicts:
     pre_build_hook

    Change-Id: I4d9ec4bd639dfe2b422534ae6fe7318905d359f9