Designate-bind charm fails to install from the "master" branch on trusty

[Tytus Kurek]

Designate-bind charm fails to install from the master branch on Trusty:

$ juju deploy --series trusty /home/tkurek/charms/charm-designate-bind/build/builds/designate-bind
$ juju status
Model Controller Cloud/Region Version SLA
designate-bind lxd-test localhost/localhost 2.3.4 unsupported

App Version Status Scale Charm Store Rev OS Notes
designate-bind error 1 designate-bind local 0 ubuntu

Unit Workload Agent Machine Public address Ports Message
designate-bind/0* error idle 0 10.130.194.61 hook failed: "install"

Machine State DNS Inst id Series AZ Message
0 started 10.130.194.61 juju-22719e-0 trusty Running

Relation provider Requirer Interface Type Message
designate-bind:cluster designate-bind:cluster openstack-ha peer joining

According to the logs, the following part is failing:

2018-04-27 10:18:16 DEBUG install Processing ./wheelhouse/charmhelpers-0.18.8.tar.gz
2018-04-27 10:18:16 DEBUG install Processing ./wheelhouse/Tempita-0.5.2.tar.gz
2018-04-27 10:18:16 DEBUG install Processing ./wheelhouse/netifaces-0.10.6.tar.gz
2018-04-27 10:18:17 DEBUG install Processing ./wheelhouse/setuptools_scm-1.17.0.tar.gz
2018-04-27 10:18:17 DEBUG install Processing ./wheelhouse/Jinja2-2.10.tar.gz
2018-04-27 10:18:17 DEBUG install Processing ./wheelhouse/setuptools-39.0.1.zip
2018-04-27 10:18:18 DEBUG install Processing ./wheelhouse/MarkupSafe-1.0.tar.gz
2018-04-27 10:18:18 DEBUG install Processing ./wheelhouse/charms.reactive-0.6.3.tar.gz
2018-04-27 10:18:18 DEBUG install Processing ./wheelhouse/six-1.11.0.tar.gz
2018-04-27 10:18:18 DEBUG install Processing ./wheelhouse/PyYAML-3.12.tar.gz
2018-04-27 10:18:19 DEBUG install Processing ./wheelhouse/pip-8.1.2.tar.gz
2018-04-27 10:18:19 DEBUG install Processing ./wheelhouse/netaddr-0.7.19.tar.gz
2018-04-27 10:18:19 DEBUG install Processing ./wheelhouse/pyaml-17.12.1.tar.gz
2018-04-27 10:18:20 DEBUG install Processing ./wheelhouse/charms.openstack-0.0.1.dev1.zip
2018-04-27 10:18:20 DEBUG install Complete output from command python setup.py egg_info:
2018-04-27 10:18:20 DEBUG install running egg_info
2018-04-27 10:18:20 DEBUG install creating pip-egg-info/charms.openstack.egg-info
2018-04-27 10:18:20 DEBUG install writing dependency_links to pip-egg-info/charms.openstack.egg-info/dependency_links.txt
2018-04-27 10:18:20 DEBUG install writing requirements to pip-egg-info/charms.openstack.egg-info/requires.txt
2018-04-27 10:18:20 DEBUG install writing top-level names to pip-egg-info/charms.openstack.egg-info/top_level.txt
2018-04-27 10:18:20 DEBUG install writing pip-egg-info/charms.openstack.egg-info/PKG-INFO
2018-04-27 10:18:20 DEBUG install writing manifest file 'pip-egg-info/charms.openstack.egg-info/SOURCES.txt'
2018-04-27 10:18:20 DEBUG install error: [Errno 2] No such file or directory: 'git'
2018-04-27 10:18:20 DEBUG install File Finder Failed for .git = setuptools_scm.git:list_files_in_archive
2018-04-27 10:18:20 DEBUG install
2018-04-27 10:18:20 DEBUG install ----------------------------------------
2018-04-27 10:18:20 DEBUG install Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-3vczam4d-build/
2018-04-27 10:18:20 DEBUG install Traceback (most recent call last):
2018-04-27 10:18:20 DEBUG install File "/var/lib/juju/agents/unit-designate-bind-0/charm/hooks/install", line 8, in <module>
2018-04-27 10:18:20 DEBUG install basic.bootstrap_charm_deps()
2018-04-27 10:18:20 DEBUG install File "lib/charms/layer/basic.py", line 93, in bootstrap_charm_deps
2018-04-27 10:18:20 DEBUG install glob('wheelhouse/*'))
2018-04-27 10:18:20 DEBUG install File "/usr/lib/python3.4/subprocess.py", line 561, in check_call
2018-04-27 10:18:20 DEBUG install raise CalledProcessError(retcode, cmd)
2018-04-27 10:18:20 DEBUG install subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-designate-bind-0/.venv/bin/pip', 'install', '-U', '--no-index', '-f', 'wheelhouse', 'wheelhouse/charmhelpers-0.18.8.tar.gz', 'wheelhouse/Tempita-0.5.2.tar.gz', 'wheelhouse/netifaces-0.10.6.tar.gz', 'wheelhouse/setuptools_scm-1.17.0.tar.gz', 'wheelhouse/Jinja2-2.10.tar.gz', 'wheelhouse/setuptools-39.0.1.zip', 'wheelhouse/MarkupSafe-1.0.tar.gz', 'wheelhouse/charms.reactive-0.6.3.tar.gz', 'wheelhouse/six-1.11.0.tar.gz', 'wheelhouse/PyYAML-3.12.tar.gz', 'wheelhouse/pip-8.1.2.tar.gz', 'wheelhouse/netaddr-0.7.19.tar.gz', 'wheelhouse/pyaml-17.12.1.tar.gz', 'wheelhouse/charms.openstack-0.0.1.dev1.zip']' returned non-zero exit status 1
2018-04-27 10:18:20 ERROR juju.worker.uniter.operation runhook.go:113 hook "install" failed: exit status 1

Running the failing command manually results with:

# /var/lib/juju/agents/unit-designate-bind-0/.venv/bin/pip install -U --no-index -f wheelhouse wheelhouse/charmhelpers-0.18.8.tar.gz wheelhouse/Tempita-0.5.2.tar.gz wheelhouse/netifaces-0.10.6.tar.gz wheelhouse/setuptools_scm-1.17.0.tar.gz wheelhouse/Jinja2-2.10.tar.gz wheelhouse/setuptools-39.0.1.zip wheelhouse/MarkupSafe-1.0.tar.gz wheelhouse/charms.reactive-0.6.3.tar.gz wheelhouse/six-1.11.0.tar.gz wheelhouse/PyYAML-3.12.tar.gz wheelhouse/pip-8.1.2.tar.gz wheelhouse/netaddr-0.7.19.tar.gz wheelhouse/pyaml-17.12.1.tar.gz wheelhouse/charms.openstack-0.0.1.dev1.zip
Ignoring indexes: https://pypi.python.org/simple
Requirement 'wheelhouse/charmhelpers-0.18.8.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/Tempita-0.5.2.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/netifaces-0.10.6.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/setuptools_scm-1.17.0.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/Jinja2-2.10.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/setuptools-39.0.1.zip' looks like a filename, but the file does not exist
Requirement 'wheelhouse/MarkupSafe-1.0.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/charms.reactive-0.6.3.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/six-1.11.0.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/PyYAML-3.12.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/pip-8.1.2.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/netaddr-0.7.19.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/pyaml-17.12.1.tar.gz' looks like a filename, but the file does not exist
Requirement 'wheelhouse/charms.openstack-0.0.1.dev1.zip' looks like a filename, but the file does not exist
Processing ./wheelhouse/charmhelpers-0.18.8.tar.gz
Exception:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/site-packages/pip/commands/install.py", line 310, in run
    wb.build(autobuilding=True)
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/site-packages/pip/wheel.py", line 750, in build
    self.requirement_set.prepare_files(self.finder)
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/site-packages/pip/req/req_set.py", line 370, in prepare_files
    ignore_dependencies=self.ignore_dependencies))
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/site-packages/pip/req/req_set.py", line 587, in _prepare_file
    session=self.session, hashes=hashes)
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/site-packages/pip/download.py", line 798, in unpack_url
    unpack_file_url(link, location, download_dir, hashes=hashes)
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/site-packages/pip/download.py", line 705, in unpack_file_url
    unpack_file(from_path, location, content_type, link)
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/site-packages/pip/utils/__init__.py", line 602, in unpack_file
    tarfile.is_tarfile(filename) or
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/tarfile.py", line 2418, in is_tarfile
    t = open(name)
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/tarfile.py", line 1548, in open
    return func(name, "r", fileobj, **kwargs)
  File "/var/lib/juju/agents/unit-designate-bind-0/.venv/lib/python3.4/tarfile.py", line 1646, in bz2open
    compresslevel=compresslevel)
  File "/usr/lib/python3.4/bz2.py", line 102, in __init__
    self._fp = _builtin_open(filename, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/root/wheelhouse/charmhelpers-0.18.8.tar.gz'

Copying the files to '/root/wheelhouse' directory and re-running the command results with:

root@juju-22719e-0:~# /var/lib/juju/agents/unit-designate-bind-0/.venv/bin/pip install -U --no-index -f wheelhouse wheelhouse/charmhelpers-0.18.8.tar.gz wheelhouse/Tempita-0.5.2.tar.gz wheelhouse/netifaces-0.10.6.tar.gz wheelhouse/setuptools_scm-1.17.0.tar.gz wheelhouse/Jinja2-2.10.tar.gz wheelhouse/setuptools-39.0.1.zip wheelhouse/MarkupSafe-1.0.tar.gz wheelhouse/charms.reactive-0.6.3.tar.gz wheelhouse/six-1.11.0.tar.gz wheelhouse/PyYAML-3.12.tar.gz wheelhouse/pip-8.1.2.tar.gz wheelhouse/netaddr-0.7.19.tar.gz wheelhouse/pyaml-17.12.1.tar.gz wheelhouse/charms.openstack-0.0.1.dev1.zip
Ignoring indexes: https://pypi.python.org/simple
Processing ./wheelhouse/charmhelpers-0.18.8.tar.gz
Processing ./wheelhouse/Tempita-0.5.2.tar.gz
Processing ./wheelhouse/netifaces-0.10.6.tar.gz
Processing ./wheelhouse/setuptools_scm-1.17.0.tar.gz
Processing ./wheelhouse/Jinja2-2.10.tar.gz
Processing ./wheelhouse/setuptools-39.0.1.zip
Processing ./wheelhouse/MarkupSafe-1.0.tar.gz
Processing ./wheelhouse/charms.reactive-0.6.3.tar.gz
Processing ./wheelhouse/six-1.11.0.tar.gz
Processing ./wheelhouse/PyYAML-3.12.tar.gz
Processing ./wheelhouse/pip-8.1.2.tar.gz
Processing ./wheelhouse/netaddr-0.7.19.tar.gz
Processing ./wheelhouse/pyaml-17.12.1.tar.gz
Processing ./wheelhouse/charms.openstack-0.0.1.dev1.zip
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/charms.openstack.egg-info
    writing top-level names to pip-egg-info/charms.openstack.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/charms.openstack.egg-info/dependency_links.txt
    writing pip-egg-info/charms.openstack.egg-info/PKG-INFO
    writing requirements to pip-egg-info/charms.openstack.egg-info/requires.txt
    writing manifest file 'pip-egg-info/charms.openstack.egg-info/SOURCES.txt'
    error: [Errno 2] No such file or directory: 'git'
    File Finder Failed for .git = setuptools_scm.git:list_files_in_archive

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-y70f8pgs-build/

This indicates an issue with charms.openstack and is probably a bug there, and not in the designate-bind charm itself. Attaching full log from the failing unit.

[Tytus Kurek (tkurek)]

The version from the master branch installs fine on Xenial. Also the stable version (from the charm store) installs fine on Trusty.

[Alex Kavanagh (ajkavanagh)]

This has also occurred on the OSCI for master branch and trusty.

[Alex Kavanagh (ajkavanagh)]

Believe it or not, this is caused by git not being installed on the target platform. This is due to a change in setuptools_scm from version 1.16.1 (19th March) to 1.17 (20th March) which looks for the 'git' command. A recent commit to master (https://github.com/pypa/setuptools_scm/commit/d917cace49b73b25f82cd594102a0c2392f37160) has fixed this issue.

The reason it's triggering on charms.openstack is that it is installed from git+https//... rather than just a a pip module from the archive; thus pip is trying to workout what the version is by looking at git tags, and if git isn't installed then setuptools_scm fails (for version 1.17 and 2.0). I'm guessing that 2.1 will release shortly that will fix the issue, as there's been a few commits.

We could:

1. Sit tight, and wait for 2.1 to be released.
2. Pre-install 1.16.1 as a requirement in one of the packages to stop the upgrade
3. pre-install git before setup tools gets a chance to run.

I can also file a bug against setuptools_scm but the problem has already been 'fixed' in a way.

The main issue is that charms.openstack is still being installed git+https:// from master -- we should probably release it to PyPi and start doing releases.

[Alex Kavanagh (ajkavanagh)]

Okay, after some discussion the fix I'm going for is to pre-install git on the unit as part of the initial charm setup; git is already installed on xenial++ so it seems reasonable to install it on trusty.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/567574

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/567574
Committed: https://git.openstack.org/cgit/openstack/charm-designate-bind/commit/?id=934d2534add4d7c31e10461dd187b7c8a9a37339
Submitter: Zuul
Branch: master

commit 934d2534add4d7c31e10461dd187b7c8a9a37339
Author: Alex Kavanagh <email address hidden>
Date: Thu May 10 13:10:22 2018 +0100

    Rebuild the charm with the new layer-openstack

    This is for bug 1767328 which resulted in the charm not installing
    on trusty.

    Change-Id: I9ce5cac843b2f0341feed0df3de48e41de84fcab
    Related-Bug: #1767328
    Depends-On: I8a7daa0fff3d5bc7b11f70cde246975987c7cb66