Import failed because apt_install() was assumed to have worked

[Andreas Hasenack]

This code:
def ns_query(address):
    try:
        import dns.resolver
    except ImportError:
        apt_install('python-dnspython')
        import dns.resolver

Failed like this:
2016-07-09 06:25:01 INFO juju-log Installing python-dnspython with options: ['--option=Dpkg::Options::=--force-confold']
2016-07-09 06:25:01 INFO install E: Could not get lock /var/lib/dpkg/lock - open (11: Resource temporarily unavailable)
2016-07-09 06:25:01 INFO install E: Unable to lock the administration directory (/var/lib/dpkg/), is another process using it?
2016-07-09 06:25:01 INFO install Traceback (most recent call last):
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/install.real", line 579, in <module>
2016-07-09 06:25:01 INFO install assess_status(CONFIGS)
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/cinder_utils.py", line 886, in assess_status
2016-07-09 06:25:01 INFO install assess_status_func(configs)()
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 1530, in _assess_status_func
2016-07-09 06:25:01 INFO install state, message = _determine_os_workload_status(*args, **kwargs)
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 930, in _determine_os_workload_status
2016-07-09 06:25:01 INFO install configs, required_interfaces)
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 984, in _ows_check_generic_interfaces
2016-07-09 06:25:01 INFO install required_interfaces)
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/openstack/utils.py", line 1265, in incomplete_relation_data
2016-07-09 06:25:01 INFO install complete_ctxts = configs.complete_contexts()
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 295, in complete_contexts
2016-07-09 06:25:01 INFO install for i in six.itervalues(self.templates)]
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 120, in complete_contexts
2016-07-09 06:25:01 INFO install self.context()
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/openstack/templating.py", line 105, in context
2016-07-09 06:25:01 INFO install _ctxt = context()
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/openstack/context.py", line 592, in __call__
2016-07-09 06:25:01 INFO install addr = get_host_ip(unit_get('private-address'))
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/network/ip.py", line 443, in get_host_ip
2016-07-09 06:25:01 INFO install ip_addr = ns_query(hostname)
2016-07-09 06:25:01 INFO install File "/var/lib/juju/agents/unit-cinder-0/charm/hooks/charmhelpers/contrib/network/ip.py", line 420, in ns_query
2016-07-09 06:25:01 INFO install import dns.resolver
2016-07-09 06:25:01 INFO install ImportError: No module named dns.resolver
2016-07-09 06:25:01 ERROR juju.worker.uniter.operation runhook.go:107 hook "install" failed: exit status 1

This code structure is used all over the place in charmhelpers/contrib/network/ip.py. Specificlaly, for python-netifaces and python-netaddr. And twice for python-dnspython: one like above, the other in get_hostname() in the same file.

We should either pass fatal=True to *all* apt_install() invocations, or just maybe bite the bullet and flip the default from fatal=False to fatal=True in charmhelpers/fetch/__init__.py's apt_install(). Or even do it in _run_apt_command().

One silly case this change would make worse is if the specified package does not exist. It would keep uselessly retrying until the retry_count limit is hit. But then again we could improve the code to check what actually failed, and exit the retry loop if the error is "unknown package".

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/343780

[David Ames (thedac)]

This required a charm-sync and a change to the install hook for most of the charms.

https://review.openstack.org/#/q/topic:bug/1601972

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/343780
Committed: https://git.openstack.org/cgit/openstack/charm-cinder/commit/?id=5ca3cc736057d6b843819073fb3c114c7af27313
Submitter: Jenkins
Branch: master

commit 5ca3cc736057d6b843819073fb3c114c7af27313
Author: David Ames <email address hidden>
Date: Mon Jul 18 09:34:28 2016 -0700

    Install python dependencies early for CH ip.py

    When using charmhelpers.contrib.network.ip apt install was quietly
    failing. Install python dependencies in the early install hook
    (before install.real).

    Charm-helpers sync to bring in apt_install with fatal=True

    Change-Id: I24ef2178eb57b67e596d3557ab17cafa041ae4c9
    Partial-Bug: 1601972