Proposed charm install fails inside fully proxied environment on keystoneauth1

[Alexander Balderson]

Installing the octavia-diskimage-retrofit inside a fully proxied environment failed when looking for keystoneauth1

2019-12-08 09:00:38 DEBUG install Successfully built pip setuptools psutil PyYAML pyaml charms.reactive charmhelpers Tempita wheel six setuptools-scm netaddr dnspython netifaces MarkupSafe charms.openstack Jinja2
2019-12-08 09:00:38 DEBUG install Installing collected packages: pip, setuptools, psutil, PyYAML, pyaml, netaddr, Tempita, MarkupSafe, Jinja2, six, charmhelpers, charms.reactive, wheel, setuptools-scm, dnspython, netifaces, charms.openstack
2019-12-08 09:00:40 DEBUG install Successfully installed Jinja2-2.10.3 MarkupSafe-1.1.1 PyYAML-5.2 Tempita-0.5.2 charmhelpers-0.20.7 charms.openstack-0.0.1.dev1 charms.reactive-1.3.0 dnspython-1.16.0 netaddr-0.7.19 netifaces-0.10.9 pip-18.1 psutil-5.6.7 pyaml-19.4.1 setuptools-41.6.0 setuptools-scm-1.17.0 six-1.13.0 wheel-0.33.6
2019-12-08 09:00:41 DEBUG install Collecting keystoneauth1
2019-12-08 09:01:41 DEBUG install Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bffa09e8>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/keystoneauth1/
2019-12-08 09:02:42 DEBUG install Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bffa0b00>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/keystoneauth1/
2019-12-08 09:03:43 DEBUG install Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bffa0128>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/keystoneauth1/
2019-12-08 09:04:45 DEBUG install Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bffa06a0>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/keystoneauth1/
2019-12-08 09:05:49 DEBUG install Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7fe8bffa0208>: Failed to establish a new connection: [Errno 101] Network is unreachable',)': /simple/keystoneauth1/
2019-12-08 09:06:49 DEBUG install Could not find a version that satisfies the requirement keystoneauth1 (from versions: )
2019-12-08 09:06:49 DEBUG install No matching distribution found for keystoneauth1
2019-12-08 09:07:09 DEBUG install Traceback (most recent call last):
2019-12-08 09:07:09 DEBUG install File "/var/lib/juju/agents/unit-octavia-diskimage-retrofit-0/charm/hooks/install", line 8, in <module>
2019-12-08 09:07:09 DEBUG install basic.bootstrap_charm_deps()
2019-12-08 09:07:09 DEBUG install File "lib/charms/layer/basic.py", line 122, in bootstrap_charm_deps
2019-12-08 09:07:09 DEBUG install check_call([pip, 'install', '-U'] + cfg.get('python_packages'))
2019-12-08 09:07:09 DEBUG install File "/usr/lib/python3.6/subprocess.py", line 311, in check_call
2019-12-08 09:07:09 DEBUG install raise CalledProcessError(retcode, cmd)
2019-12-08 09:07:09 DEBUG install subprocess.CalledProcessError: Command '['/var/lib/juju/agents/unit-octavia-diskimage-retrofit-0/.venv/bin/pip', 'install', '-U', 'keystoneauth1', 'python-glanceclient']' returned non-zero exit status 1.
2019-12-08 09:07:09 ERROR juju.worker.uniter.operation runhook.go:132 hook "install" failed: exit status 1
2019-12-08 09:07:09 INFO juju.worker.uniter resolver.go:126 awaiting error resolution for "install" hook

Crashdump is attached.

[Alexander Balderson (asbalderson)]

Bundle

[Alexander Balderson (asbalderson)]

crashdump

[Frode Nordahl (fnordahl)]

The ``keystone1`` package is distributed with the charm, so the question is why ``layer-basic`` / ``setuptools`` attempts to download it anyway.

It may be dependencies missing that should have been included at build time or there may have been regressions in layer-basic or charm-tools.

[Nikolay Vinogradov (nikolay.vinogradov)]

The most recent version of the charm still tries to install keystone and fails in the fully proxied environment, please help to fix.

[Nikolay Vinogradov (nikolay.vinogradov)]

During an ongoing deployment, we've faced this same issue, which is blocking now the UA handover.

[Gábor Mészáros (gabor.meszaros)]

added field-high.

[Liam Young (gnuoy)]

Looks like these are relevant:

https://github.com/juju-solutions/layer-basic/issues/149

and

https://github.com/openstack/charm-octavia-diskimage-retrofit/commit/92067add1e122f3e0c74c30361d532eb74791860

[Liam Young (gnuoy)]

To fix this issue I think we need to get these landed:

https://github.com/juju-solutions/layer-basic/pull/156
https://review.opendev.org/#/c/708900/1

[Liam Young (gnuoy)]

https://review.opendev.org/#/c/708900/ no longer depends on https://github.com/juju-solutions/layer-basic/pull/156

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/708900
Committed: https://git.openstack.org/cgit/openstack/charm-octavia-diskimage-retrofit/commit/?id=ffc9774544c14e1379dab58d68552fadadbfbcf0
Submitter: Zuul
Branch: master

commit ffc9774544c14e1379dab58d68552fadadbfbcf0
Author: Liam Young <email address hidden>
Date: Thu Feb 20 15:35:43 2020 +0000

    Install deps from wheelhouse

    To make the charm install properly from behind a proxy and to
    make it consistent with other python requirements, install
    'keystoneauth1', 'python-glanceclient' from the wheelhouse. This
    is how it was originally done but was changed to work around
    layer basic issue #149 *1. This change reverts the workaround and
    pins setuptools. In the longer term *2 will remove the needs
    for the setuptools pinning.

    *1 https://github.com/juju-solutions/layer-basic/issues/149
    *2 https://github.com/juju-solutions/layer-basic/pull/156

    Closes-Bug: #1855766

    Change-Id: Ifc750a5a1ee8441bb5296fc0059fe90932c5b3cd

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/20.02
Review: https://review.opendev.org/709055

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/709055
Committed: https://git.openstack.org/cgit/openstack/charm-octavia-diskimage-retrofit/commit/?id=6a2b2dca51b1126cee8869fea072e7f784948e0e
Submitter: Zuul
Branch: stable/20.02

commit 6a2b2dca51b1126cee8869fea072e7f784948e0e
Author: Liam Young <email address hidden>
Date: Thu Feb 20 15:35:43 2020 +0000

    Install deps from wheelhouse

    To make the charm install properly from behind a proxy and to
    make it consistent with other python requirements, install
    'keystoneauth1', 'python-glanceclient' from the wheelhouse. This
    is how it was originally done but was changed to work around
    layer basic issue #149 *1. This change reverts the workaround and
    pins setuptools. In the longer term *2 will remove the needs
    for the setuptools pinning.

    *1 https://github.com/juju-solutions/layer-basic/issues/149
    *2 https://github.com/juju-solutions/layer-basic/pull/156

    Closes-Bug: #1855766

    Change-Id: Ifc750a5a1ee8441bb5296fc0059fe90932c5b3cd
    (cherry picked from commit ffc9774544c14e1379dab58d68552fadadbfbcf0)