bootstrap-ansible.sh not installing ansible==2.4.3.0

Bug #1751316 reported by Matt Thompson
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
openstack-ansible
Fix Released
Critical
Jesse Pretorius

Bug Description

I currently have an AIO deployed w/ master -- initial deployment went fine. Periodically, I pull down changes in /opt/openstack-ansible and re-run scripts/bootstrap-ansible.sh to grab latest roles. scripts/bootstrap-ansible.sh runs fine but then running any playbooks w/ openstack-ansible fails because the system has ansible 2.3.2.0 installed and not 2.4.3.0.

Here's some output from scripts/bootstrap-ansible.sh:

+ PIP_COMMAND=/opt/ansible-runtime/bin/pip
+ PIP_OPTS+=' --constraint global-requirement-pins.txt'
+ PIP_OPTS+=' --constraint https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?id=83dbef854e3b2b699f9382a993997a94bba1f5cb'
+ /opt/ansible-runtime/bin/pip install --constraint global-requirement-pins.txt --constraint 'https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?id=83dbef854e3b2b699f9382a993997a94bba1f5cb' -r requirements.txt ansible==2.4.3.0
Ignoring jsbeautifier: markers 'python_version == "3.4"' don't match your environment
Ignoring hyperframe: markers 'python_version == "3.4"' don't match your environment
Ignoring cssutils: markers 'python_version == "3.4"' don't match your environment
Ignoring kaitaistruct: markers 'python_version == "3.4"' don't match your environment
Ignoring unicodecsv: markers 'python_version == "2.7"' don't match your environment
Ignoring pathlib: markers 'python_version == "2.7"' don't match your environment
Ignoring tornado: markers 'python_version == "3.4"' don't match your environment
Ignoring backports.ssl-match-hostname: markers 'python_version == "2.7"' don't match your environment
Ignoring ruamel.yaml: markers 'python_version == "3.4"' don't match your environment
Ignoring html2text: markers 'python_version == "3.4"' don't match your environment
Ignoring funcsigs: markers 'python_version == "2.7"' don't match your environment
Ignoring dnspython3: markers 'python_version == "3.4"' don't match your environment
Ignoring hpack: markers 'python_version == "3.4"' don't match your environment
Ignoring brotlipy: markers 'python_version == "3.4"' don't match your environment
Ignoring mypy: markers 'python_version == "3.4"' don't match your environment
Ignoring urwid: markers 'python_version == "3.4"' don't match your environment
Ignoring singledispatch: markers 'python_version == "2.7"' don't match your environment
Ignoring thriftpy: markers 'python_version == "2.7"' don't match your environment
Ignoring argh: markers 'python_version == "3.4"' don't match your environment
Ignoring weakrefmethod: markers 'python_version == "2.7"' don't match your environment
Ignoring blinker: markers 'python_version == "3.4"' don't match your environment
Ignoring typed-ast: markers 'python_version == "3.4"' don't match your environment
Ignoring h2: markers 'python_version == "3.4"' don't match your environment
Ignoring mitmproxy: markers 'python_version == "3.4"' don't match your environment
Ignoring EditorConfig: markers 'python_version == "3.4"' don't match your environment
Ignoring ndg-httpsclient: markers 'python_version == "2.7"' don't match your environment
Ignoring functools32: markers 'python_version == "2.7"' don't match your environment
Ignoring watchdog: markers 'python_version == "3.4"' don't match your environment
Ignoring pathtools: markers 'python_version == "3.4"' don't match your environment
Ignoring qpid-python: markers 'python_version == "2.7"' don't match your environment
Ignoring scandir: markers 'python_version == "2.7"' don't match your environment
Ignoring futures: markers 'python_version == "2.7"' don't match your environment
Ignoring happybase: markers 'python_version == "2.7"' don't match your environment
Ignoring sortedcontainers: markers 'python_version == "3.4"' don't match your environment
Ignoring ndg-httpsclient: markers 'python_version < "3.0"' don't match your environment
Collecting pyopenssl==17.5.0 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 14))
  Downloading http://172.29.236.100:8181/packages/79/db/7c0cfe4aa8341a5fab4638952520d8db6ab85ff84505e12c00ea311c3516/pyOpenSSL-17.5.0-py2.py3-none-any.whl (53kB)
Collecting netaddr==0.7.19 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 16))
  Downloading http://172.29.236.100:8181/packages/ba/97/ce14451a9fd7bdb5a397abf99b24a1a6bb7a1a440b019bebd2e9a0dbec74/netaddr-0.7.19-py2.py3-none-any.whl (1.6MB)
Collecting prettytable==0.7.2 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 17))
Collecting pyyaml==3.12 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 20))
Collecting pyasn1==0.4.2 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 54))
  Downloading http://172.29.236.100:8181/packages/ba/fe/02e3e2ee243966b143657fb8bd6bc97595841163b6d8c26820944acaec4d/pyasn1-0.4.2-py2.py3-none-any.whl (71kB)
Collecting python_memcached==1.59 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 58))
  Downloading http://172.29.236.100:8181/packages/f5/90/19d3908048f70c120ec66a39e61b92c253e834e6e895cd104ce5e46cbe53/python_memcached-1.59-py2.py3-none-any.whl
Collecting six==1.11.0 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 60))
  Downloading http://172.29.236.100:8181/packages/67/4b/141a581104b1f6397bfa78ac9d43d8ad29a7ca43ea90a2d863fe3056e86a/six-1.11.0-py2.py3-none-any.whl
Collecting virtualenv==15.1.0 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 89))
  Downloading http://172.29.236.100:8181/packages/6f/86/3dc328ee7b1a6419ebfac7896d882fba83c48e3561d22ddddf38294d3e83/virtualenv-15.1.0-py2.py3-none-any.whl (1.8MB)
Collecting ansible==2.3.2.0 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 91))
Collecting paramiko==2.4.0 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 171))
  Downloading http://172.29.236.100:8181/packages/be/9f/2b899b028aec1f3973253c0cf8dda6fbff65f4930f7ebedc43033e9f1b18/paramiko-2.4.0-py2.py3-none-any.whl (192kB)
Collecting jinja2==2.10 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 172))
  Downloading http://172.29.236.100:8181/packages/7f/ff/ae64bacdfc95f27a016a7bed8e8686763ba4d277a78ca76f32659220a731/Jinja2-2.10-py2.py3-none-any.whl (126kB)
Collecting bcrypt==3.1.4 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 263))
  Downloading http://172.29.236.100:8181/packages/72/8d/57da727cbb4ddf0295b0665a93bd8f46fbbd2a48b76cbb0896ca7d27301f/bcrypt-3.1.4-cp35-cp35m-manylinux1_x86_64.whl (54kB)
Collecting pynacl==1.2.1 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 294))
  Downloading http://172.29.236.100:8181/packages/c9/9c/c8d4ae0f590faded4af55381131ad4db8729214eba6537c520622ba3548a/PyNaCl-1.2.1-cp35-cp35m-manylinux1_x86_64.whl (692kB)
Collecting markupsafe==1.0 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 295))
Collecting cryptography==2.1.4 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 326))
  Downloading http://172.29.236.100:8181/packages/15/4e/c0a0641dc0b4bba7c1dfcf66e30ef34effe7f7dc20a37459e9e052afc4cf/cryptography-2.1.4-cp35-cp35m-manylinux1_x86_64.whl (2.2MB)
Requirement already up-to-date: setuptools==38.5.1 in /opt/ansible-runtime/lib/python3.5/site-packages (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 2))
Collecting pycrypto==2.6.1 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 29))
Collecting cffi==1.11.4 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 80))
  Downloading http://172.29.236.100:8181/packages/cd/e7/2e146fd7fe666727dd67d8a882c8692a40524541fb0f785300b011a12ed0/cffi-1.11.4-cp35-cp35m-manylinux1_x86_64.whl (419kB)
Collecting asn1crypto==0.24.0 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 277))
  Downloading http://172.29.236.100:8181/packages/ea/cd/35485615f45f30a510576f1a56d1e0a7ad7bd8ab5ed7cdc600ef7cd06222/asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)
Collecting idna==2.6 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 41))
  Downloading http://172.29.236.100:8181/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl (56kB)
Collecting pycparser==2.18 (from -c http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt (line 117))
  Downloading http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/pycparser-2.18-py2.py3-none-any.whl (206kB)
Installing collected packages: six, asn1crypto, idna, pycparser, cffi, cryptography, pyopenssl, netaddr, prettytable, pyyaml, pycrypto, pyasn1, python-memcached, virtualenv, markupsafe, jinja2, pynacl, bcrypt, paramiko, ansible
Successfully installed ansible-2.3.2.0 asn1crypto-0.24.0 bcrypt-3.1.4 cffi-1.11.4 cryptography-2.1.4 idna-2.6 jinja2-2.10 markupsafe-1.0 netaddr-0.7.19 paramiko-2.4.0 prettytable-0.7.2 pyasn1-0.4.2 pycparser-2.18 pycrypto-2.6.1 pynacl-1.2.1 pyopenssl-17.5.0 python-memcached-1.59 pyyaml-3.12 six-1.11.0 virtualenv-15.1.0

As you can see, we request ansible==2.4.3.0 but ansible-2.3.2.0 is getting installed. If I look at requirements_absolute_requirements.txt, I can see ansible==2.3.2.0 listed:

root@aio1:/opt/openstack-ansible# curl -s http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt | grep ansible
ansible==2.3.2.0
root@aio1:/opt/openstack-ansible#

A hacky workaround has been to move /root/.pip aside, run scripts/bootstrap-ansible.sh, and then continue on.

Please let me know if I can provide any further info.

Thanks!

--Matt

Revision history for this message
Matt Thompson (mattt416) wrote :

Additionally, here's my /root/.pip/pip.conf:

root@aio1:/opt/openstack-ansible# cat /root/.pip/pip.conf
# Ansible managed

[global]
timeout = 120
index-url = http://172.29.236.100:8181/simple
constraint =
        http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/requirements_absolute_requirements.txt
trusted-host =
        172.29.236.100

[install]
upgrade = True
pre = True
find-links =
        http://172.29.236.100:8181/os-releases/17.0.0/ubuntu-16.04-x86_64/
root@aio1:/opt/openstack-ansible#

Changed in openstack-ansible:
status: New → Confirmed
importance: Undecided → Critical
assignee: nobody → Jesse Pretorius (jesse-pretorius)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible (master)

Fix proposed to branch: master
Review: https://review.openstack.org/547526

Changed in openstack-ansible:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible (master)

Reviewed: https://review.openstack.org/547526
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=6b94b21e18181eab1eddb1f3040c430051871a20
Submitter: Zuul
Branch: master

commit 6b94b21e18181eab1eddb1f3040c430051871a20
Author: Jesse Pretorius <email address hidden>
Date: Fri Feb 23 17:34:41 2018 +0000

    Isolate the Ansible bootstrap

    As we move forward with the python build/install simplification,
    more of the config is being applied in the pip conf, rather than
    being given as CLI options in each ansible task. This is good
    for providing a consistent experience, but has side effects for
    things like the Ansible bootstrap.

    When bootstrapping Ansible, we want to make use of a specific
    set of constraints which is not polluted by additional config
    on the host. We do not want the constraints from the repo
    server because they happen to include a constraint for Ansible
    which clashes with our requirement. As such, we only want the
    native OpenStack upper-constraints.

    In this patch we use the --isolated CLI option to ignore the
    host's pip configuration. We also introduce the ability to set
    extra pip install options by using the PIP_OPTS environment
    variable. This can be used to set the installation to use an
    alternative pypi mirror, or for extra links, etc.

    Change-Id: Ic966bafd04c4c01b3d93851a0e3ec2c1f3312f28
    Closes-Bug: #1751316

Changed in openstack-ansible:
status: In Progress → Fix Released
Revision history for this message
Jun Go (gojun077) wrote :

Request: Could you also apply this fix to "stable/pike" branch in addition to master?

Thanks

Revision history for this message
Jesse Pretorius (jesse-pretorius) wrote :

@Jun Can you please explain why it's necessary to back port it, given that the problem as reported does not exist in stable/pike?

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible 18.0.0.0b1

This issue was fixed in the openstack/openstack-ansible 18.0.0.0b1 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible 18.0.0.0b2

This issue was fixed in the openstack/openstack-ansible 18.0.0.0b2 development milestone.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/625529

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/625572

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible (stable/pike)

Reviewed: https://review.openstack.org/625572
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=e86956fec6c253307d0f974247477dd165a8f124
Submitter: Zuul
Branch: stable/pike

commit e86956fec6c253307d0f974247477dd165a8f124
Author: Jesse Pretorius <email address hidden>
Date: Fri Feb 23 17:34:41 2018 +0000

    Isolate the Ansible bootstrap

    As we move forward with the python build/install simplification,
    more of the config is being applied in the pip conf, rather than
    being given as CLI options in each ansible task. This is good
    for providing a consistent experience, but has side effects for
    things like the Ansible bootstrap.

    When bootstrapping Ansible, we want to make use of a specific
    set of constraints which is not polluted by additional config
    on the host. We do not want the constraints from the repo
    server because they happen to include a constraint for Ansible
    which clashes with our requirement. As such, we only want the
    native OpenStack upper-constraints.

    In this patch we use the --isolated CLI option to ignore the
    host's pip configuration. We also introduce the ability to set
    extra pip install options by using the PIP_OPTS environment
    variable. This can be used to set the installation to use an
    alternative pypi mirror, or for extra links, etc.

    Change-Id: Ic966bafd04c4c01b3d93851a0e3ec2c1f3312f28
    Closes-Bug: #1751316
    (cherry picked from commit 6b94b21e18181eab1eddb1f3040c430051871a20)

tags: added: in-stable-pike
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible (stable/queens)

Reviewed: https://review.openstack.org/625529
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=35e57ff59dd89d658cf577b244b544bc1d938b0a
Submitter: Zuul
Branch: stable/queens

commit 35e57ff59dd89d658cf577b244b544bc1d938b0a
Author: Jesse Pretorius <email address hidden>
Date: Fri Feb 23 17:34:41 2018 +0000

    Isolate the Ansible bootstrap

    As we move forward with the python build/install simplification,
    more of the config is being applied in the pip conf, rather than
    being given as CLI options in each ansible task. This is good
    for providing a consistent experience, but has side effects for
    things like the Ansible bootstrap.

    When bootstrapping Ansible, we want to make use of a specific
    set of constraints which is not polluted by additional config
    on the host. We do not want the constraints from the repo
    server because they happen to include a constraint for Ansible
    which clashes with our requirement. As such, we only want the
    native OpenStack upper-constraints.

    In this patch we use the --isolated CLI option to ignore the
    host's pip configuration. We also introduce the ability to set
    extra pip install options by using the PIP_OPTS environment
    variable. This can be used to set the installation to use an
    alternative pypi mirror, or for extra links, etc.

    Change-Id: Ic966bafd04c4c01b3d93851a0e3ec2c1f3312f28
    Closes-Bug: #1751316
    (cherry picked from commit 6b94b21e18181eab1eddb1f3040c430051871a20)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible 16.0.24

This issue was fixed in the openstack/openstack-ansible 16.0.24 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible 17.1.6

This issue was fixed in the openstack/openstack-ansible 17.1.6 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.