simplejson version error while upgrading from 9.0.11 to 10.1.14

[Shane Cunningham]

Upgrading 9.0.11 to 10.1.14

The upgrade stops with the following error.

TASK: [heat_domain_user | Create heat domain] *********************************
failed: [infra01_heat_apis_container-9536d6a6] => {"changed": true, "cmd": ". /root/openrc\nopenstack --os-identity-api-version=3 \\\n --os-auth-url=http://172.29.236.10:5000/v3 \\\n --os-user-domain-name=Default \\\n --os-project-domain-name=Default \\\n domain create --or-show heat --description \"Owns users and projects created by heat\"", "delta": "0:00:00.595825", "end": "2015-09-23 21:45:24.796940", "rc": 1, "start": "2015-09-23 21:45:24.201115"}
stderr: ERROR: openstackclient.shell Exception raised: simplejson 3.6.5 is installed but simplejson>=2.2.0,<=3.3.1 is required by ['python-neutronclient']

root@infra01_heat_apis_container-9536d6a6:/# pip list | grep simple
simplejson (3.6.5)

Uninstalling simplejson and reinstalling inside the container allows setup-everything.yml to finish.

root@infra01_heat_apis_container-9536d6a6:/# pip uninstall simplejson
  Successfully uninstalled simplejson-3.6.5

oot@infra01_heat_apis_container-9536d6a6:/# pip install simplejson
Ignoring indexes: https://pypi.python.org/simple
Collecting simplejson
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:79: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. For more information, see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
  InsecurePlatformWarning
  Downloading https://mirror.rackspace.com/rackspaceprivatecloud/python_packages/10.1.14/simplejson-3.3.1-cp27-none-linux_x86_64.whl (103kB)
    100% |████████████████████████████████| 106kB 1.1MB/s
Installing collected packages: simplejson
Successfully installed simplejson-3.3.1

In http://mirror.rackspace.com/rackspaceprivatecloud/python_packages/9.0.11/ and http://mirror.rackspace.com/rackspaceprivatecloud/python_packages/icehouse/ it has simplejson-3.6.5-cp27-none-linux_x86_64.whl. In http://mirror.rackspace.com/rackspaceprivatecloud/python_packages/10.1.14/ it has simplejson-3.3.1-cp27-none-linux_x86_64.whl.

After 10.1.14 finished deploying, some containers still have 3.6.5, but others were (downgraded?) 3.3.1. I didn't get versions before the upgrade but I would think they were all 3.6.5. The heat_api_container is the only container that threw an error.

root@infra01:/opt/openstack-ansible/rpc_deployment# ansible -m shell all_containers -a "pip list | grep simplejson" | grep simplejson
simplejson (3.6.5)
simplejson (3.6.5)
simplejson (3.3.1)
simplejson (3.3.1)
simplejson (3.3.1)
simplejson (3.3.1)
simplejson (3.3.1)
simplejson (3.6.5)
simplejson (3.3.1)
simplejson (3.3.1)
simplejson (3.6.5)
simplejson (3.6.5)
simplejson (3.3.1)
simplejson (3.3.1)
simplejson (3.3.1)
simplejson (3.3.1)

Not sure there's a solution for this, at the very least a documentation update similar to http://docs.rackspace.com/rpc/api/v10/bk-rpc-v10-releasenotes/content/rpc-upgrade-v9-v10-prereq.html.

[Bjoern (bjoern-t)]

This is redundant to the request to downgrade pip wheels in needed since this library is not the only issue.
We have currently eventlet, oslo.messaging and requests and we need to update the documentation to manually downgrade this library prior to the upgrade. Internally I have instructions.

[Shane Cunningham (appprod0)]

Correct, it is redundant but wanted to file this to document simplejson as I had not seen documentation for downgrading that package.

[Jesse Pretorius (jesse-pretorius)]

The ability to for the re-installation of the pip wheels (ensuring that the upgrade/downgrade accordingly between tags) is already available in Juno by executing, for example: openstack-ansible -e pip_install_options="--force-reinstall" setup-openstack.yml

This will be added to Kilo+ through https://review.openstack.org/229383

[Jesse Pretorius (jesse-pretorius)]

We're just going to do some testing to confirm that it works as expected across all the required version changes.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/229383
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=2c19010ff9a936c5d0ad3d72e2015308c105ff0f
Submitter: Jenkins
Branch: master

commit 2c19010ff9a936c5d0ad3d72e2015308c105ff0f
Author: Jesse Pretorius <email address hidden>
Date: Wed Sep 30 12:48:13 2015 +0100

    Allow pip extra arguments to be passed to playbooks

    This patch adds the variable 'pip_install_options' which is passed to the pip
    install module as extra arguments in order to allow the use of options like
    '--force-reinstall' when executing playbooks.

    eg: openstack-ansible -e pip_install_options="--force-reinstall" \
            setup-openstack.yml

    This is required due to constant upstream changes in dependencies which
    result in python wheel version upgrades and downgrades between tagged
    versions of openstack-ansible.

    The intention is that this can be used whenever a deployer switches between
    tags for both upgrades and downgrades.

    DocImpact
    Closes-Bug: #1489251
    Closes-Bug: #1499451
    Related-Bug: #1501114
    Change-Id: I996185e009a4c4af4f23798619bdbd0d490360c9

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/229542
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible/commit/?id=f5e3b1171f41ff10b60725c7c30d13a18f3156ca
Submitter: Jenkins
Branch: kilo

commit f5e3b1171f41ff10b60725c7c30d13a18f3156ca
Author: Jesse Pretorius <email address hidden>
Date: Wed Sep 30 12:48:13 2015 +0100

    Allow pip extra arguments to be passed to playbooks

    This patch adds the variable 'pip_install_options' which is passed to the pip
    install module as extra arguments in order to allow the use of options like
    '--force-reinstall' when executing playbooks.

    eg: openstack-ansible -e pip_install_options="--force-reinstall" \
            setup-openstack.yml

    This is required due to constant upstream changes in dependencies which
    result in python wheel version upgrades and downgrades between tagged
    versions of openstack-ansible.

    The intention is that this can be used whenever a deployer switches between
    tags for both upgrades and downgrades.

    DocImpact
    Closes-Bug: #1489251
    Closes-Bug: #1499451
    Related-Bug: #1501114
    Change-Id: I996185e009a4c4af4f23798619bdbd0d490360c9
    (cherry picked from commit 2c19010ff9a936c5d0ad3d72e2015308c105ff0f)

[Jesse Pretorius (jesse-pretorius)]

Juno has the option of using pip_install_options when running the playbooks and this should be used. Due to this functionality being available, this bug is being closed as invalid for Juno.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.11 release.

[Doug Hellmann (doug-hellmann)]

This issue was fixed in the openstack/openstack-ansible 11.2.12 release.

[Davanum Srinivas (DIMS) (dims-v)]

This issue was fixed in the openstack/openstack-ansible 11.2.14 release.