Rally install fails on existing deployment creation

Bug #1782372 reported by Gilles Mocellin
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack-Ansible
Fix Released
Undecided
shubham

Bug Description

After deploying my new OpenStack Cloud with OpenStack-Ansible tag 17.0.6, I wanted to know if everything worked.

So I thought about Rally and try to install it.

Reading the doc here https://docs.openstack.org/openstack-ansible-os_rally/queens/
I saw nothing special to configure.

When I launch the playbook os-rally-install.yml, I get :

TASK [os_rally : Setup the Rally deployment] ***********************************************************************************************
Wednesday 18 July 2018 16:26:39 +0200 (0:00:01.626) 0:00:34.712 ********
fatal: [fidcllabs-oct-02_utility_container-d49781cc]: FAILED! => {"changed": true, "cmd": ". /root/openrc\n . /openstack/venvs/rally-17.0.6/
bin/activate\n /openstack/venvs/rally-17.0.6/bin/rally deployment create --fromenv --name=existing", "delta": "0:00:01.174017", "end": "2018
-07-18 14:26:41.096022", "failed": true, "msg": "non-zero return code", "rc": 231, "start": "2018-07-18 14:26:39.922005", "stderr": "", "std
err_lines": [], "stdout": "2018-07-18 14:26:40.977 15491 WARNING rally.common.plugin.discover [-] OpenStack plugins moved to the separate pa
ckage (see https://pypi.org/project/rally-openstack). In-tree OpenStack plugins will be removed from the Rally main package soon.\u001b[00m\
nEnv manager got invalid spec: \n[\"There is no Platform plugin with name: 'existing@openstack'\"]", "stdout_lines": ["2018-07-18 14:26:40.9
77 15491 WARNING rally.common.plugin.discover [-] OpenStack plugins moved to the separate package (see https://pypi.org/project/rally-openst
ack). In-tree OpenStack plugins will be removed from the Rally main package soon.\u001b[00m", "Env manager got invalid spec: ", "[\"There is
 no Platform plugin with name: 'existing@openstack'\"]"]}

No other errors before.
Did I miss something ?

Revision history for this message
Gilles Mocellin (gilles-mocellin) wrote :

Ah, I can add some specificities of my deployment :
- Multi domain, with one backed by Active Directory
- Different internal and public endpoints, managed by haproxy
- openrc points to the Default domain admin account, with internal endpoints
- flat Netwok for external shared network
- VXLAN for private networks with Linux Bridges
- Ceph for glance/nova/cinder with erasure coding pools

Revision history for this message
Guilherme Steinmuller Pimentel (guilhermesp) wrote :
Download full text (15.1 KiB)

I can reproduce it in openstack-ansible all-in-one with tag 17.0.6 using Ubuntu 16.04 Xenial.

Here's a detailed output of the failing task:

TASK [os_rally : Setup the Rally deployment] ***************************************************************************************************************************************************************
task path: /etc/ansible/roles/os_rally/tasks/rally_db_setup.yml:33
Wednesday 18 July 2018 18:03:46 -0300 (0:00:01.374) 0:00:21.409 ********
container_name: "aio1_utility_container-d90e0c62"
physical_host: "aio1"
Container confirmed
Container type "lxc"
Using module file /opt/ansible-runtime/local/lib/python2.7/site-packages/ansible/modules/commands/command.py
container_name: "aio1_utility_container-d90e0c62"
physical_host: "aio1"
Container confirmed
Container type "lxc"
<172.29.236.100> ESTABLISH SSH CONNECTION FOR USER: root
<172.29.236.100> SSH: ansible.cfg set ssh_args: (-C)(-o)(ControlMaster=auto)(-o)(ControlPersist=60s)
<172.29.236.100> SSH: ANSIBLE_HOST_KEY_CHECKING/host_key_checking disabled: (-o)(StrictHostKeyChecking=no)
<172.29.236.100> SSH: ansible_password/ansible_ssh_pass not set: (-o)(KbdInteractiveAuthentication=no)(-o)(PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey)(-o)(PasswordAuthentication=no)
<172.29.236.100> SSH: ANSIBLE_REMOTE_USER/remote_user/ansible_user/user/-u set: (-o)(User=root)
<172.29.236.100> SSH: ANSIBLE_TIMEOUT/timeout set: (-o)(ConnectTimeout=5)
<172.29.236.100> SSH: PlayContext set ssh_extra_args: (-o)(UserKnownHostsFile=/dev/null)(-o)(StrictHostKeyChecking=no)(-o)(ServerAliveInterval=64)(-o)(ServerAliveCountMax=1024)(-o)(Compression=no)(-o)(TCPKeepAlive=yes)(-o)(VerifyHostKeyDNS=no)(-o)(ForwardX11=no)(-o)(ForwardAgent=yes)(-T)
<172.29.236.100> SSH: found only ControlPersist; added ControlPath: (-o)(ControlPath=/root/.ansible/cp/53b4b74265)
<172.29.236.100> SSH: EXEC ssh -vvv -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=root -o ConnectTimeout=5 -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no -o ServerAliveInterval=64 -o ServerAliveCountMax=1024 -o Compression=no -o TCPKeepAlive=yes -o VerifyHostKeyDNS=no -o ForwardX11=no -o ForwardAgent=yes -T -o ControlPath=/root/.ansible/cp/53b4b74265 172.29.236.100 'lxc-attach --clear-env --name aio1_utility_container-d90e0c62 -- su - root -c '"'"'/bin/sh -c '"'"'"'"'"'"'"'"'/usr/bin/python && sleep 0'"'"'"'"'"'"'"'"''"'"''
<172.29.236.100> (1, '\n{"changed": true, "end": "2018-07-18 18:03:47.509716", "stdout": "2018-07-18 18:03:47.426 3596 WARNING rally.common.plugin.discover [-] OpenStack plugins moved to the separate package (see https://pypi.org/project/rally-openstack). In-tree OpenStack plugins will be removed from the Rally main package soon.\\u001b[00m\\nEnv manager got invalid spec: \\n[\\"There is no Platform plugin with name: \'existing@openstack\'\\"]", "cmd": ". /root/openrc\\n . /openstack/venvs/rally-17.0.6/bin/activate\\n /openstack/venvs/rally-17.0.6/bin/rally deployment create --fromenv --nam...

Revision history for this message
Gilles Mocellin (gilles-mocellin) wrote :

Hi !

Yes, the command fails also xhen launched manually in the utility container :

root@fidcllabs-oct-02:~# lxc-attach -n $(lxc-ls -1 | grep utility)
root@fidcllabs-oct-02-utility-container-d49781cc:~# . /root/openrc
root@fidcllabs-oct-02-utility-container-d49781cc:~# . /openstack/venvs/rally-17.0.6/bin/activate
(rally-17.0.6) root@fidcllabs-oct-02-utility-container-d49781cc:~# /openstack/venvs/rally-17.0.6/bin/rally deployment create --fromenv --name existing
2018-07-19 07:26:37.789 16287 WARNING rally.common.plugin.discover [-] OpenStack plugins moved to the separate package (see https://pypi.org/project/rally-openstack). In-tree OpenStack plugins will be removed from the Rally main package soon.
Env manager got invalid spec:
["There is no Platform plugin with name: 'existing@openstack'"]
(rally-17.0.6) root@fidcllabs-oct-02-utility-container-d49781cc:~#

So I think there is some modifications in rally itself, plugins removed.
As the error says, I've tried to install with pip rally-openstack in the venv, but there's a dependancy problem, either on 1.2.0, 1.1.0 and 1.0.0 versions of rally-openstack, due to a dependence on kubernetes :

(rally-17.0.6) root@fidcllabs-oct-02-utility-container-d49781cc:~# pip install rally-openstack==1.0.0
[...]
Requirement already up-to-date: certifi>=14.05.14 in /openstack/venvs/rally-17.0.6/lib/python2.7/site-packages (from kubernetes>1.0.0->rally-openstack==1.0.0)
Collecting websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 (from kubernetes>1.0.0->rally-openstack==1.0.0)
  Could not find a version that satisfies the requirement websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 (from kubernetes>1.0.0->rally-openstack==1.0.0) (from versions: 0.40.0)
No matching distribution found for websocket-client!=0.40.0,!=0.41.*,!=0.42.*,>=0.32.0 (from kubernetes>1.0.0->rally-openstack==1.0.0)

Strangely, pip cannot find anything but le 0.40.0 release for websocket-client.
There are more (last is 0.48.0), as you can see here :
https://pypi.org/project/websocket-client/#history

I cannot install 0.48.0 manually :
(rally-17.0.6) root@fidcllabs-oct-02-utility-container-d49781cc:~# pip install websocket-client==0.48.0
Collecting websocket-client==0.48.0
  Could not find a version that satisfies the requirement websocket-client==0.48.0 (from versions: 0.40.0)
No matching distribution found for websocket-client==0.48.0

Problem with the local pip cache/repo ?

Revision history for this message
Gilles Mocellin (gilles-mocellin) wrote :

Sorry, I didn't see the end of your post Guilherme.

So, strange, you have been able to install via pip rally-openstack, not me.

Revision history for this message
Guilherme Steinmuller Pimentel (guilhermesp) wrote :

Gilles, which OS are you using? It is quite odd these dependencies....

Just to complement this bug, here's the change log from rally:

https://github.com/openstack/rally/blob/master/CHANGELOG.rst#100---2018-06-20

Revision history for this message
Gilles Mocellin (gilles-mocellin) wrote :

Ubuntu 16.04 : I followed the deployment guide.
But Since I started, I switch from OSA 17.0.5 to 17.0.6.
- git pull
- git checkout 17.0.6
- scripts/bootstrap-ansible.sh
- openstack-ansible setup-everything.yml

Changed in openstack-ansible:
status: New → Confirmed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible-os_rally (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/584089

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible-os_rally (master)

Fix proposed to branch: master
Review: https://review.openstack.org/584096

Changed in openstack-ansible:
assignee: nobody → Guilherme Steinmuller Pimentel (guilhermesp)
status: Confirmed → In Progress
Revision history for this message
Gilles Mocellin (gilles-mocellin) wrote :

I manage to pass my dependencies problems by launching the repo-build.yml playbook.
So, I suspect we can only install from the repo, there is no failback to Internet with pip.

Revision history for this message
Jesse Pretorius (jesse-pretorius) wrote :

Yes, currently rally is installed unconstrained which is resulting in you getting a mixed bag. We've solved this is master and will need to port the patches back:

https://review.openstack.org/#/q/topic:constrain-rally-builds+(status:open+OR+status:merged)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible (stable/pike)

Fix proposed to branch: stable/pike
Review: https://review.openstack.org/585166

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/585167

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible-os_rally (stable/queens)

Fix proposed to branch: stable/queens
Review: https://review.openstack.org/585170

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to openstack-ansible-os_rally (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/585176

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible-os_rally (master)

Reviewed: https://review.openstack.org/584096
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_rally/commit/?id=3da7a036d9e3738b8802470127ccb0f20da3882b
Submitter: Zuul
Branch: master

commit 3da7a036d9e3738b8802470127ccb0f20da3882b
Author: Guilherme Steinmüller <email address hidden>
Date: Thu Jul 19 17:20:25 2018 -0300

    Change rally pip package

    This is motivated by the recent change of rally
    project [1] which states that In-tree OpenStack
    plugins will be removed from the Rally main package soon.

    Also, we do not have constraint for it, we need to pin it
    to keep consistent build.

    [1] https://github.com/openstack/rally/blob/master/CHANGELOG.rst#100---2018-06-20

    Change-Id: I9121867301fc69ba6623110fa6e54c2da0b5ae2c
    Closes-Bug: 1782372

Changed in openstack-ansible:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to openstack-ansible-os_rally (stable/queens)

Reviewed: https://review.openstack.org/585170
Committed: https://git.openstack.org/cgit/openstack/openstack-ansible-os_rally/commit/?id=6baad96a484172794ffe7c7c637cd3562bcc061a
Submitter: Zuul
Branch: stable/queens

commit 6baad96a484172794ffe7c7c637cd3562bcc061a
Author: Jesse Pretorius <email address hidden>
Date: Tue May 29 12:11:41 2018 +0100

    Implement pypi-based install and package pinning for non u-c packages

    Rally does not adhere to the global requirements process, so its
    requirements extend beyond those managed by upper-constraints, and
    sometimes are incompatible with upper constraints.

    We have no need to track their development from git commit to git
    commit, and can rather rely on stable releases which we verify are
    working with the upper constraints whenever we do a pinning bump.

    This keeps things simpler for us, and also allows the repo build
    process in the integrated build to do a constrained venv build
    instead of doing it unconstrained and breaking stable branch builds
    down the line.

    Closes-Bug: 1782372
    Needed-By: https://review.openstack.org/585165
    Change-Id: If3d0120cf6f6f78463cf7805bc046382c3b6d44e
    (cherry picked from commit 321fb7e69aa4d87b07b1725f37adf0e1b596fcf5)

tags: added: in-stable-queens
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on openstack-ansible (stable/pike)

Change abandoned by Jesse Pretorius (odyssey4me) (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/585166

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on openstack-ansible (stable/queens)

Change abandoned by Jesse Pretorius (odyssey4me) (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/585165

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on openstack-ansible (stable/ocata)

Change abandoned by Jesse Pretorius (odyssey4me) (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/585167

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on openstack-ansible-os_rally (stable/ocata)

Change abandoned by Jesse Pretorius (odyssey4me) (<email address hidden>) on branch: stable/ocata
Review: https://review.openstack.org/585176

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on openstack-ansible-os_rally (stable/pike)

Change abandoned by Jesse Pretorius (odyssey4me) (<email address hidden>) on branch: stable/pike
Review: https://review.openstack.org/585175

Revision history for this message
shubham (shubham-ranjan) wrote :

hi,
I installed Rally lately. Even i faced the same issue.
please refer this link : http://lists.openstack.org/pipermail/openstack-dev/2018-August/133070.html

In case of any issue feel free to ask.

Changed in openstack-ansible:
assignee: Guilherme Steinmuller Pimentel (guilhermesp) → shubham (shubham-ranjan)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on openstack-ansible-os_rally (stable/queens)

Change abandoned by Mohammed Naser (<email address hidden>) on branch: stable/queens
Review: https://review.openstack.org/584089
Reason: addressed in https://review.openstack.org/#/q/topic:bug/1782372+(status:open+OR+status:merged)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_rally queens-eol

This issue was fixed in the openstack/openstack-ansible-os_rally queens-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_rally rocky-eol

This issue was fixed in the openstack/openstack-ansible-os_rally rocky-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_rally stein-eol

This issue was fixed in the openstack/openstack-ansible-os_rally stein-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_rally train-eol

This issue was fixed in the openstack/openstack-ansible-os_rally train-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_rally ussuri-eol

This issue was fixed in the openstack/openstack-ansible-os_rally ussuri-eol release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_rally yoga-eom

This issue was fixed in the openstack/openstack-ansible-os_rally yoga-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_rally victoria-eom

This issue was fixed in the openstack/openstack-ansible-os_rally victoria-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_rally wallaby-eom

This issue was fixed in the openstack/openstack-ansible-os_rally wallaby-eom release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/openstack-ansible-os_rally xena-eom

This issue was fixed in the openstack/openstack-ansible-os_rally xena-eom release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.