Requirements for fuel-qa are failed to install on CI servers: "ImportError: No module named pbr_json"

[Dennis Dmitriev]

Steps to reproduce:

1) Login to any CI server, for example: srv81-bud.infra.mirantis.net

2) Create a virtual env:
   virtualenv --system-site-packages ./devops-venv

3) Activate the virtual env:
  . ./devops-venv/bin/activate

4) Clone the fuel-qa repository from 'master' branch:
  git clone https://github.com/openstack/fuel-qa

5) Install requirements for fuel-qa repository via PyPI:
  pip install -r ./fuel-qa/fuelweb_test/requirements.txt --upgrade

Expected result: All requirements are installed without errors.

Actual result: Requirements are failed to install with the following error:

...
Collecting positional>=1.0.1 (from python-keystoneclient>=0.3.2->-r ./requirements.txt (line 10))
  Downloading positional-1.0.1.tar.gz
...
    Installed /tmp/pip-build-HQmjWt/positional/pbr-1.8.1-py2.7.egg
...
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-HQmjWt/positional/setup.py", line 26, in <module>
...
    ImportError: No module named pbr_json

The cause is that there is a system package installed with the outdated version 0.7.0:

$ dpkg -l|grep pbr
ii python-pbr 0.7.0-0ubuntu2

$ pip list|grep pbr
pbr (0.7.0)

Proposed solution:
================

Remove the package 'python-pbr' from the system:

$ apt-get remove python-pbr

Then, install the requirements for fuel-qa. New version of 'pbr' will be installed via PyPI into the virtual environment:

$ pip list|grep pbr
pbr (0.11.1)

The same issue was already fixed in the https://bugs.launchpad.net/mos/+bug/1513363 for build CI, but there was just an upgrade of the package.
I think that removing the package from the system is better to avoid the same issues in the future.

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/17213
Submitter: Igor Shishkin <email address hidden>
Branch: master

Commit: 1a297cec01f33ea53813d8620648857739072188
Author: Pawel Brzozowski <email address hidden>
Date: Thu Feb 18 15:21:13 2016

remove python-pbr module from manifests

- only required by ISO and package building
- distribution version conflicts with fuel-main

Closes-Bug: #1547035
Change-Id: Ic53d391c57234d979f89b8affa2d9abbf5dbe89b

[Pawel Brzozowski (pbrzozowski)]

@Denis, completely removing of python-pbr seems to brake some ISO and package building. Artur Kaszuba suggested to use virtualenv switch '--no-site-packages' to simply ignore locally installed package. What do you think about it?

[Dennis Dmitriev (ddmitriev)]

@Pawel, yes, 'virtualenv --no-site-packages ...' works fine.

It is about the option in the puppet-manifests/modules/venv/manifests/init.pp , right?

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: master
Change author: Artur Kaszuba <email address hidden>
Review: https://review.fuel-infra.org/17225

[Dennis Dmitriev (ddmitriev)]

There is an another bug : setuptools that is installed on the host, has an issue.
After installing fuel-devops via PyPI, the file bin/dos.py doesn't have the executable flag.

This issue is also fixed by using " --no-site-packages ... "

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/17225
Submitter: Artur Kaszuba <email address hidden>
Branch: master

Commit: e03c46525cfb25ea863d83c0bfd95793975e5a88
Author: Artur Kaszuba <email address hidden>
Date: Thu Feb 18 18:03:10 2016

Change systest virtualenv preparation

This change will disable global python modules in virtualenv
used for systest

Change-Id: I6f6feb3f8cefa5bee6f9430e09a62f707bd74027
Related-Bug: #1547035

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/17237
Submitter: Aleksandra Fedorova <email address hidden>
Branch: master

Commit: ba7096d9952534680f2079d3511613b858bf5bdf
Author: Serhii Boikov <email address hidden>
Date: Thu Feb 25 10:16:56 2016

Fix for custom_system_test_devops3.0

- Options for creation of venv have been changed due to bug in
  system package "python-setuptools".

- Duplicated entries have been removed, add additional ENV variables.

- Option "--no-site-packages" has been removed due to it's deprectaion.
  Now it is default behaivor.

- Added ability to deactivate VENV with all custom variables.
  To avoid confusion with variables if another VENV activated in the same session.

Change-Id: I4aa5189a2e8da5adc9a6a5e3a17d81df27a37b56
Related-Bug:#1547035

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: master
Change author: Serhii Boikov <email address hidden>
Review: https://review.fuel-infra.org/18362

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/18362
Submitter: Aleksandra Fedorova <email address hidden>
Branch: master

Commit: 6017911fe2c95ab685ba6b5f853fcda4854c3159
Author: Serhii Boikov <email address hidden>
Date: Thu Apr 7 11:32:16 2016

When test exits with exit code not 0 job immediately fail.

 - Added workaround so job doesn't stop on test fail
   and erase env in the end of the test.

Change-Id: Idd44d9011a65cc8e1ec242662f5fa210f4803b15
Related-bug:#1547035