run_tests always requires selenium (even without '--with-selenium')

[Thomas Goirand]

When packaging horizon, distros may use ./run_tests.sh -N and supply the required python packages externally. If selenium is not present on the system and run_tests.sh is invoked without selenium (i.e. the arguments --with-selenium and --only-selenium are NOT used), it will fail.

Below is the Python stack dump when building Horizon Juno RC1 in Debian:

======================================================================
ERROR: Failure: ImportError (No module named selenium.common)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/loader.py", line 414, in loadTestsFromName
    addr.filename, addr.module)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/usr/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/zigo/sources/openstack/juno/horizon/build-area/horizon-2014.2~rc1/horizon/test/webdriver.py", line 24, in <module>
    from selenium.common import exceptions as selenium_exceptions
ImportError: No module named selenium.common

Please make the selenium import optional (with an except ImportError: to catch it).

[Gary W. Smith (gary-w-smith)]

Thanks for the bug report. Please help by clarifying:
1. What command / options are you using that cause this output?
2. Selenium is licensed under apache 2.0 (http://www.seleniumhq.org/about/license.jsp), the same as OpenStack, so why is the license an issue?
3. As far as I can see, Selenium is only used in the test code, which I would not expect to be distributed with the final product. So even if it was non-free, why should this be an issue?

[Matthias Runge (mrunge)]

Gary,
when not installing python-selenium, and executing run_tests.sh -N this happens regularly.

Python-selenium distributes a pre-compiled binary only available for x86_64 and i686. This is a blocker for some distros.

[Gary W. Smith (gary-w-smith)]

Thanks for the clarification.

[Gary W. Smith (gary-w-smith)]

One slightly subversive way to replicate this bug while using a virtual env is:
1. Use run_tests.sh to build the virtual environment (e.g. ./run_tests.sh -p)
2. rm -rf .venv/lib/python2.7/site-packages/selenium*
3. ./run_tests.sh

Note that to recover from this altered virtual environment, you can re-install selenium with:
4 ./tools/with_venv.sh pip install selenium

[Thomas Goirand (thomas-goirand)]

Yes, the issue is at build time, when running the unit tests. The unit tests shall, in no condition, impose the use of selenium, which indeed, is non-free because it contains some not-built binary blobs (the browser plug-in .xpi).

[Doug Fish (drfish)]

I think this is going to be significant for distros. I've upped the severity and marked with the juno-rc-potential tag

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/126777

[Thomas Goirand (thomas-goirand)]

Gary W. Smith, I just wanted to explain a bit more.

FYI, distribution *do* run the unit tests of every OpenStack components, to make sure that they do work in the environment they will be installed in. In Debian, there's also the "autopkgtest" thing, in which tests are run on installed packages. In such environment, it would make sense to ship the unit tests with the package.

Because of that, it's important that all unit tests are running correctly, without the need of non-free components like (unfortunately) Selenium. One way to fix the (rather bad) situation with Selenium would be to have its browser pluggin built together with the rest of the code.

FYI, the non-free component of python-selenium is in: py/selenium/webdriver/firefox/webdriver.xpi. It's easy to uncompress, as .xpi are in fact zip files. Once uncompressed, you'll see that it contains a number of non-free components, like Windows pre-built DLL files, or Linux pre-built .so files. These would need to be created at build time.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/126777
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=aad4565d9f64a97f10a9528f2c0efc5039343f2c
Submitter: Jenkins
Branch: master

commit aad4565d9f64a97f10a9528f2c0efc5039343f2c
Author: Gary W. Smith <email address hidden>
Date: Wed Oct 8 09:22:53 2014 -0700

    Remove selenium dependency when not using selenium tests

    Closes-Bug: 1377372
    Change-Id: I6a493989d7280eaa2a1c999a9d1be4365aa77d52

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/juno
Review: https://review.openstack.org/130945

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/130945
Committed: https://git.openstack.org/cgit/openstack/horizon/commit/?id=697888b94ac628825aa9388c427a5613bab3a78f
Submitter: Jenkins
Branch: stable/juno

commit 697888b94ac628825aa9388c427a5613bab3a78f
Author: Gary W. Smith <email address hidden>
Date: Wed Oct 8 09:22:53 2014 -0700

    Remove selenium dependency when not using selenium tests

    Closes-Bug: 1377372
    Change-Id: I6a493989d7280eaa2a1c999a9d1be4365aa77d52
    (cherry picked from commit aad4565d9f64a97f10a9528f2c0efc5039343f2c)