setup.py errors and docs won't build without sphinx

[Ken Pepple]

trying to build docs or run setup.py fails without sphinx, which isn't required by tools/pip-requires nor our wiki instructions ...

(.nova-venv)k:trunk kpepple$ python setup.py install
Traceback (most recent call last):
  File "setup.py", line 24, in <module>
    from sphinx.setup_command import BuildDoc
ImportError: No module named sphinx.setup_command

Asking to just include this as part of pip-requires so that it just works.

[Brian Schott (bfschott)]

Our hudson server still had trouble building nova trunk with clean virtual environment "./run_tests.sh -f". I hacked in the following into our hpc_trunk branch as a workaround. It looks like the dependencies are getting triggered out of order.

bschott@ironman:~/Source/nova/hpc-trunk$ bzr diff -r 465 tools/install_venv.py
=== modified file 'tools/install_venv.py'
--- tools/install_venv.py 2011-01-11 18:11:30 +0000
+++ tools/install_venv.py 2011-01-24 16:23:31 +0000
@@ -93,6 +93,17 @@
   # get it in stalled in the right order
   run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, 'greenlet'],
               redirect_output=False)
+ #
+ # BFS workaround for glance build error, remove when fixed
+ #
+ run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, 'sphinx'],
+ redirect_output=False)
+ run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, 'argparse'],
+ redirect_output=False)
+ #
+ # BFS end workaround
+ #
+
   run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, '-r', PIP_REQUIRES],
               redirect_output=False)
   run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, TWISTED_NOVA],

--- log ---

bschott@ironman:~/Source/nova/trunk$ bzr pull
Using saved parent location: bzr+ssh://bazaar.launchpad.net/%2Bbranch/nova/
 M nova/virt/xenapi/vmops.py
 M tools/pip-requires
All changes applied successfully.
Now on revision 604.
bschott@ironman:~/Source/nova/trunk$ ls -al
total 144
drwxr-xr-x 30 bschott staff 1020 Jan 22 12:29 .
drwxr-xr-x 9 bschott staff 306 Jan 24 11:33 ..
drwxr-xr-x 7 bschott staff 238 Jan 11 11:24 .bzr
-rw-r--r-- 1 bschott staff 200 Jan 11 11:24 .bzrignore
-rw-r--r-- 1 bschott staff 125 Jan 11 11:24 .gitignore
-rw-r--r-- 1 bschott staff 1646 Jan 22 12:29 .mailmap
-rw-r--r-- 1 bschott staff 2072 Jan 22 12:29 Authors
drwxr-xr-x 11 bschott staff 374 Jan 11 11:24 CA
-rw-r--r-- 1 bschott staff 1345 Jan 11 11:24 HACKING
-rw-r--r-- 1 bschott staff 10143 Jan 11 11:24 LICENSE
-rw-r--r-- 1 bschott staff 1070 Jan 22 12:29 MANIFEST.in
-rw-r--r-- 1 bschott staff 837 Jan 22 12:29 README
-rw-r--r-- 1 bschott staff 17 Jan 11 11:24 babel.cfg
drwxr-xr-x 19 bschott staff 646 Jan 22 12:29 bin
-rwxr-xr-x 1 bschott staff 787 Jan 11 11:24 builddeb.sh
drwxr-xr-x 3 bschott staff 102 Jan 11 11:24 bzrplugins
drwxr-xr-x 5 bschott staff 170 Jan 22 12:29 contrib
drwxr-xr-x 9 bschott staff 306 Jan 11 11:24 doc
drwxr-xr-x 3 bschott staff 102 Jan 22 12:29 etc
-rw-r--r-- 1 bschott staff 41 Jan 22 12:29 krm_mapping.json.sample
drwxr-xr-x 3 bschott staff 102 Jan 11 11:24 locale
drwxr-xr-x 33 bschott staff 1122 Jan 22 12:29 nova
drwxr-xr-x 3 bschott staff 102 Jan 11 11:24 plugins
-rw-r--r-- 1 bschott staff 829 Jan 11 11:24 pylintrc
-rw-r--r-- 1 bschott staff 2240 Jan 22 12:29 run_tests.py
-rwxr-xr-x 1 bschott staff 2205 J...

[Jay Pipes (jaypipes)]

I'm wondering why, since Nova also requires sphinx to build, there is a problem when installing Glance but not a problem when installing Nova? There's GOT to be a better solution than requiring Sphinx in order to just build Glance...

[Vish Ishaya (vishvananda)]

It got added to nova pip-requires as well.

Vish

On Jan 24, 2011, at 10:12 AM, Jay Pipes wrote:

> I'm wondering why, since Nova also requires sphinx to build, there is a
> problem when installing Glance but not a problem when installing Nova?
> There's GOT to be a better solution than requiring Sphinx in order to
> just build Glance...
>
> --
> You received this bug notification because you are a member of Nova Bug
> Team, which is subscribed to OpenStack Compute (nova).
> https://bugs.launchpad.net/bugs/706683
>
> Title:
> setup.py errors and docs won't build without sphinx
>
> Status in OpenStack Compute (Nova):
> Fix Committed
>
> Bug description:
> trying to build docs or run setup.py fails without sphinx, which isn't
> required by tools/pip-requires nor our wiki instructions ...
>
> (.nova-venv)k:trunk kpepple$ python setup.py install
> Traceback (most recent call last):
> File "setup.py", line 24, in <module>
> from sphinx.setup_command import BuildDoc
> ImportError: No module named sphinx.setup_command
>
> Asking to just include this as part of pip-requires so that it just
> works.
>
>

[Ken Pepple (ken-pepple)]

I am not seeing this sphinx problem after the pip-requires addition ... i can `run_test.sh -f` on both glance from glance/trunk (revno 50) and nova/trunk (rev 604) without issue ... however if this is a problem elsewhere, it might be prudent include the run_command(['tools/with_venv.sh', 'pip', 'install', '-E', venv, 'sphinx'] but probably not the argvparse as it doesn't error out

[Ken Pepple (ken-pepple)]

I think I've found the bug here: the tools/pip-requires file in glance-0.1.3.tar.gz is not the same as the tools/pip-requires in glance/trunk (revno: 50). The file in the glance dist tarball is still using the pre-revno 49 version of the file.

wil contact jay.pipes and/or file glance bug to see what is amiss in the dist process ...

[John Dewey (retr0h)]

I am new to nova and wanted to pick up a bug to work on. I am working on getting my development environment setup.

I am on a Mac using Python 2.6.6 following the Mac instructions at : http://nova.openstack.org/devref/development.environment.html

I continually run into the problem reported in the original report.

It looks as though the PIP_REQUIRES are evaluated in a different order than specified in the file (at least on my system),
and installs glance before sphinx _or_ as Ken pointed out will pip resolve glance's dependencies on install?

I am on irc if you want details on my environment.

[John Dewey (retr0h)]

This may be a semi convoluted test, however it does indicate pip doesn't evaluate the dependencies in the order listed.

$:~% pip install -r pip_deps
Downloading/unpacking zzyzz (from -r pip_deps (line 4))
  Could not find any downloads that satisfy the requirement zzyzz (from -r pip_deps (line 4))
No distributions at all found for zzyzz (from -r pip_deps (line 4))
Storing complete log in /Users/jdewey/.pip/pip.log

$:~% cat pip_deps
foo
bar
baz
zzyzz

[Brian Schott (bfschott)]

Ken, you're right. We figured the change hadn't propagated out to the tarball yet. So, pip doesn't know that glance requires sphinx. Wasn't sure about argparse from your earlier patch, so I just included it in my hack. I pushed this into our team hpc-trunk branch, just to keep our local hudson server happy.

John, my fix above solves the problem in a brute-force fashion if you want to just hack your tools/install_venv.py file temporarily. The problem is as Ken describes. There is no guarantee on build order because pip traverses the dependency tree and the glance package is missing sphinx dependency. Nothing to fix in nova or pip for that matter.

If you want to tackle a mac-related bug, I've given up on my circular dependency on OpenSSL 1.0a loaded via macports. I just let the X509 unit tests fail on mac for now. Not sure the details. It has been around for a long time there is NO WAY to easily revert to an older package via macports. So, basically, this bug breaks most macports package dependencies on OpenSSL.
https://bugs.launchpad.net/nova/+bug/615518

[Jay Pipes (jaypipes)]

Hey guys, working on cutting a new tarball for Glance as we speak...sorry for the delay. Reference: https://bugs.launchpad.net/glance/+bug/707180