from migrate.versioning import exceptions fails with recent venv

[Brian Schott]

No virtual environment found...create one? (Y/n)
done.

....

======================================================================
ERROR: <nose.suite.ContextSuite context=nova.tests>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/bschott/source/nova/hpc-trunk/.nova-venv/lib/python2.6/site-packages/nose/suite.py", line 208, in run
    self.setUp()
  File "/home/bschott/source/nova/hpc-trunk/.nova-venv/lib/python2.6/site-packages/nose/suite.py", line 291, in setUp
    self.setupContext(ancestor)
  File "/home/bschott/source/nova/hpc-trunk/.nova-venv/lib/python2.6/site-packages/nose/suite.py", line 314, in setupContext
    try_run(context, names)
  File "/home/bschott/source/nova/hpc-trunk/.nova-venv/lib/python2.6/site-packages/nose/util.py", line 478, in try_run
    return func()
  File "/home/bschott/source/nova/hpc-trunk/nova/tests/__init__.py", line 41, in setup
    migration.db_sync()
  File "/home/bschott/source/nova/hpc-trunk/nova/db/migration.py", line 33, in db_sync
    return IMPL.db_sync(version=version)
  File "/home/bschott/source/nova/hpc-trunk/nova/utils.py", line 367, in __getattr__
    backend = self.__get_backend()
  File "/home/bschott/source/nova/hpc-trunk/nova/utils.py", line 362, in __get_backend
    self.__backend = __import__(name, None, None, fromlist)
  File "/home/bschott/source/nova/hpc-trunk/nova/db/sqlalchemy/migration.py", line 25, in <module>
    from migrate.versioning import exceptions as versioning_exceptions
ImportError: cannot import name exceptions

[Brian Schott (bfschott)]

Ubuntu 10.4.1

Package: python
Version: 2.6.5-0ubuntu1

Package: python-migrate
Version: 0.6-4~lucid1

Package: python-sqlalchemy
Version: 0.6.3-1ubuntu0~lucid1

migrate/versioning/exceptions doesn't exist.
bschott@island100:~/source/nova/trunk/.nova-venv/lib/python2.6/site-packages/migrate/versioning$

bschott@island100:~/source/nova/trunk/.nova-venv/lib/python2.6/site-packages/migrate/versioning$ ls -l
total 220
-rw-r--r-- 1 bschott bschott 12986 2011-02-11 17:52 api.py
-rw-r--r-- 1 bschott bschott 14694 2011-02-11 17:54 api.pyc
-rw-r--r-- 1 bschott bschott 669 2011-02-11 17:52 cfgparse.py
-rw-r--r-- 1 bschott bschott 1671 2011-02-11 17:54 cfgparse.pyc
-rw-r--r-- 1 bschott bschott 339 2011-02-11 17:52 config.py
-rw-r--r-- 1 bschott bschott 490 2011-02-11 17:54 config.pyc
-rw-r--r-- 1 bschott bschott 9732 2011-02-11 17:52 genmodel.py
-rw-r--r-- 1 bschott bschott 8604 2011-02-11 17:54 genmodel.pyc
-rw-r--r-- 1 bschott bschott 161 2011-02-11 17:52 __init__.py
-rw-r--r-- 1 bschott bschott 365 2011-02-11 17:54 __init__.pyc
-rw-r--r-- 1 bschott bschott 3123 2011-02-11 17:52 migrate_repository.py
-rw-r--r-- 1 bschott bschott 3994 2011-02-11 17:54 migrate_repository.pyc
-rw-r--r-- 1 bschott bschott 2059 2011-02-11 17:52 pathed.py
-rw-r--r-- 1 bschott bschott 3645 2011-02-11 17:54 pathed.pyc
-rw-r--r-- 1 bschott bschott 7399 2011-02-11 17:52 repository.py
-rw-r--r-- 1 bschott bschott 10801 2011-02-11 17:54 repository.pyc
-rw-r--r-- 1 bschott bschott 8655 2011-02-11 17:52 schemadiff.py
-rw-r--r-- 1 bschott bschott 8956 2011-02-11 17:54 schemadiff.pyc
-rw-r--r-- 1 bschott bschott 7337 2011-02-11 17:52 schema.py
-rw-r--r-- 1 bschott bschott 8497 2011-02-11 17:54 schema.pyc
drwxr-xr-x 2 bschott bschott 4096 2011-02-11 17:54 script
-rw-r--r-- 1 bschott bschott 6460 2011-02-11 17:52 shell.py
-rw-r--r-- 1 bschott bschott 6678 2011-02-11 17:54 shell.pyc
-rw-r--r-- 1 bschott bschott 2905 2011-02-11 17:52 template.py
-rw-r--r-- 1 bschott bschott 5642 2011-02-11 17:54 template.pyc
drwxr-xr-x 6 bschott bschott 4096 2011-02-11 17:54 templates
drwxr-xr-x 2 bschott bschott 4096 2011-02-11 17:54 util
-rw-r--r-- 1 bschott bschott 6615 2011-02-11 17:52 version.py
-rw-r--r-- 1 bschott bschott 10180 2011-02-11 17:54 version.pyc

[Brian Schott (bfschott)]

Uploaded:
https://code.launchpad.net/~usc-isi/nova/lp-717467
Not sure how this behaves on Ubuntu 10.10 or other distributions but fixes the issue or 10.4.

[justinsb (justin-fathomdb)]

Ubuntu 10.10. Same problem; fix in branch solves the problem for me - thanks!

[Thierry Carrez (ttx)]

From Vish on the merge proposal:

I believe this is actually an issue because python-migrate recently changed. I think a better fix would be to actually put the (older) version of python-migrate into pip-requires. Your fix will break running the tests without a virtualenv unless we create custom python-migrate packages.

Adapting description.

[Jay Pipes (jaypipes)]

We should fix this without having to have a specific version of python-migrate.

Something like this should fix it on both systems and not stick us on an old version of python-migrate:

try:
    from migrate.versioning import exceptions as versioning_exceptions
except ImportError:
    try:
        # python-migration changed location of exceptions after 1.6.3
        # See LP Bug #717467
        from migrate import exceptions as versioning_exceptions
    except ImportError:
        sys.exit(_("python-migrate is not installed. Exiting."))

[Brian Lamar (blamar)]

"not stick us on an old version of python-migrate"

Isn't a key point of having something like a "pip-requires" file to allow the project to rely on always having a stable API to work against for all external libraries? Currently "pip-requires" has a ton of version-specific libraries and I wouldn't want to have to support N versions of M libraries.

Shouldn't the fix just be to specify "sqlalchemy-migrate==0.6" and be done with it until a bug with that library affects us or we need a new feature from it?

[Jay Pipes (jaypipes)]

"Shouldn't the fix just be to specify "sqlalchemy-migrate==0.6" and be done with it until a bug with that library affects us or we need a new feature from it?"

No. pip-requires is only for developers to use when developing Nova in a virtualenv. It isn't related to distribution and packaging and contains requirements for libraries (such as sphinx) that are *not* required when installing Nova.

Having specific versions in pip-requires is a mistake IMHO.

We have similar fixes to what I proposed above in both Swift and Glance for such things like the python-urlparse library parsing URIs differently depending on the version of the library installed.

In short, I think it's better to fix the software to run properly for multiple NON-buggy versions of a software library than to require a specific version of a library when another perfectly-valid version of the library is commonly installed on machines.

Just my 2 cents,
jay

[Vish Ishaya (vishvananda)]

+1 To Jay. If we have an easy way to support both we should. Otherwise, nova will suddenly break when ubuntu packages the new version
of sqlalchemy-migrate.

Vish

On Feb 14, 2011, at 11:24 AM, Jay Pipes wrote:

> "Shouldn't the fix just be to specify "sqlalchemy-migrate==0.6" and be
> done with it until a bug with that library affects us or we need a new
> feature from it?"
>
> No. pip-requires is only for developers to use when developing Nova in a
> virtualenv. It isn't related to distribution and packaging and contains
> requirements for libraries (such as sphinx) that are *not* required when
> installing Nova.
>
> Having specific versions in pip-requires is a mistake IMHO.
>
> We have similar fixes to what I proposed above in both Swift and Glance
> for such things like the python-urlparse library parsing URIs
> differently depending on the version of the library installed.
>
> In short, I think it's better to fix the software to run properly for
> multiple NON-buggy versions of a software library than to require a
> specific version of a library when another perfectly-valid version of
> the library is commonly installed on machines.
>
> Just my 2 cents,
> jay
>
> --
> You received this bug notification because you are a member of Nova Bug
> Team, which is subscribed to OpenStack Compute (nova).
> https://bugs.launchpad.net/bugs/717467
>
> Title:
> from migrate.versioning import exceptions fails with recent venv
>
> Status in OpenStack Compute (Nova):
> In Progress
>
> Bug description:
> No virtual environment found...create one? (Y/n)
> done.
>
> ....
>
> ======================================================================
> ERROR: <nose.suite.ContextSuite context=nova.tests>
> ----------------------------------------------------------------------
> Traceback (most recent call last):
> File "/home/bschott/source/nova/hpc-trunk/.nova-venv/lib/python2.6/site-packages/nose/suite.py", line 208, in run
> self.setUp()
> File "/home/bschott/source/nova/hpc-trunk/.nova-venv/lib/python2.6/site-packages/nose/suite.py", line 291, in setUp
> self.setupContext(ancestor)
> File "/home/bschott/source/nova/hpc-trunk/.nova-venv/lib/python2.6/site-packages/nose/suite.py", line 314, in setupContext
> try_run(context, names)
> File "/home/bschott/source/nova/hpc-trunk/.nova-venv/lib/python2.6/site-packages/nose/util.py", line 478, in try_run
> return func()
> File "/home/bschott/source/nova/hpc-trunk/nova/tests/__init__.py", line 41, in setup
> migration.db_sync()
> File "/home/bschott/source/nova/hpc-trunk/nova/db/migration.py", line 33, in db_sync
> return IMPL.db_sync(version=version)
> File "/home/bschott/source/nova/hpc-trunk/nova/utils.py", line 367, in __getattr__
> backend = self.__get_backend()
> File "/home/bschott/source/nova/hpc-trunk/nova/utils.py", line 362, in __get_backend
> self.__backend = __import__(name, None, None, fromlist)
> File "/home/bschott/source/nova/hpc-trunk/nova/db/sqlalchemy/migration.py", line 25, in <module>
> from migrate.versioning import exceptions as versioning_exceptions
> ImportError: cannot import name exceptions
>
> ...

[Brian Schott (bfschott)]

I will try Jay's suggestion right now and resubmit. I was stuck on the fact of no __version__ to branch on, didn't think about "try".

[Trey Morris (tr3buchet)]

another +1 to Jay. How's the fix Brian? (bug's biting me too..)

[Brian Schott (bfschott)]

Done! This time for sure :-).

[Brian Schott (bfschott)]

LP question, do I need to resubmit for review? if so, how.

[Jay Pipes (jaypipes)]

Just set the merge proposal to Work In Progress while you work on the branch. Then set the merge prop back to Needs Review, and that will trigger an email to all prior reviewers and re-diff the merge prop.

I did the above to your merge prop, so no need to do that again.

Cheers,
Jay

[Thierry Carrez (ttx)]

Verified fixed on 2011.1.1 candidate (bzr653) -- supports both cases.