nova-manage db sync throws an import error

Bug #1347243 reported by Jeff Silverman
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
4.1.x
Fix Committed
High
MOS Nova

Bug Description

I am following the instructions in

When I run nova-manage db sync, I get the following:

<email address hidden>:~# su -s /bin/sh -c "nova-manage db sync" nova
Command failed, please check log for more info
<email address hidden>:~#
<email address hidden>:~# tail /var/log/nova/nova-manage.log
2014-07-22 15:18:35.279 18396 CRITICAL nova [-] ImportError: cannot import name exceptions

<email address hidden>:~#

I did some checking. nova-manage is a python program, so I ran it with the debugger pdb. I find:

(Pdb) c
Command failed, please check log for more info
Traceback (most recent call last):
  File "/usr/lib64/python2.6/pdb.py", line 1296, in main
    pdb._runscript(mainpyfile)
  File "/usr/lib64/python2.6/pdb.py", line 1215, in _runscript
    self.run(statement)
  File "/usr/lib64/python2.6/bdb.py", line 372, in run
    exec cmd in globals, locals
  File "<string>", line 1, in <module>
  File "/usr/bin/nova-manage", line 10, in <module>
    sys.exit(main())
  File "/usr/lib/python2.6/site-packages/nova/cmd/manage.py", line 1374, in main
    ret = fn(*fn_args, **fn_kwargs)
  File "/usr/lib/python2.6/site-packages/nova/cmd/manage.py", line 883, in sync
    return migration.db_sync(version)
  File "/usr/lib/python2.6/site-packages/nova/db/migration.py", line 29, in db_sync
    return IMPL.db_sync(version=version)
  File "/usr/lib/python2.6/site-packages/nova/utils.py", line 427, in __getattr__
    backend = self.__get_backend()
  File "/usr/lib/python2.6/site-packages/nova/utils.py", line 423, in __get_backend
    self.__backend = __import__(name, None, None, fromlist)
  File "/usr/lib/python2.6/site-packages/nova/db/sqlalchemy/migration.py", line 20, in <module>
    from migrate.versioning import api as versioning_api
  File "/usr/lib/python2.6/site-packages/migrate/versioning/api.py", line 32, in <module>
    from migrate.versioning import (exceptions, repository, schema, version,
  File "/usr/lib/python2.6/site-packages/migrate/versioning/schema.py", line 10, in <module>
    from sqlalchemy import exceptions as sa_exceptions
ImportError: cannot import name exceptions
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /usr/lib/python2.6/site-packages/migrate/versioning/schema.py(10)<module>()
-> from sqlalchemy import exceptions as sa_exceptions
(Pdb)

<email address hidden>:~# python /usr/bin/nova-manage --version
2014.1.1
<email address hidden>:~#

What other information would be helpful in pursuing this problem?

Thank you

Jeff Silverman

Tags: nova migrate
Revision history for this message
Jeff Silverman (jeffsilverm) wrote :

I forgot to paste the URL. I am following the instructions in http://docs.openstack.org/trunk/install-guide/install/yum/content/nova-controller-install.html. I am at To install and configure Compute controller components step 3.

I apologize for the confusion.

Jeff Silverman

Revision history for this message
Jeff Silverman (jeffsilverm) wrote :

I have been doing some more poking at the problem. Using the debugger, I have isolated the problem to:

(Pdb) b /usr/lib/python2.6/site-packages/migrate/versioning/api.py:32
Breakpoint 1 at /usr/lib/python2.6/site-packages/migrate/versioning/api.py:32
(Pdb) c
> /usr/lib/python2.6/site-packages/migrate/versioning/api.py(32)<module>()
-> from migrate.versioning import (exceptions, repository, schema, version,
(Pdb) list
 27
 28 import sys
 29 import inspect
 30 import logging
 31
 32 B-> from migrate.versioning import (exceptions, repository, schema, version,
 33 script as script_) # command name conflict
 34 from migrate.versioning.util import catch_known_errors, with_engine
 35
 36
 37 log = logging.getLogger(__name__)
(Pdb) n
ImportError: 'cannot import name exceptions'
> /usr/lib/python2.6/site-packages/migrate/versioning/api.py(32)<module>()
-> from migrate.versioning import (exceptions, repository, schema, version,
(Pdb)

I think I have some very old code:

<email address hidden>:/usr/lib/python2.6/site-packages/migrate/versioning# ls -lort exceptions.*
-rw-r--r-- 1 root 1606 Jul 4 2010 exceptions.py
-rw-r--r-- 2 root 4777 Dec 4 2011 exceptions.pyo
-rw-r--r-- 2 root 4777 Dec 4 2011 exceptions.pyc
<email address hidden>:/usr/lib/python2.6/site-packages/migrate/versioning# pwd
/usr/lib/python2.6/site-packages/migrate/versioning
<email address hidden>:/usr/lib/python2.6/site-packages/migrate/versioning#

-> from migrate.versioning import (exceptions, repository, schema, version,
(Pdb) from migrate.versioning import exceptions
(Pdb) from migrate.versioning import repository
(Pdb) from migrate.versioning import schema
*** ImportError: cannot import name exceptions
(Pdb)
(Pdb) from migrate.versioning import version
(Pdb)

<email address hidden>:/usr/lib/python2.6/site-packages/migrate/versioning# python -V
Python 2.6.6
<email address hidden>:/usr/lib/python2.6/site-packages/migrate/versioning# cat /etc/redhat-release
CentOS release 6.5 (Final)
<email address hidden>:/usr/lib/python2.6/site-packages/migrate/versioning#

<email address hidden>:/usr/lib/python2.6/site-packages/migrate/versioning# ls -l exceptions* __init__*
-rw-r--r-- 1 root root 1606 Jul 4 2010 exceptions.py
-rw-r--r-- 2 root root 4777 Dec 4 2011 exceptions.pyc
-rw-r--r-- 2 root root 4777 Dec 4 2011 exceptions.pyo
-rw-r--r-- 1 root root 161 Jul 4 2010 __init__.py
-rw-r--r-- 2 root root 327 Dec 4 2011 __init__.pyc
-rw-r--r-- 2 root root 327 Dec 4 2011 __init__.pyo
<email address hidden>:/usr/lib/python2.6/site-packages/migrate/versioning#

Revision history for this message
Dmitry Ilyin (idv1985) wrote :

Strage. I does'n throw this error on my deployments. Both Ubuntu and Centos...

Oh, wait! You aren't using Fuel, aren't ye? And you can't import sqlalchemy but it's present.
Hmm... looks linke OpenStack components use their own sqlalchemy componets insead of system-wide and perhaps they mess with python's include path to do so.

Revision history for this message
Jeff Silverman (jeffsilverm) wrote : Re: [Bug 1347243] Re: nova-manage db sync throws an import error
Download full text (4.8 KiB)

Dmitry,

Thank you for your quick response.

<email address hidden>:~# echo $PYTHONPATH

<email address hidden>:~# su -s /bin/sh -c "nova-manage db
sync" nova
Command failed, please check log for more info
<email address hidden>:~# echo $PYTHONPATH

<email address hidden>:~# tail
/var/log/nova/nova-manage.log
2014-07-22 15:18:35.279 18396 CRITICAL nova [-] ImportError: cannot import
name exceptions

2014-07-22 19:58:15.653 21403 CRITICAL nova [-] ImportError: cannot import
name exceptions

2014-07-23 09:29:04.627 28760 CRITICAL nova [-] ImportError: cannot import
name exceptions

<email address hidden>:~#

How do I tell if I am using fuel? How do I diagnose which sqlalchemy I am
using?

Many thanks,

Jeff Silverman

On Wed, Jul 23, 2014 at 2:30 AM, Dmitry Ilyin <email address hidden>
wrote:

> Strage. I does'n throw this error on my deployments. Both Ubuntu and
> Centos...
>
> Oh, wait! You aren't using Fuel, aren't ye? And you can't import
> sqlalchemy but it's present.
> Hmm... looks linke OpenStack components use their own sqlalchemy componets
> insead of system-wide and perhaps they mess with python's include path to
> do so.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1347243
>
> Title:
> nova-manage db sync throws an import error
>
> Status in Fuel: OpenStack installer that works:
> New
>
> Bug description:
> I am following the instructions in
>
> When I run nova-manage db sync, I get the following:
>
> <email address hidden>:~# su -s /bin/sh -c "nova-manage
> db sync" nova
> Command failed, please check log for more info
> <email address hidden>:~#
> <email address hidden>:~# tail
> /var/log/nova/nova-manage.log
> 2014-07-22 15:18:35.279 18396 CRITICAL nova [-] ImportError: cannot
> import name exceptions
>
> <email address hidden>:~#
>
>
> I did some checking. nova-manage is a python program, so I ran it with
> the debugger pdb. I find:
>
> (Pdb) c
> Command failed, please check log for more info
> Traceback (most recent call last):
> File "/usr/lib64/python2.6/pdb.py", line 1296, in main
> pdb._runscript(mainpyfile)
> File "/usr/lib64/python2.6/pdb.py", line 1215, in _runscript
> self.run(statement)
> File "/usr/lib64/python2.6/bdb.py", line 372, in run
> exec cmd in globals, locals
> File "<string>", line 1, in <module>
> File "/usr/bin/nova-manage", line 10, in <module>
> sys.exit(main())
> File "/usr/lib/python2.6/site-packages/nova/cmd/manage.py", line 1374,
> in main
> ret = fn(*fn_args, **fn_kwargs)
> File "/usr/lib/python2.6/site-packages/nova/cmd/manage.py", line 883,
> in sync
> return migration.db_sync(version)
> File "/usr/lib/python2.6/site-packages/nova/db/migration.py", line 29,
> in db_sync
> return IMPL.db_sync(version=version)
> File "/usr/lib/python2.6/site-packages/nova/utils.py", line 427, in
> __getattr__
> backend = self.__get_backend()
> File "/usr/lib/python2.6/...

Read more...

Revision history for this message
Jeff Silverman (jeffsilverm) wrote :
Download full text (5.5 KiB)

Dmitry,

We are not using fuel.

Jeff

On Wed, Jul 23, 2014 at 9:30 AM, Jeff Silverman <email address hidden>
wrote:

> Dmitry,
>
> Thank you for your quick response.
>
> <email address hidden>:~# echo $PYTHONPATH
>
> <email address hidden>:~# su -s /bin/sh -c "nova-manage
> db sync" nova
> Command failed, please check log for more info
> <email address hidden>:~# echo $PYTHONPATH
>
> <email address hidden>:~# tail
> /var/log/nova/nova-manage.log
> 2014-07-22 15:18:35.279 18396 CRITICAL nova [-] ImportError: cannot
> import name exceptions
>
> 2014-07-22 19:58:15.653 21403 CRITICAL nova [-] ImportError: cannot import
> name exceptions
>
> 2014-07-23 09:29:04.627 28760 CRITICAL nova [-] ImportError: cannot import
> name exceptions
>
> <email address hidden>:~#
>
> How do I tell if I am using fuel? How do I diagnose which sqlalchemy I am
> using?
>
>
> Many thanks,
>
>
> Jeff Silverman
>
>
>
>
> On Wed, Jul 23, 2014 at 2:30 AM, Dmitry Ilyin <email address hidden>
> wrote:
>
>> Strage. I does'n throw this error on my deployments. Both Ubuntu and
>> Centos...
>>
>> Oh, wait! You aren't using Fuel, aren't ye? And you can't import
>> sqlalchemy but it's present.
>> Hmm... looks linke OpenStack components use their own sqlalchemy
>> componets insead of system-wide and perhaps they mess with python's include
>> path to do so.
>>
>> --
>> You received this bug notification because you are subscribed to the bug
>> report.
>> https://bugs.launchpad.net/bugs/1347243
>>
>> Title:
>> nova-manage db sync throws an import error
>>
>> Status in Fuel: OpenStack installer that works:
>> New
>>
>> Bug description:
>> I am following the instructions in
>>
>> When I run nova-manage db sync, I get the following:
>>
>> <email address hidden>:~# su -s /bin/sh -c "nova-manage
>> db sync" nova
>> Command failed, please check log for more info
>> <email address hidden>:~#
>> <email address hidden>:~# tail
>> /var/log/nova/nova-manage.log
>> 2014-07-22 15:18:35.279 18396 CRITICAL nova [-] ImportError: cannot
>> import name exceptions
>>
>> <email address hidden>:~#
>>
>>
>> I did some checking. nova-manage is a python program, so I ran it with
>> the debugger pdb. I find:
>>
>> (Pdb) c
>> Command failed, please check log for more info
>> Traceback (most recent call last):
>> File "/usr/lib64/python2.6/pdb.py", line 1296, in main
>> pdb._runscript(mainpyfile)
>> File "/usr/lib64/python2.6/pdb.py", line 1215, in _runscript
>> self.run(statement)
>> File "/usr/lib64/python2.6/bdb.py", line 372, in run
>> exec cmd in globals, locals
>> File "<string>", line 1, in <module>
>> File "/usr/bin/nova-manage", line 10, in <module>
>> sys.exit(main())
>> File "/usr/lib/python2.6/site-packages/nova/cmd/manage.py", line
>> 1374, in main
>> ret = fn(*fn_args, **fn_kwargs)
>> File "/usr/lib/python2.6/site-packages/nova/cmd/manage.py", line 883,
>> in sync
>> return migration.db_sync(version)
>> File "/usr/lib/python2.6/site-packages/nova/db...

Read more...

Revision history for this message
Jeff Silverman (jeffsilverm) wrote :
Download full text (8.3 KiB)

Here is the sys.path list at the point where the import fails.

(Pdb) b /usr/lib/python2.6/site-packages/migrate/versioning/schema.py:10
Breakpoint 1 at /usr/lib/python2.6/site-packages/migrate/versioning/schema.py:10

(Pdb) c
> /usr/lib/python2.6/site-packages/migrate/versioning/schema.py(10)<module>()
-> from sqlalchemy import exceptions as sa_exceptions
(Pdb) list
  5 import logging
  6
  7 from sqlalchemy import (Table, Column, MetaData, String, Text, Integer,

  8 create_engine)
  9 from sqlalchemy.sql import and_
 10 B-> from sqlalchemy import exceptions as sa_exceptions
 11 from sqlalchemy.sql import bindparam
 12
 13 from migrate.changeset import SQLA_07

 14 from migrate.versioning import exceptions, genmodel, schemadiff
 15 from migrate.versioning.repository import Repository
(Pdb) print sys.path
['/usr/bin', '/usr/lib64/python26.zip', '/usr/lib64/python2.6',
'/usr/lib64/python2.6/plat-linux2', '/usr/lib64/python2.6/lib-tk',
'/usr/lib64/python2.6/lib-old', '/usr/lib64/python2.6/lib-dynload',
'/usr/lib64/python2.6/site-packages',
'/usr/lib64/python2.6/site-packages/PIL',
'/usr/lib64/python2.6/site-packages/gtk-2.0',
'/usr/lib/python2.6/site-packages',
'/usr/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg-info']

(Pdb)

sqlalchemy is installed:

<email address hidden>:~# yum list | fgrep -i sqlal
Repository sl-release-el6 is listed more than once in the configuration
python-sqlalchemy.x86_64 0.8.4-1.el6
@openstack-icehouse
python-sqlalchemy0.7.x86_64 0.7.8-1.el6 @epel-6-x86_64
python-flask-sqlalchemy.noarch 0.16-2.el6 epel-6-x86_64
python-sqlalchemy-debuginfo.x86_64 0.8.4-1.el6 openstack-icehouse
python-sqlalchemy0.8.x86_64 0.8.2-4.el6 epel-6-x86_64
python-zope-sqlalchemy.noarch 0.4-3.el6 centos-6.5-base
<email address hidden>:~#

I think this was the wrong place to file this bug - I am sorry. I
think this bug is redundant with 1347827
<https://bugs.launchpad.net/openstack-manuals/+bug/1347827>.

Jeff

On Wed, Jul 23, 2014 at 9:41 AM, Jeff Silverman <email address hidden>
wrote:

> Dmitry,
>
> We are not using fuel.
>
> Jeff
>
>
>
> On Wed, Jul 23, 2014 at 9:30 AM, Jeff Silverman <email address hidden>
> wrote:
>
>> Dmitry,
>>
>> Thank you for your quick response.
>>
>> <email address hidden>:~# echo $PYTHONPATH
>>
>> <email address hidden>:~# su -s /bin/sh -c "nova-manage
>> db sync" nova
>> Command failed, please check log for more info
>> <email address hidden>:~# echo $PYTHONPATH
>>
>> <email address hidden>:~# tail
>> /var/log/nova/nova-manage.log
>> 2014-07-22 15:18:35.279 18396 CRITICAL nova [-] ImportError: cannot
>> import name exceptions
>>
>> 2014-07-22 19:58:15.653 21403 CRITICAL nova [-] ImportError: cannot
>> import name exceptions
>>
>> 2014-07-23 09:29:04.627 28760 CRITICAL nova [-] ImportError: cannot
>> import name exceptions
>>
>> <email address hidden>:~#
>>
>> How do I tell if I am using fuel? How do I diagnose which sqlalchemy I
>> am using?
>>
>>
...

Read more...

Revision history for this message
Jeff Silverman (jeffsilverm) wrote :

I resolved the issue. In sqlalchemy, they renamed a file from exceptions.py to exc.py. The solution was to create a symlink in the directory /usr/lib/python2.7/site-packages/sqlalchemy/ from exc.py to exceptions.py.

Dmitry Ilyin (idv1985)
Changed in fuel:
status: New → Invalid
Mike Scherbakov (mihgen)
Changed in fuel:
milestone: none → 6.0
affects: fuel → mos
Changed in mos:
milestone: 6.0 → none
milestone: none → 4.1.2
status: Invalid → Triaged
importance: Undecided → Medium
tags: added: nova
tags: added: migrate
Changed in mos:
assignee: nobody → Fuel OSCI Team (fuel-osci)
Roman Vyalov (r0mikiam)
Changed in mos:
assignee: Fuel OSCI Team (fuel-osci) → MOS Nova (mos-nova)
Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

python-migrate <= 0.8 is incompatible with SQLAlchemy >= 0.8 and should be updated

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

oops, ^ python-migrate < 0.8

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

Roman V., this is not a Nova, but rather purely a python-migrate version issue. Nova is just a project affected by this bug. AFAIU, fuel-osci team is in charge of updating python dependencies.

Revision history for this message
Roman Podoliaka (rpodolyaka) wrote :

I checked the package mirros, only 4.0.x and 4.1.x branches must be affected.

no longer affects: mos
Revision history for this message
Ivan Udovichenko (iudovichenko) wrote :
Revision history for this message
Ivan Udovichenko (iudovichenko) wrote :

I've written a message to Roman Podolyaka about branches which we need.

Revision history for this message
Ivan Udovichenko (iudovichenko) wrote :
  • Ubuntu Edit (141.3 KiB, application/x-debian-package)

You need to use Migrate >= 0.8.2

I'm attaching packages. You may use them.

Revision history for this message
Ivan Udovichenko (iudovichenko) wrote :
  • Centos Edit (221.2 KiB, application/x-redhat-package-manager)
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.