postgres newton post upgrade failure DBAPIError exception wrapped from (psycopg2.ProgrammingError) column build_requests.instance_uuid does not exist

[Matthew Thode]

This could be related to https://bugs.launchpad.net/nova/+bug/1630446 but I am reporting it here because it might not be :D

Error was encountered after a m->n migration, db sync went fine. Error log is attached.

[Matthew Thode (prometheanfire)]

I think 018_instance_groups.py is supposed to create those columns. Is that restricted to mysql?

[Matthew Thode (prometheanfire)]

for reference, looking at this

https://review.openstack.org/gitweb?p=openstack/nova.git;a=blob;f=nova/db/sqlalchemy/api_migrations/migrate_repo/versions/018_instance_groups.py

Here's what's in the database for those tables, it doesn't seem to match up with what's in 018.

nova=# select * from instance_group_policy;
 created_at | updated_at | deleted_at | deleted | id | policy | group_id
------------+------------+------------+---------+----+--------+----------
(0 rows)

nova=# select * from instance_group_member;
 created_at | updated_at | deleted_at | deleted | id | instance_id | group_id
------------+------------+------------+---------+----+-------------+----------
(0 rows)

nova=# select * from instance_groups;
 created_at | updated_at | deleted_at | deleted | id | user_id | project_id | uuid | name
------------+------------+------------+---------+----+---------+------------+------+------
(0 rows)

[Sylvain Bauza (sylvain-bauza)]

https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/018_instance_groups.py is not executed by nova-manage db sync but rather nova-manage api_db sync which is for populating the API DB.

FWIW, we added instance_uuid in https://github.com/openstack/nova/blob/master/nova/db/sqlalchemy/api_migrations/migrate_repo/versions/013_build_request_extended_attrs.py

The right workflow is :
 - nova-manage db sync && nova-manage api_db sync
 - nova-manage db migrations

Closing it as Invalid, this is a deployment issue.

[Roman Podoliaka (rpodolyaka)]

Matthew, are you sure you also executed:

nova-manage api_db sync

?

These failures look like api db was not populated properly.

[Sylvain Bauza (sylvain-bauza)]

I'm sure this is due to the API tables not being populated, hence that bug being incomplete, we don't need the reporter's reply.

[Sylvain Bauza (sylvain-bauza)]

Graaah, I meant "hence that bug being invalid" :-)

[Matthew Thode (prometheanfire)]

Thanks for pointing that out, forgot to do the api_db upgrade...

[Matthew Thode (prometheanfire)]

yep, forgot to run the api_db upgrade, thanks for pointing it out