OpenStack Compute (Nova)

Migration 074 should not need connection_type defined if no data is present (new installs)

Reported by Adam Gandelman on 2012-01-24
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
High
Brian Waldon

Bug Description

The recently added migration 074_change_flavor_local_gb.py references the config flag connection_type in its upgrade. Since this flag only applies to nodes running nova-compute services, it may not be set on the node where database migrations are actually run, resulting in exception.Error("Need connection_type specified to run migration")

Vish Ishaya (vishvananda) wrote :

Not sure exactly what the fix for this would be. Do you want a better error message? Perhaps we could make it say: Unable to determine connection type for migration, please specify it in your flagfile or on the command line like
nova-manage --connection_type=<type> db sync

Dan Prince (dan-prince) wrote :

Hi Adam,

I'm not entirely happy that we have to rely on flags within the database migrations either... but given that we needed this to pull off the xenapi/libvirt parity changes... explicitly requiring the flag to be set makes sense I think.

If we were to default the connection_type flag to 'libvirt' it may cover the common case but would leave users of XenServer scratching their heads when they try to boot instances (this causes very cryptic errors).

Requiring it to be set explicitly makes this behaviour fail fast which in my opinion makes the most sense. Yes it can be a bit painful but like Vish pointed out above there are a couple different ways to go about it.

Vish: ++ on improving the error message of the exception.

Thierry Carrez (ttx) on 2012-01-25
Changed in nova:
milestone: none → essex-3
Thierry Carrez (ttx) wrote :

Hmm. I think Adam's point is that connection_type is nova-compute-specific, and on a multi-node setup you may be running the DB migration from a host that is not nova-compute and therefore has no connection_type set...

That sounds pretty brittle to me, especially in packaging as they "run nova-manage db sync" during package install/upgrades.

Changed in nova:
importance: Undecided → High
status: New → Incomplete
Thierry Carrez (ttx) wrote :

also don't we support multiple connection_type on a single cloud ?

In theory yes, but do to the issue that this migration fixes, It was pretty useless. Your disks would be very different in kvm vs xen. I still haven't heard of a good suggestion with how to deal with it. You can specify the flag on the command line when you do the sync command if it isn't set, but I don't know of another way for nova to guess which style of image you were using.

On Jan 25, 2012, at 4:52 AM, Thierry Carrez wrote:

> also don't we support multiple connection_type on a single cloud ?
>
> --
> You received this bug notification because you are subscribed to
> OpenStack Compute (nova).
> https://bugs.launchpad.net/bugs/921294
>
> Title:
> Migration 074 assumes connection_type is defined locally during
> migrations
>
> Status in OpenStack Compute (Nova):
> Incomplete
>
> Bug description:
> The recently added migration 074_change_flavor_local_gb.py references
> the config flag connection_type in its upgrade. Since this flag only
> applies to nodes running nova-compute services, it may not be set on
> the node where database migrations are actually run, resulting in
> exception.Error("Need connection_type specified to run migration")
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nova/+bug/921294/+subscriptions

I guess for Ubuntu the solution is to call "nova-manage --connection_type=libvirt db sync" explicitely (since that's what they support in packaging).

The issue I have with this is that it also applies to first-time users, not just on people migrating from a previous version... For them, rather than creating it right, without making strange assumptions, we create the broken database and then ask the user to make the choice on how to unbreak it...

Thierry Carrez (ttx) wrote :

Maybe when there is no data yet, we could avoid the difficult question, so that it only bites old users.

I like that idea. If there is no data in the db we could pick the xen version (which i think is more useful)

Vish

On Jan 25, 2012, at 8:24 AM, Thierry Carrez wrote:

> Maybe when there is no data yet, we could avoid the difficult question,
> so that it only bites old users.
>
> --
> You received this bug notification because you are subscribed to
> OpenStack Compute (nova).
> https://bugs.launchpad.net/bugs/921294
>
> Title:
> Migration 074 assumes connection_type is defined locally during
> migrations
>
> Status in OpenStack Compute (Nova):
> Incomplete
>
> Bug description:
> The recently added migration 074_change_flavor_local_gb.py references
> the config flag connection_type in its upgrade. Since this flag only
> applies to nodes running nova-compute services, it may not be set on
> the node where database migrations are actually run, resulting in
> exception.Error("Need connection_type specified to run migration")
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nova/+bug/921294/+subscriptions

Thierry Carrez (ttx) on 2012-01-25
summary: - Migration 074 assumes connection_type is defined locally during
- migrations
+ Migration 074 should not need connection_type defined if no data is
+ present
Changed in nova:
status: Incomplete → Triaged

There is a default set of instance_types defined in the 008 migration. Is there ever a situation where there is no data present?

What's wrong with the error message? It seems clear what needs to happen to allow the migration to run ("Need connection_type specified to run migration").

Or is the "why" about the message the problem?

Thierry Carrez (ttx) wrote :

My understanding is that the parameter is necessary for Nova to understand how to migrate existing data, which ends up meaning different things based on whether you use libvirt or xen.

If there is no upgrade and the database was just created with the default data, there is no need to require a parameter set to interpret the past: the result should be always the same, and not based on the value of a parameter...

summary: Migration 074 should not need connection_type defined if no data is
- present
+ present (new installs)
Thierry Carrez (ttx) on 2012-01-25
Changed in nova:
assignee: nobody → Brian Waldon (bcwaldon)
Adam Gandelman (gandelman-a) wrote :

My main concerns come from looking at this from a packagers perspective. With things like XCP support in Ubuntu coming closer to seeing the light of day, this kinda stuff makes it really difficult to have a nova-common package that can work out of the box with any corresponding compute setup/package. Granted, currently only libvirt is really supported and we already make assumptions about this in the common nova.conf. Working around this by adding '--connection_type=' to nova.conf in nova-common is doable in the short term, but if we want packaging to support a variety of configurations (I do), we might (packagers) need to rethink some things (perhaps nova-common-{libvirt, xcp, etc})

Do I understand the proposed solution?

- New users with empty databases and no connection_type get the xen version by default.
- Exsitng users with data in the database will need to specify connection_type explicitly during migration
- From Ubuntu POV, we can continue to be opinionated about what users are deploying, and set connection_type explicitly=libvirt in packaging

Brian Waldon (bcwaldon) on 2012-01-25
Changed in nova:
status: Triaged → In Progress

I agree, the solution sucks all around. I just wish we hadn't let libvirt and xenapi diverge in interpretation originally and then let it stay that way for so long.

If the definition of "empty database" is no instances, I'm fine with defaulting to connection_type == 'xenapi' for the migration. It's certainly more user friendly behavior for new users.

Sounds like packagers have it easy, they generally only support libvirt or xenapi and not both.

Vish Ishaya (vishvananda) wrote :

> Do I understand the proposed solution?

Yes that is it

Reviewed: https://review.openstack.org/3439
Committed: http://github.com/openstack/nova/commit/b4523032029604df34e045d6f6777a695328cff4
Submitter: Jenkins
Branch: master

commit b4523032029604df34e045d6f6777a695328cff4
Author: Brian Waldon <email address hidden>
Date: Wed Jan 25 15:20:38 2012 -0800

    Ignore connection_type when no instances exist

    In migration 74, we had required that the connection_type flag
    be set. That's annoying for new deployments, so bypass this check
    if there are no instances in the databse. Fixes bug 921294

    Change-Id: I9b829e80ad7fa7ded3c7a471cb68c9b342d973bb

Changed in nova:
status: In Progress → Fix Committed

Reviewed: https://review.openstack.org/3444
Committed: http://github.com/openstack/nova/commit/a3f9aa2e63e965119aad7f65a0f97265bdf23c07
Submitter: Jenkins
Branch: milestone-proposed

commit a3f9aa2e63e965119aad7f65a0f97265bdf23c07
Author: Brian Waldon <email address hidden>
Date: Wed Jan 25 15:20:38 2012 -0800

    Ignore connection_type when no instances exist

    In migration 74, we had required that the connection_type flag
    be set. That's annoying for new deployments, so bypass this check
    if there are no instances in the databse. Fixes bug 921294

    Change-Id: I9b829e80ad7fa7ded3c7a471cb68c9b342d973bb

Changed in nova:
status: Fix Committed → Fix Released
Thierry Carrez (ttx) on 2012-04-05
Changed in nova:
milestone: essex-3 → 2012.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers