No ability to update consumer's project and/or user external ID

Bug #1779717 reported by Jay Pipes
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Jay Pipes

Bug Description

Even though in Placement API microversion 1.12, we began requiring project and user ID in the PUT /allocations request payload, the project ID and user ID for an *existing* consumer are never actually updated.

What this means is that if you created an allocation for consumer X before 1.12, like so:

```
PUT /allocations/X
{
  "allocations": ...
}
```

a consumer record will be created for the allocation and the project and user ID for that consumer record will be the CONF.placement.incomplete_consumer_project_id value (or user ID value). If I want to set the project and user ID of the allocation's consumer to some other value, there's no way I can do that.

If I do the following:

```
OpenStack-API-Version: placement 1.28
PUT /allocations/X
{
  "allocations": ...
  "consumer_generation": 1,
  "project_id": $NEW_PROJECT_ID,
  "user_id": $NEW_USER_ID
}
```

The $NEW_PROJECT_ID and $NEW_USER_ID values are merely ignored.

Tags: api placement
Revision history for this message
Jay Pipes (jaypipes) wrote :

The reason these values are ignored is because utils.ensure_consumer() is called from _new_allocations() in the allocation.py handler. This function simply looks up the consumer object and returns it if it exists, creating one otherwise. If the allocations payload had a different project or user, util.ensure_consumer() makes no attempt whatsoever to update the consumer's project/user info.

I'm currently working on a fix for this.

Revision history for this message
Matt Riedemann (mriedem) wrote :
Revision history for this message
Jay Pipes (jaypipes) wrote :

Yes, also true of ``POST /allocations``

Revision history for this message
Chris Dent (cdent) wrote :

Presumably we only want to change the project and user id on a consumer record if it is either empty or has the sentinel values from CONF?

Otherwise, we have something a bit strange: At time X (and before Y) an allocation with consumer uuid has project and user foo, and at time Y (and every moment thereafter), then the uuid is reused for some reason, has project and user bar. Is that legit?

In the context of instance uuids being the consumer uuid, this isn't something that is likely to happen. And _most_ of the time consumer uuids won't be re-used. But there's nothing in our original specification of /allocations/{uuid} that they won't be or can't be. And we've intended placement to be flexible about who its clients are.

Or are we going to say that a once a consumer uuid exists, it can only ever be available to the (real) project and user ids that first created it?

Do these concerns go away if we say that consumer objects are somewhat ephemeral and are deleted after the last allocation associated with it goes away?

Revision history for this message
Jay Pipes (jaypipes) wrote :

"Presumably we only want to change the project and user id on a consumer record if it is either empty or has the sentinel values from CONF?"

Currently, those are the only two scenarios that make sense, yes.

I can certainly see a scenario in the future of "transferring ownership" of an instance (or volume or whatever) from one user or project to another. This has been something that has been requested for like 5 years in other OpenStack projects (ownership transfer) but we haven't done much about it.

Revision history for this message
Matt Riedemann (mriedem) wrote :

This directly impacts me here: https://review.openstack.org/#/c/574488/

That's the nova-manage placement heal_allocations change to migrate allocations created with the sentinel values from config to use the real values from the instance, which we're going to need to fix if we ever plan on using allocations for quota in nova.

I agree that you should only be able to change the project/user if it's None (not set previously but with ensure_consumer I'm not sure this can happen now - especially since reading allocations out of the DB sets the consumer if it doesn't exist using the config values), or if it's the sentinel config values (hopefully operators don't change those once set....) - otherwise you're allowing transferring ownership of the resource and that's not something we've allowed on other resources in openstack, like transferring instance X in project A from project B, even though people have asked for that in the past.

Revision history for this message
Matt Riedemann (mriedem) wrote :

Ed pointed out in IRC that one could simply delete the existing allocations and then re-create them with the new project/user as a workaround.

Matt Riedemann (mriedem)
Changed in nova:
importance: Medium → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/581086

Changed in nova:
status: Triaged → In Progress
Revision history for this message
Matt Riedemann (mriedem) wrote :

Moved back to Triaged. https://review.openstack.org/581086 is fixing a different bug.

Changed in nova:
status: In Progress → Triaged
tags: added: api
Revision history for this message
OpenStack Infra (hudson-openstack) wrote :

Fix proposed to branch: master
Review: https://review.openstack.org/581139

Changed in nova:
status: Triaged → In Progress
Changed in nova:
assignee: Jay Pipes (jaypipes) → Eric Fried (efried)
Jay Pipes (jaypipes)
Changed in nova:
assignee: Eric Fried (efried) → Jay Pipes (jaypipes)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/581139
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=4f9a7da5819b4cc2c181506c682a092caff40fb3
Submitter: Zuul
Branch: master

commit 4f9a7da5819b4cc2c181506c682a092caff40fb3
Author: Jay Pipes <email address hidden>
Date: Mon Jul 9 17:26:52 2018 -0400

    update project/user for consumer in allocation

    Adds code to the util.ensure_consumer() method that updates the
    consumer's project and user foreign key identifiers when a user supplies
    a different external project or user identifier in the paylod of PUT
    /allocations/{consumer_uuid} or POST /allocations.

    Change-Id: I9368ad2280551ffa9a011d3a30c4b45861305455
    Closes-bug: #1779717
    Related-Bug: #1781008

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 18.0.0.0b3

This issue was fixed in the openstack/nova 18.0.0.0b3 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.