Optionally encode project IDs in fernet tokens

[Jose Castro Leon]

The proposal is to allow an operator to disable the encoding of the project_id in fernet via a configuration setting. In this way operators that have project ids in a different uuid format than the hex format, can opt-out for the try/catch conversion with uuid resulting in a valid encoding/decoding of this information inside the fernet token. The configuration setting will be disabled by default, not modifying the default behavior.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/399652

[Lance Bragstad (lbragstad)]

Hi Jose,

For context, we opted to convert the UUID hex format to bytes *before* message packing it because it results in a smaller token length over all. Allowing a configuration option to opt out of that optimization would results in larger tokens.

Are you unpacking the tokens somewhere and wanting to inspect the values? I was able to do that and convert the UUID byte representation back to hex:

http://cdn.pasteraw.com/ahkn83bbsuoq2m5ffho3ydd2kcc6oba

Thoughts?

[Adam Young (ayoung)]

This seems like a misunderstanding. But you could use a different token formatter instead of a new config setting to make this work.

[Jose Castro Leon (jose-castro-leon)]

I completely understand the reasons for using hex format on the fernet tokens.
But this is a long story, we started with the LDAP backend and picking a uuid format (with dashes). Then we kept the project ids when migrated to SQL identity. As we have a setup with more than 3k projects, and now more than 12 openstack components changing the ID format is not possible.

The problem is not the encoding, when we decode them we don't have the dashes and the select query on the backend will fail, because there is no uuid comparison just an equal

I know that avoid converting them makes the tokens bigger, but at least allows us to move to fernet.
_
Adam, on fernet they are several different formatters, are you thinking in changing them all?

[Lance Bragstad (lbragstad)]

Jose, do you know specifically what version of UUID you originally started using?

[Lance Bragstad (lbragstad)]

Jose, in the past keystone would create ID using the UUID format with dashes, but that was a long time ago and predates me. Now, when creating projects, keystone will create projects and use a hex format of the ID. If *all* your projects are adhering to the format with dashes, do you have something other than keystone creating the projects?

[Lance Bragstad (lbragstad)]

Extending the BasePayload object within keystone/token/providers/fernet/token_formatters.py would achieve the same results:

http://cdn.pasteraw.com/htn79m729bk6wuikvgkwaj96phsozsi

[Jose Castro Leon (jose-castro-leon)]

This is not binded with projects created by keystone itself, we have a lifecycle management tool that creates/deletes the projects. In case that we use your last comment, it would mean that all uuids in the format will not be encoded and then the token will be much bigger.

[Lance Bragstad (lbragstad)]

The diff I posted in comment #9 would still allow the tokens to be smaller because it message packing them as their byte representation. It just ensures that when the token is converted back from bytes - it's done so with the actual dashes in the ID.

[Jose Castro Leon (jose-castro-leon)]

Right now, we have 2 different formats on our project database, the one I mentioned earlier, that came back from the LDAP assignment, managed by an external tool. We also have the hex format on a domain for heat.
My suggestion was an opt-in configuration setting, that allows us to keep in sync with upstream, does not increase the token size that much and allows us to move to fernet tokens.

[Lance Bragstad (lbragstad)]

I thought all project IDs had dashes in them (at least according to comment #4)? So you do have project IDs in your deployment that are the uuid type and also don't have dashes? Ideally those types of IDs would work just fine with keystone upstream token formatters. It sounds like you're solving for the subset of project IDs that have dashes in them.

[Jose Castro Leon (jose-castro-leon)]

Trying to revive this one again, is there any concern into provide a option that does not encode the project ids in fernet? I know that the tokens will become a bit bigger but I still think this is affordable.

[Lance Bragstad (lbragstad)]

Was there any progress made with the proposed fix from comment #7?

[Lance Bragstad (lbragstad)]

Automatically unassigning due to inactivity.

[Morgan Fainberg (mdrnstm)]

I'm assuming this is now a dead issue. Marking it as incomplete so it can expire out. It sounds like this is simply not a major concern due to the lack of responses over the course of almost a year.

[Morgan Fainberg (mdrnstm)]

Revisiting to mark as wont fix after reading through it. This is a misunderstanding of the fernet packing and that the token payload is intended to be a blackbox and decoded/expanded by keystone itself.

ID formats (dashes, no dashes, etc) are the purview of keystone.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Colleen Murphy (<email address hidden>) on branch: master
Review: https://review.openstack.org/399652
Reason: The launchpad bug is marked as Won't Fix, and this change hasn't been updated in over two years, so I'm going to administratively abandon this change. Feel free to restore or to reopen the bug if you think closing this is a mistake and you would like to keep working on it.