Cannot delete or show a trust with an expired date

[Gilles Dubreuil]

Using keystone V3 trusts (Kilo 2015.1.0)

Deleting a non expired trust works as expected, meanwhile when trying to remove an expired trust, Keystone returns the trust doesn't exist.

Also nothing in the documentation [1] indicates expired trusts cannot be removed or have to flushed from database (using a separate process):

[1] http://specs.openstack.org/openstack/keystone-specs/api/v3/identity-api-v3-os-trust-ext.html

--- Trust list ---
# openstack --os-username adminv3 --os-password testing --os-auth-url http://192.168.64.11:5000/v3 --os-project-name openstackv3 --os-identity-api-version 3 --os-user-domain-name admin_domain --os-project-domain-name admin_domain trust list --format csv
"ID","Expires At","Impersonation","Project ID","Trustee User ID","Trustor User ID"
"38861fcd00594aa2ac4bfb6355833025","2018-01-01T00:00:00.000000Z",True,"78e22bb71862481dbe8335b4ce4551e8","ac994e5701d644b6a3ac78c9dd1ad04a","24b047f52ff94029923f7f0ea982f03f"
"3b98a91e56df4f4f879ae982083fdfd4","2015-01-01T00:00:00.000000Z",False,"78e22bb71862481dbe8335b4ce4551e8","ac994e5701d644b6a3ac78c9dd1ad04a","24b047f52ff94029923f7f0ea982f03f"
-----------------

--- Trust delete a non-expired trust ---
# openstack --os-username adminv3 --os-password testing --os-auth-url http://192.168.64.11:5000/v3 --os-project-name openstackv3 --os-identity-api-version 3 --os-user-domain-name admin_domain --os-project-domain-name admin_domain trust delete 38861fcd00594aa2ac4bfb6355833025
--------------------------------------------

--- Trust delete an expired trust ---
# openstack --os-username adminv3 --os-password testing --os-auth-url http://192.168.64.11:5000/v3 --os-project-name openstackv3 --os-identity-api-version 3 --os-user-domain-name admin_domain --os-project-domain-name admin_domain trust delete 3b98a91e56df4f4f879ae982083fdfd4
ERROR: openstack No trust with a name or ID of '3b98a91e56df4f4f879ae982083fdfd4' exists.
--------------------------------------------

----Keystone log file ---
2015-07-08 16:29:18.134 1635 WARNING keystone.common.wsgi [-] Could not find trust: 3b98a91e56df4f4f879ae982083fdfd4
--------------------------

[Gilles Dubreuil (gdubreui)]

Bad copy/paste:

Please replace the last "--- Trust delete a non expired trust ---" by "--- Trust delete an expired trust ---

[Gilles Dubreuil (gdubreui)]

Not sure if this is a python-openstackclient (tested with 1.5.0) or keystone issue.

[Dolph Mathews (dolph)]

Gilles: Bug descriptions are actually editable, so I've made the change in comment #1 for you.

[Dolph Mathews (dolph)]

This is by design in keystone, though, so there's reason to make it Won't Fix. Why do you want to be able to delete expired trusts? Both have the consequence of making a trust unusable.

[Steve Martinelli (stevemar)]

Yeah, it's already unusable, so i'm not sure what the benefit it. Might mark it as opinion

[Gilles Dubreuil (gdubreui)]

Not sure I understand the design idea behind.
For instance what's the point of keeping outdated trusts?

What is probably inconsistent in this design is the fact a user can create an object (the trust) but cannot delete the object in question. I mean if there is a desire to keep log of anything that should happen behind the curtain, maybe in a shadow DBMS table and not at the expense of the user.

[apal (yun-song)]

GET trust failed with 404 NOT FOUND too.
{code}
    Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: {"trust": {"impersonation": true, "trustor_user_id": "c873c9f351de4c348806f7aa3c462a68", "roles": [{"name": "DelegatedRole--874289898"}], "expires_at": "2015-07-14T16:23:30.742911Z", "trustee_user_id": "0c8b6735b1d744318158286608d0c8b3", "project_id": "4770e24cb4894707a20760fc35373f79"}}
    Response - Headers: {'status': '201', 'content-length': '828', 'vary': 'X-Auth-Token', 'server': 'Apache/2.4.10 (Debian)', 'connection': 'close', 'date': 'Tue, 14 Jul 2015 20:23:40 GMT', 'content-type': 'application/json'}
        Body: {"trust": {"impersonation": true, "roles_links": {"self": "http://host:port/v3/OS-TRUST/trusts/91a63490e5374c618613a3d60a9d4c1a/roles", "previous": null, "next": null}, "deleted_at": null, "trustor_user_id": "c873c9f351de4c348806f7aa3c462a68", "links": {"self": "http://host:port/v3/OS-TRUST/trusts/91a63490e5374c618613a3d60a9d4c1a"}, "roles": [{"id": "d10f03faf42b48a18392296ec8eeba07", "links": {"self": "http://host:port/v3/roles/d10f03faf42b48a18392296ec8eeba07"}, "name": "DelegatedRole--874289898"}], "remaining_uses": null, "expires_at": "2015-07-14T16:23:30.742911Z", "trustee_user_id": "0c8b6735b1d744318158286608d0c8b3", "project_id": "4770e24cb4894707a20760fc35373f79", "id": "91a63490e5374c618613a3d60a9d4c1a"}}
2015-07-14 16:23:31,067 12281 DEBUG [tempest_lib.common.rest_client] Request (TrustsV3TestJSON:test_trust_expire): 404 GET http://host:port/v3/OS-TRUST/trusts/91a63490e5374c618613a3d60a9d4c1a 0.031s
    Request - Headers: {'Content-Type': 'application/json', 'Accept': 'application/json', 'X-Auth-Token': '<omitted>'}
        Body: None
    Response - Headers: {'status': '404', 'content-length': '115', 'vary': 'X-Auth-Token', 'server': 'Apache/2.4.10 (Debian)', 'connection': 'close', 'date': 'Tue, 14 Jul 2015 20:23:37 GMT', 'content-type': 'application/json'}
        Body: {"error": {"message": "Could not find trust: 91a63490e5374c618613a3d60a9d4c1a", "code": 404, "title": "Not Found"}}
{code}

[Adam Young (ayoung)]

While this was by design, to allow audit-ability, the new addition of the auditing capabilities trumps the need to keep the trusts in the database.

As trust get used more ,they are going to start filling up the database, and will need to be flushed, just as tokens do, though far less often.

[Gilles Dubreuil (gdubreui)]

Per apal's comment, an expired trust is effectively visible in the trusts list but it cannot be shown.
This breaks down any automated process parsing the trusts and trying to obtain details for each.
For instance the puppet-keystone project for the trust provider.

# openstack trust list -f csv
"ID","Expires At","Impersonation","Project ID","Trustee User ID","Trustor User ID"
"19437fbe20e14a7783d70ba5e28d43bf","",False,"78e22bb71862481dbe8335b4ce4551e8","ac994e5701d644b6a3ac78c9dd1ad04a","24b047f52ff94029923f7f0ea982f03f"
"3b98a91e56df4f4f879ae982083fdfd4","2015-01-01T00:00:00.000000Z",False,"78e22bb71862481dbe8335b4ce4551e8","ac994e5701d644b6a3ac78c9dd1ad04a","24b047f52ff94029923f7f0ea982f03f"
"758d5ae236f647379e799bcbf6d9be7f","2010-01-01T00:00:00.000000Z",True,"78e22bb71862481dbe8335b4ce4551e8","ac994e5701d644b6a3ac78c9dd1ad04a","24b047f52ff94029923f7f0ea982f03f"

# openstack trust show 19437fbe20e14a7783d70ba5e28d43bf -f shell
deleted_at="None"
expires_at="None"
id="19437fbe20e14a7783d70ba5e28d43bf"
impersonation="False"
project_id="78e22bb71862481dbe8335b4ce4551e8"
redelegation_count="0"
remaining_uses="None"
roles="admin foo "
trustee_user_id="ac994e5701d644b6a3ac78c9dd1ad04a"
trustor_user_id="24b047f52ff94029923f7f0ea982f03f"

# openstack trust show 3b98a91e56df4f4f879ae982083fdfd4 -f shell
ERROR: openstack No trust with a name or ID of '3b98a91e56df4f4f879ae982083fdfd4' exists.

[jiaxi (tjxiter)]

Cannot delete trust with an expired date? I was debuging the reason.
Who can give me some tips?

[jiaxi (tjxiter)]

The trust which is expired can't be deleted.
The reason is that before delete it, we should find it, but can't find it.

So the bug lies at we can't find the expired trust.,

[Gilles Dubreuil (gdubreui)]

I sort of understand the not delete reason based on expiry, it's like a enable/disable sort of flag
but not being to list them is really weird

[Gilles Dubreuil (gdubreui)]

Still need a flush method.
How is the user (admin) supposed to delete the expired trusts without hacking the database directly?

[jiaxi (tjxiter)]

Gilles Dubreuil (gdubreui)
How is the user (admin) supposed to delete the expired trusts without hacking the database directly?

------------------------------------
It's werid to say that..

We should delete expored trusts like the other in use trusts with hacking the database.

[jiaxi (tjxiter)]

The command can't be edit.It's too bad.

expored --> expired

[Gilles Dubreuil (gdubreui)]

For an easy flush method (test/dev env):

set the system date prior to the expiry date of the trust to remove!

[jiaxi (tjxiter)]

Gilles Dubreuil (gdubreui) :

You want to set the system date prior to the expiry date of the trust ???

First , you should find the trust.

But the a expired trust can't be found.

[Gilles Dubreuil (gdubreui)]

@jiaxi,

Expired trust can be listed: using OSC => openstack list
The expires_at is visible there.

[Gilles Dubreuil (gdubreui)]

My latest update,

Besides a need to easily flush the database out of the expired trusts record,
the fact remains that the expired trust probably shouldn't be visible in the list by default.
Although, I'm not sure if that should be handled by a client (OSC) or via Keystone.

Here's a real case example:
In puppet-keystone, the provider, in order to handle trusts,
must 'purge' the expired trusts from the list before being able to handle them since the expired ones cannot be showed individually.

---------------------------------
  def self.expired_purge(list)
    list.delete_if do |trust|
      unless trust[:expires_at] == ''
        trust[:expires_at] <= Time.new.strftime("%Y-%m-%dT%H:%M:%S.%6NZ")
      end
    end
  end

  def self.roles_get(list)
    list.each do |trust|
     details = request('trust', 'show', trust[:id])
     trust.merge!(:roles => details[:roles].split(' '))
    end
  end

  def self.instances
    list = request('trust', 'list')
    # bug/1473292
    expired_purge(list)
    roles_get(list)
...
  end
---------------------------------

[Adam Young (ayoung)]

removing it from list is trivial:

http://git.openstack.org/cgit/openstack/keystone/tree/keystone/trust/backends/sql.py#n156

Just like it filters by the ones deleted, it can also filter by:

now = timeutils.utcnow()
            if now > ref.expires_at:

But thie rows will still be in the database. A purge is called for, the question is 'when?'

[Adam Young (ayoung)]

I think the query would be:

now = timeutils.utcnow()
trusts = session.query(TrustModel).filter(TrustModel.deleted_at == None, TrustModel.expires_at<=now )

[Tony Wang (wwwjfy)]

For when to flush trust, I +1 for when token_flush is executed.
And I think @ayoung's solution is the right way to go, that outdated/invalid data should not be returned at all.

[Lance Bragstad (lbragstad)]

What's wrong with adding a keystone-manage trust_flush method? We have that approach with expired persistent tokens as a way to clean up tokens from the backend. That should address the sql backend issue.

[Steve Martinelli (stevemar)]

Automatically unassigning due to inactivity.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/589378

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/589378
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=8232dabcf9cf182c4dc34eafecbe5b60b2438ed1
Submitter: Zuul
Branch: master

commit 8232dabcf9cf182c4dc34eafecbe5b60b2438ed1
Author: Vishakha Agarwal <email address hidden>
Date: Tue Aug 7 12:16:51 2018 +0530

    Implement Trust Flush via keystone-manage.

    Creates a cli entry 'trust_flush' which removes
    all expired trusts.

    Change-Id: I1c85b67d24e05db86c85e722fbd773a411c24ac4
    Closes-Bug: #1473292

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/604970

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/604970
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=6aba78a7a213335bf38ddfb4c0e1ef329653305e
Submitter: Zuul
Branch: master

commit 6aba78a7a213335bf38ddfb4c0e1ef329653305e
Author: Vishakha Agarwal <email address hidden>
Date: Tue Sep 25 10:45:47 2018 +0530

    Purge soft-deleted trusts

    This patch adds the functionality for purging both
    expired trusts as well as non-expired soft-deleted
    trusts, since those soft-deleted trusts are
    as likely to bloat the database as expired trusts.

    Related to patch-
    https://review.openstack.org/#/c/589378/

    Change-Id: I3c74f2345a944ce03a8189c4e66c3c37350cd34f
    Related-Bug: #1473292

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/607897

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/607897
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=7e42d333c7c7205ff5ae4b4276f5f81db76278aa
Submitter: Zuul
Branch: master

commit 7e42d333c7c7205ff5ae4b4276f5f81db76278aa
Author: Vishakha Agarwal <email address hidden>
Date: Thu Oct 4 15:55:01 2018 +0530

    Adding 'date' for trust_flush

    This patch adds functionality to purge
    expired and soft-deleted trusts older
    than the given date.

    Change-Id: I0bd47e57f8650182e38b4f70e04cb53338fce474
    Related-Bug: #1473292

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.openstack.org/613485

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.openstack.org/614065

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: stable/rocky
Review: https://review.openstack.org/614068

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/614716

[OpenStack Infra (hudson-openstack)]

Change abandoned by Vishakha Agarwal (<email address hidden>) on branch: stable/rocky
Review: https://review.openstack.org/613485

[OpenStack Infra (hudson-openstack)]

Change abandoned by Vishakha Agarwal (<email address hidden>) on branch: stable/rocky
Review: https://review.openstack.org/614065

[OpenStack Infra (hudson-openstack)]

Change abandoned by Vishakha Agarwal (<email address hidden>) on branch: stable/rocky
Review: https://review.openstack.org/614068

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/614716
Committed: https://git.openstack.org/cgit/openstack/keystone/commit/?id=25e0e1474ae736c5212cda2f7bd7fda31c3749b8
Submitter: Zuul
Branch: master

commit 25e0e1474ae736c5212cda2f7bd7fda31c3749b8
Author: Vishakha Agarwal <email address hidden>
Date: Thu Nov 1 14:20:33 2018 +0530

    Add abstract method in trusts base.py

    This is a follow-up patch to add an abstract
    method for flush_expired_and_soft_deleted_trusts
    in base.py to avoid any burst in case people
    supplying their own backend.

    Change-Id: Ib326abddce239e87cd4cd37d06fa3b55112ee134
    Related-Patch: https://review.openstack.org/#/c/589378/
    Related-Bug: #1473292

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/keystone 15.0.0.0rc1 release candidate.