Some objects are not deleted after functional test

[Jeremy Liu]

When running functional test, the generated orders and containers are not deleted.

[Pavlo Shchelokovskyy (pshchelo)]

I'd like to bump this very old issue with some comments and ideas.

What is happening is the following:

- the castellan Python API consumer calls 'create_key' or 'create_key_pair'
- castellan creates an order, in case of key pair also a container is being created
- castellan returns to the caller only the IDs of created secrets

As a result, the API consumer is not aware that any intermediate entities were created (orders, containers) and thus is not aware that when time comes to delete those secrets, those orders and containers should be deleted as well.

This results in orders and containers piling up, and what's more, even those deleted (soft-deleted) secrets can not be purged from the Barbican DB during maintenance as they are still referenced by the not yet deleted orders and containers. Also, possible existing quotas for orders and containers are being exhausted and cause unexpected problems when trying to 'create a secret'. This sutuation is most pronounced when castellan API callers are themselves automated systems like Nova or Cinder,
where the end user may not even be aware about any interaction with Barbican at all, user is just creating an encrypted volume, or an instance that happens to use encrypted local ephemeral storage in Nova.

So my idea would be the following:
if we assume that user is only ever interested in the secrets themselves, castellan should itself delete results of any intermediate steps taken to create those secrets, properly hiding any gory details of implementation from the user.

[Pavlo Shchelokovskyy (pshchelo)]

I've already filed a bug for Barbican in this regard https://storyboard.openstack.org/#!/story/2010625 but after more thinking, it seems that the problem is solely with castellan and how it leaves intermediate entities behind.

The only small problem on Barbican side is that for whatever reason, default policies do not allow every user who was able to create an order to also delete it (why???). This however can be accounted for (by ignoring such errors and logging warnings) or policies could be amended.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/openstack/castellan/+/877423

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/castellan/+/877423
Committed: https://opendev.org/openstack/castellan/commit/6637a8b75602b7b59fe0663ac3ff00a8293b34da
Submitter: "Zuul (22348)"
Branch: master

commit 6637a8b75602b7b59fe0663ac3ff00a8293b34da
Author: Pavlo Shchelokovskyy <email address hidden>
Date: Mon Sep 29 10:39:44 2025 +0200

    Cleanup intermediate orders and containers

    When creating keys or keypairs with Barbican, castellan creates
    intermediate entities like orders and containers, yet returns to the
    API caller only IDs of created secrets.

    As a result, the API consumer is not aware of those entities and the
    fact that they have to be deleted as well when secrets are deleted.
    This leads to orders and containers piling up, and also the deleted
    secrets can not be purged from DB as they are still being referenced
    by not deleted containers and orders. This is especially pronounced
    when the API caller is actually some other OpenStack service, so the
    user may not even be aware that Barbican was involved in the first
    place.

    This patch attempts to delete those intermediate entities before
    returning the secret IDs to the caller. For the sake of backward
    compatibility, and to account for a fact that with default Barbican
    policies, not every user who can create order can delete it too,
    any errors on such attempts are ignored and only logged as warnings.

    Closes-Bug: 1638018
    Change-Id: Icdaa44a29fdc59fce107078bff3a7544beb2b3c7
    Signed-off-by: Dejan Sanader <email address hidden>