[3.4.1][k8s] Consume secrets via CMR fails

[Pedro Guimarães]

Using a controller 3.4.1 to manage 2x models
1) "localhost" cloud with sysbench charm: https://pastebin.canonical.com/p/727fjHH9N9/
2) "k8s" cloud with mysql+mysql-router charms: https://pastebin.canonical.com/p/SJdyXDyhZd/

The mysql model is consuming the sysbench relation and fails with the logs:
unit-mysql-k8s-1: 20:54:22 ERROR juju.worker.uniter.context cannot apply changes: granting secrets access: cannot change access to "secret://266ce8ab-a7ee-4e6e-80e1-0d4ae6960408/co7fen0v1m728bg5cffg" for "application-sysbench": sharing consumer secrets across a cross model relation not supported
model-266ce8ab-a7ee-4e6e-80e1-0d4ae6960408: 20:54:22 DEBUG juju.worker.caasadmission received admission request for unit-mysql-k8s-1 of /v1, Kind=ServiceAccount in namespace database
model-266ce8ab-a7ee-4e6e-80e1-0d4ae6960408: 20:54:22 DEBUG juju.worker.caasadmission received admission request for unit-mysql-k8s-1 of /v1, Kind=ServiceAccount in namespace database
model-266ce8ab-a7ee-4e6e-80e1-0d4ae6960408: 20:54:22 DEBUG juju.worker.caasadmission received admission request for unit-mysql-k8s-1 of rbac.authorization.k8s.io/v1, Kind=Role in namespace database
model-266ce8ab-a7ee-4e6e-80e1-0d4ae6960408: 20:54:22 DEBUG juju.worker.caasadmission received admission request for unit-mysql-k8s-1 of rbac.authorization.k8s.io/v1, Kind=RoleBinding in namespace database
unit-mysql-k8s-1: 20:54:22 DEBUG juju.kubernetes.provider opening model "database".
unit-mysql-k8s-1: 20:54:22 ERROR juju.worker.uniter.operation hook "database-relation-changed" (via hook dispatching script: dispatch) failed: granting secrets access: cannot change access to "secret://266ce8ab-a7ee-4e6e-80e1-0d4ae6960408/co7fen0v1m728bg5cffg" for "application-sysbench": sharing consumer secrets across a cross model relation not supported

------------------------------------------------------------------------------------------------------

Full logs after the database model consumed the relation and related with mysql-router: https://pastebin.canonical.com/p/8XvhNG9DFV/

[Ian Booth (wallyworld)]

This should be fixed in 3.4.2. I'll mark as Fix Released on that basis.

[Marcelo Henrique Neppel (neppel)]

Hi, Ian!

I tested it on 3.4.2 with a test charm, and it still doesn't work. The test charm (described in the steps below) has an action to create and grant access to a secret through CMR (the action fails if I try to grant access to a secret created on the consumer side of the relation, which works correctly if the relation is not a CMR).

Those are the steps I used:

juju add-model dev
juju add-model dev1

git clone https://github.com/marceloneppel/consumer-secret-test-k8s-operator.git
cd consumer-secret-test-k8s-operator
charmcraft pack
juju deploy ./consumer-secret-test-k8s-operator_ubuntu-22.04-*.charm provider -m dev
juju deploy ./consumer-secret-test-k8s-operator_ubuntu-22.04-*.charm consumer -m dev1

juju switch dev
juju offer provider:provider
juju consume -m dev1 dev.provider

juju relate -m dev1 provider consumer:consumer

juju run -m dev1 consumer/0 share-secret # This leads to the error

juju run provider/0 share-secret # This works fine, as it’s the provider side of the relation

[Ian Booth (wallyworld)]

CMR secrets currently do not support granting access to secrets from the consumer side. You can only grant access to secrets created by the offering application. There's no guarantee of being about to route traffic from the offering controller to the consuming controller, so there's a lot of potential foot guns there. If it were enabled, it would work for some scenarios but fail in many others and so careful thought would be needed on how to expose this such that the user experience was not poor.