The drivers_private_storage is expected to be able to save new entries using the same keys as previously deleted entries.
Unfortunately, the misinterpretation of a parameter in the manila DB API led to a bug preventing the above behavior from working properly.
This line is the misinterpretation which confirms the expected behavior: https://github.com/openstack/manila/blob/3a9f67a90f8a8dac3cc08b1f19dd66b40da303cb/manila/db/sqlalchemy/api.py#L3508
The "read_deleted=None" parameter prevents the following line from being executed because deleted entries are not retrieved: https://github.com/openstack/manila/blob/3a9f67a90f8a8dac3cc08b1f19dd66b40da303cb/manila/db/sqlalchemy/api.py#L3519
One very easy way to reproduce this problem is to add "db_api.driver_private_data_delete(self.ctxt, test_id)" between the 2 update calls in https://github.com/openstack/manila/blob/3a9f67a90f8a8dac3cc08b1f19dd66b40da303cb/manila/tests/db/sqlalchemy/test_api.py#L1651
Driver private data was designed to store items that drivers can manipulate, along the design, we may have percolated the idea that driver_private_data is analogous to any other table that is part of the Manila database schema, it isn't.
Manila database has hooks, and there are genuine use cases when Administrators of a Manila installation can access or manipulate data outside of the manila-api (that's the reason why manila-manage exists). Our database schema thus has a contract on upgrades or retention for audits. The same thing cannot be said of driver_
With that said, I think we should hard delete rows from this store, there is absolutely no valid reason why data deleted by the driver needs to be accessible after deletes.