Metadata caching is fundamentally broken

[Sven Anderson]

The caching backend for memcached/dogpile stores whole Python objects by creating a deep copy. Metadata is cached by storing the metadata Python object, and therefore the whole object tree underneath, including the "instance" attribute. Whenever the metadata is fetched, it also fetches the old version of instance with it, and works on it. This has several implications, like for example lazy fetched fields of the "instance" object are never cached, if they are not pre-fetched before caching. Also it might implicate security issues.

It would be probably a better approach to actually cache the DB query results only, and not the whole populated "living" object.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/285530

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/285562

[Markus Zoeller (markus_z) (mzoeller)]

@Sven Anderson:

It seems you are actively working on that [1], thanks for your efforts.
I'm switching the status to "In Progress" and set you as assignee to
signal that.
I don't yet now which prio this should have (I tend to "high"), but
let me check that with other folks in #openstack-nova.

References:
[1] https://review.openstack.org/#/q/topic:bug/1549814

[Markus Zoeller (markus_z) (mzoeller)]

This bug is suspicious, so I add the "mitaka-rc-potential" tag for potential RC blockers.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Michael Still (<email address hidden>) on branch: master
Review: https://review.openstack.org/285562
Reason: This code hasn't been updated in a long time, and is in merge conflict. I am going to abandon this review, but feel free to restore it if you're still working on this.

[OpenStack Infra (hudson-openstack)]

Change abandoned by Michael Still (<email address hidden>) on branch: master
Review: https://review.openstack.org/285530
Reason: This code hasn't been updated in a long time, and is in merge conflict. I am going to abandon this review, but feel free to restore it if you're still working on this.

[Sean Dague (sdague)]

Caching models like this probably really should see a spec just to think through the access patterns, especially as people keep proposing to make MD writable from the guest, which impacts this.