Arbitrarily-large resource properties should be allowed in events
Bug #1524013 reported by
Zane Bitter
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenStack Heat |
Fix Released
|
Medium
|
Crag Wolfe |
Bug Description
For bug 1493858 we implemented a workaround that ensures that events still get stored even when the resource properties are too large to fit. But for the long term we really need to implement a database migration that would allow us to start storing arbitrarily large properties in the events.
Given that we also store the latest properties in each resource, maybe we could take this as an opportunity to deduplicate the data and have a separate Properties table that could be referenced from both events and (in the case of the latest properties) the resource. (This difficulty here would be ensuring that they get removed at the correct time, in both the legacy and convergence workflows.)
Changed in heat: | |
milestone: | none → newton-1 |
Changed in heat: | |
milestone: | newton-1 → newton-2 |
Changed in heat: | |
milestone: | newton-2 → newton-3 |
Changed in heat: | |
milestone: | newton-3 → newton-rc1 |
Changed in heat: | |
milestone: | newton-rc1 → ocata-1 |
Changed in heat: | |
milestone: | ocata-1 → ocata-2 |
Changed in heat: | |
milestone: | ocata-2 → ocata-3 |
Changed in heat: | |
assignee: | Crag Wolfe (cwolfe) → Zane Bitter (zaneb) |
Changed in heat: | |
assignee: | Zane Bitter (zaneb) → Crag Wolfe (cwolfe) |
To post a comment you must log in.
As stated in the bug, the issue with the events table is that the properties_ data column is a heat.db. sqlalchemy. types.Json
resource_properties is constrained to 2^16 bytes. However, the
resource.
type or 2^32 bytes (4 GB). I think for our purposes, a 4GB column will
satisfy the "arbitrarily large properties" requirement.
So, we can move the properties_data into a separate table, call it properties_ data (rpd). Given that assumption, there are
resource_
subtle variations in how we could structure the data.
We could have foreign key column (fk_resource_id) in rpd to refer to propeties_ data). One possibility is to enforce the policy
resource.id. But we'll need a way of identifying what the current rpd
entry is (i.e. what is currently stored in
resource.
where only the current rpd entry does have a not-null fk_resource_id.
The events table is more straightforward. We can just replace the resource_ properties with a foreign key column that points to
events.
rpd.id. However, We will need to take care to prune an rpd entry
whenever its event is pruned.
I plan on getting started on the above soon. If anyone has strong
opinions on the above, it would be good to hear them now. :-)