FileStorageCache is necessary
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
pyjuju |
Confirmed
|
Wishlist
|
Unassigned |
Bug Description
It'd be great to have an interface to the remote FileStorage implementations which dealt with downloading files, and caching them locally. Here is a suggested interface:
class FileStorageCache:
def __init__(self, file_storage, local_cache_path):
"""
@param file_storage: FileStorage instance to wrap.
@param local_path: Local directory to use as the cache.
"""
def get(self, name):
"""See FileStorage.get()'s documentation.
This method behaves the same way, but will avoid downloading the file
a second time if it was already available locally.
"""
Among other things, this will be used to enable the Machine Agent to download formulas just once, as defined in bug #617535.
Changed in ensemble: | |
status: | New → Confirmed |
importance: | Undecided → Low |
Changed in ensemble: | |
assignee: | nobody → Jim Baker (jimbaker) |
Changed in ensemble: | |
milestone: | 0.2 → 0.3 |
Changed in ensemble: | |
assignee: | Jim Baker (jimbaker) → nobody |
Changed in ensemble: | |
status: | In Progress → Confirmed |
Changed in ensemble: | |
milestone: | 0.3 → none |
Changed in ensemble: | |
importance: | Low → Wishlist |
What is the desired caching policy? Some standard questions:
1. File consistency. Are these files effectively immutable (ignoring dev of course), very infrequently changed (so a global expire as seen in programs using SIGHUP), or otherwise? If the latter, do we need to consider that file A and file B depend on each other such that loading A' needs to have a consistently loaded B'?
2. Size of cache values. Are the cached objects both individually small and their number small? Or do we need some variant on LRU?