netaddr cached indexes are not coherent when changes occur to OUI or IAB data

Bug #1655452 reported by Mike Pontillo on 2017-01-10
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
MAAS
Undecided
Unassigned
netaddr
New
Unknown
python-netaddr (Ubuntu)
Undecided
Unassigned

Bug Description

For some background, see Launchpad bug #1655447.[1]

The `netaddr.eui.ieee` package is responsible for indexing IEEE data at build time, and generating index files that the `netaddr` upstream code ships with.

This works fine until you have a long-running process relying on the data. For long-running processes, the index is cached the first time the OUI class is used (or the `netaddr.eui.ieee` package is imported). That means if `netaddr` is updated to a more recent version while the process is running, undefined behavior occurs until the process is restarted. This is because the cache becomes out-of-sync with the IEEE data on disk. (For example, we have observed `netaddr` crashing with an `IndexError` in Launchpad bug #1655049.[2])

In order to use `netaddr` in a long-running process, the cache must be re-read from disk if it changes.

Ideally, `netaddr` could also employ some type of integrity check, such as metadata in the cache file to ensure it is consistent with the data on-disk. Since a thorough integrity check might be expensive enough to defeat the purpose of an on-disk index, a basic sanity check (such as file size; possibly a hash of a subset of the file) should be sufficient.

Alternatively, the on-disk cache of the index could be dropped, and it could be calculated in memory. (Then recalculated if the data file changes.) However, this may be a problem on embedded systems without much CPU power.

---

[1]: https://launchpad.net/bugs/1655447
[2]: https://launchpad.net/bugs/1655049

Mike Pontillo (mpontillo) wrote :

Marking 'Opinion' for MAAS, since we may need to patch this in MAAS until a fix lands upstream in `netaddr`. I'd really rather not, if the fix can get into -updates.

summary: - netaddr cached indexes are not coherent when changes occr to OUI or IAB
- indexes
+ netaddr cached indexes are not coherent when changes occur to OUI or IAB
+ data
Changed in maas:
status: Confirmed → Opinion
Changed in python-netaddr (Ubuntu):
status: New → Confirmed
Changed in netaddr:
status: Unknown → New
Changed in maas:
importance: Critical → Undecided
no longer affects: maas/2.1
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.