netaddr cached indexes are not coherent when changes occur to OUI or IAB data

[Mike Pontillo]

For some background, see Launchpad bug #1655447.[1]

The `netaddr.eui.ieee` package is responsible for indexing IEEE data at build time, and generating index files that the `netaddr` upstream code ships with.

This works fine until you have a long-running process relying on the data. For long-running processes, the index is cached the first time the OUI class is used (or the `netaddr.eui.ieee` package is imported). That means if `netaddr` is updated to a more recent version while the process is running, undefined behavior occurs until the process is restarted. This is because the cache becomes out-of-sync with the IEEE data on disk. (For example, we have observed `netaddr` crashing with an `IndexError` in Launchpad bug #1655049.[2])

In order to use `netaddr` in a long-running process, the cache must be re-read from disk if it changes.

Ideally, `netaddr` could also employ some type of integrity check, such as metadata in the cache file to ensure it is consistent with the data on-disk. Since a thorough integrity check might be expensive enough to defeat the purpose of an on-disk index, a basic sanity check (such as file size; possibly a hash of a subset of the file) should be sufficient.

Alternatively, the on-disk cache of the index could be dropped, and it could be calculated in memory. (Then recalculated if the data file changes.) However, this may be a problem on embedded systems without much CPU power.

---

[1]: https://launchpad.net/bugs/1655447
[2]: https://launchpad.net/bugs/1655049

[Mike Pontillo (mpontillo)]

Marking 'Opinion' for MAAS, since we may need to patch this in MAAS until a fix lands upstream in `netaddr`. I'd really rather not, if the fix can get into -updates.