[2.1] netaddr raises IndexError when looking up some OUI registrations
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
| MAAS |
Critical
|
LaMont Jones | ||
| 2.1 |
Critical
|
Unassigned | ||
| maas (Ubuntu) |
Undecided
|
MAAS Maintainers |
Bug Description
Distro: Ubuntu 16.10
MAAS version: 2.1.2+bzr5555-
For some node MAC addresses (I have a relatively old servers) I am unable to open MAAS dashboard page or Nodes page due to
"list index out of range"
error.
2017-01-
2017-01-
Traceback (most recent call last):
File "/usr/lib/
self.
File "/usr/lib/
return target()
File "/usr/lib/
task()
File "/usr/lib/
task()
--- <exception caught here> ---
File "/usr/lib/
result = inContext.theWork()
File "/usr/lib/
inContext.
File "/usr/lib/
return self.currentCon
File "/usr/lib/
return func(*args,**kw)
File "/usr/lib/
return func(*args, **kwargs)
File "/usr/lib/
result = func(*args, **kwargs)
File "/usr/lib/
return func_outside_
File "/usr/lib/
return func(*args, **kwargs)
File "/usr/lib/
return func(*args, **kwds)
File "/usr/lib/
return super(MachineHa
File "/usr/lib/
for obj in objs
File "/usr/lib/
for obj in objs
File "/usr/lib/
return self.dehydrate(obj, data, for_list=for_list)
File "/usr/lib/
obj, data, for_list=for_list)
File "/usr/lib/
data[
File "/usr/lib/
return get_vendor_
File "/usr/lib/
return data.oui.
File "/usr/lib/
return OUI(self.value >> 24)
File "/usr/lib/
self.
File "/usr/lib/
record['org'] = line.split(None, 2)[2]
builtins.
For now I have worked around the issue by catching IndexError in "/usr/lib/
But this probably needs some investigation and fix - without the workaround I cannot use MAAS.
Related branches
- Mike Pontillo (community): Approve on 2017-01-09
-
Diff: 35 lines (+16/-1)2 files modifiedsrc/maasserver/utils/mac.py (+3/-1)
src/maasserver/utils/tests/test_mac.py (+13/-0)
- LaMont Jones (community): Approve on 2017-01-09
-
Diff: 35 lines (+16/-1)2 files modifiedsrc/maasserver/utils/mac.py (+3/-1)
src/maasserver/utils/tests/test_mac.py (+13/-0)
description: | updated |
LaMont Jones (lamont) wrote : | #1 |
LaMont Jones (lamont) wrote : | #2 |
I found some that exhibit the issue (00-22-59, 00-23-89) for starters.
LaMont Jones (lamont) wrote : | #3 |
MAAS was incomplete in the workaround for Bug#1628761.
Changed in maas (Ubuntu): | |
status: | New → In Progress |
Changed in maas: | |
status: | New → In Progress |
Changed in maas (Ubuntu): | |
assignee: | nobody → MAAS Maintainers (maas-maintainers) |
Changed in maas: | |
assignee: | nobody → LaMont Jones (lamont) |
milestone: | none → 2.2.0 |
importance: | Undecided → Critical |
summary: |
- List index out of range on main page and nodes page + [2.1] List index out of range on main page and nodes page |
summary: |
- [2.1] List index out of range on main page and nodes page + [2.1] netaddr raises IndexError when looking up some OUIs |
summary: |
- [2.1] netaddr raises IndexError when looking up some OUIs + [2.1] netaddr raises IndexError when looking up some OUI registrations |
Changed in maas: | |
status: | In Progress → Fix Committed |
Mike Pontillo (mpontillo) wrote : | #4 |
Hm. I think the root cause of this might be that the index is not updated often enough (that is, it could be a Debian package mismatch).
The `netaddr` package defines the following:
ls -la /usr/lib/
lrwxrwxrwx 1 root root 38 Oct 23 2015 /usr/lib/
readlink -f /usr/lib/
/usr/share/
The `ieee-data` provides the actual data:
dpkg -S /usr/share/
ieee-data: /usr/share/
... so if one package is updated but not the other, the index in `netaddr` will become incorrect. I ran some tests with `netaddr` upstream and it seems like the library works fine with any OUI, even OUIs with incorrectly encoded characters in them, etc.
To test this theory, you could run netaddr as follows to regenerate the indexes:
sudo python3 -m netaddr.eui.ieee
You would need to restart MAAS after doing this, since netaddr keeps the indexes cached in memory.
LaMont Jones (lamont) wrote : | #5 |
Interesting point. Here is what I see:
[xenial]
ii ieee-data 20150531.1 all OUI and IAB listings
ii python3-netaddr 0.7.18-1 all manipulation of various common netwo
% python3
Python 3.5.2 (default, Nov 17 2016, 17:05:23)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import netaddr
>>> netaddr.
'Guangzhou New Postcom Equipment Co.,Ltd.'
[zesty]
ii ieee-data 20160613.1 all OUI and IAB listings
ii python3-netaddr 0.7.18-2 all manipulation of various common network
% python3
Python 3.5.2+ (default, Dec 13 2016, 14:16:35)
[GCC 6.2.1 20161212] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import netaddr
>>> netaddr.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/
return OUI(self.value >> 24)
File "/usr/lib/
self.
File "/usr/lib/
record['org'] = line.split(None, 2)[2]
IndexError: list index out of range
>>>
That is netaddr is not sufficiently hardened against the newer ieee-data. While it's conceivable that the actual bug is in ieee-data, it would seem that hardening the code to better deal with it would be good.
The entry that breaks reads:
00-22-59 (hex) Guangzhou New Postcom Equipment Co.,Ltd.
002259 (base 16) Guangzhou New Postcom Equipment Co.,Ltd.
while the same entry in the working file is:
00-22-59 (hex) Guangzhou New Postcom Equipment Co.,Ltd.
002259 (base 16) Guangzhou New Postcom Equipment Co.,Ltd.
(Note the leading pair of spaces...)
Mike Pontillo (mpontillo) wrote : | #6 |
Thanks for the test case. Here's what I saw after running `sudo python3 -m netaddr.eui.ieee` on my system:
>>> import netaddr
>>> netaddr.
'Guangzhou New Postcom Equipment Co.,Ltd.'
So I conclude that the indexes are out of date.
My guess is `netaddr` expects to ship with the entire OUI database, but the Debian maintainer didn't want several different copies of the OUI database around the filesystem.
ptylenda (tylenda-piotr) wrote : | #7 |
Thanks for quick response, I see you already have the MACs which cause problems, if needed I can print the ones which caused problem at my side
LaMont Jones (lamont) wrote : | #8 |
The actual source of the issue seems to be that netaddr indexes a file that it doesn't own (from ieee-data package), and then assumes that it won't change underneath it (either because of an ieee-data update, or a netaddr update in the case of a running process), which results in various exceptions and other errors from the code. I believe that a fresh bug is being filed against python-netaddr on the issue.
Mike Pontillo (mpontillo) wrote : | #9 |
I filed two additional bugs to track this.
(1) For the netaddr package, so that the indexes remain consistent with the IEEE data:
https:/
(2) For the bug in netaddr which allows a long-running process to retain an old cached index:
https:/
... also filed upstream, at:
Changed in maas: | |
status: | Fix Committed → Fix Released |
Can you provide the first 3 octets of one of the bad MACs? If you're not sure which one, the following is sure to catch it:
sudo maas-region dbshell cast(mac_ address as varchar) for 8) from maasserver_ interface;
select distinct substring(
This would appear to be a bug in the python3-netaddr, which MAAS should probably armor itself against.