UDEV rules ignored for ethernet (eth*) devices

[Zakhar]

Binary package hint: yelp

Hello,

I'm trying to set my network interfaces so that they don't get random every boot.
(eg assign eth0 to a network interface with a given MAC addr, and eth1 to the other one)

I threw in a udev rule (in fact just modified the rules that was automatically generated and set the ethX in it) but the system ignores my udev rule.

Here is all the info :

$cat /etc/udev/rules.d/70-persistent-net.rules
# This file maintains persistent names for network interfaces.
# See udev(7) for syntax.
#
# Entries are automatically added by the 75-persistent-net-generator.rules
# file; however you are also free to add your own entries.

# PCI device 0x11ab:0x4362 (sky2)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:18:f3:cd:83:2c", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"

# PCI device 0x11ab:0x4362 (sky2)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:18:f3:cd:73:8c", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:18:f3:cd:83:2c
          UP BROADCAST MULTICAST MTU:1500 Metric:1
          Packets reçus:0 erreurs:0 :0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 lg file transmission:1000
          Octets reçus:0 (0.0 B) Octets transmis:0 (0.0 B)
          Interruption:19

eth1 Link encap:Ethernet HWaddr 00:18:f3:cd:73:8c
          inet adr:192.168.0.130 Bcast:192.168.0.255 Masque:255.255.255.0
          adr inet6: fe80::218:f3ff:fecd:738c/64 Scope:Lien
          UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
          Packets reçus:1101 erreurs:0 :0 overruns:0 frame:0
          TX packets:1156 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 lg file transmission:1000
          Octets reçus:682128 (682.1 KB) Octets transmis:336863 (336.8 KB)
          Interruption:16

We see that the MAC addr 00:18:f3:cd:83:2c should have eth1 according to udev, but in fact it gets eth0

udevadm info gets me nothing wrong about my rules selection criteria:

$ udevadm info -a -p /sys/class/net/eth0

Udevadm info starts with the device specified by the devpath and then
walks up the chain of parent devices. It prints for every device
found, all possible attributes in the udev rules key format.
A rule to match, can be composed by the attributes of the device
and the attributes from one single parent device.

  looking at device '/devices/pci0000:00/0000:00:1c.3/0000:04:00.0/net/eth0':
    KERNEL=="eth0"
    SUBSYSTEM=="net"
    DRIVER==""
    ATTR{addr_len}=="6"
    ATTR{dev_id}=="0x0"
    ATTR{ifalias}==""
    ATTR{iflink}=="2"
    ATTR{ifindex}=="2"
    ATTR{features}=="0x109a3"
    ATTR{type}=="1"
    ATTR{link_mode}=="0"
    ATTR{address}=="00:18:f3:cd:83:2c"
    ATTR{broadcast}=="ff:ff:ff:ff:ff:ff"
    ATTR{carrier}=="0"
    ATTR{dormant}=="0"
    ATTR{operstate}=="down"
    ATTR{mtu}=="1500"
    ATTR{flags}=="0x1003"
    ATTR{tx_queue_len}=="1000"

  looking at parent device '/devices/pci0000:00/0000:00:1c.3/0000:04:00.0':
    KERNELS=="0000:04:00.0"
    SUBSYSTEMS=="pci"
    DRIVERS=="sky2"
    ATTRS{vendor}=="0x11ab"
    ATTRS{device}=="0x4362"
    ATTRS{subsystem_vendor}=="0x1043"
    ATTRS{subsystem_device}=="0x8142"
    ATTRS{class}=="0x020000"
    ATTRS{irq}=="30"
    ATTRS{local_cpus}=="00000000,00000003"
    ATTRS{local_cpulist}=="0-1"
    ATTRS{modalias}=="pci:v000011ABd00004362sv00001043sd00008142bc02sc00i00"
    ATTRS{numa_node}=="0"
    ATTRS{broken_parity_status}=="0"
    ATTRS{msi_bus}==""

I am on Karmic 64

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=9.10
DISTRIB_CODENAME=karmic
DISTRIB_DESCRIPTION="Ubuntu 9.10"

$ uname -a
Linux zakhar-desktop 2.6.31-21-generic #59-Ubuntu SMP Wed Mar 24 07:28:27 UTC 2010 x86_64 GNU/Linux

$ lspci
(.....)
03:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E Gigabit Ethernet Controller (rev 20)
04:00.0 Ethernet controller: Marvell Technology Group Ltd. 88E8053 PCI-E Gigabit Ethernet Controller (rev 20)

$dmesg
(...)
[ 1.738451] sky2 driver version 1.23
[ 1.738545] sky2 0000:04:00.0: PCI INT A -> GSI 19 (level, low) -> IRQ 19
[ 1.738560] sky2 0000:04:00.0: setting latency timer to 64
[ 1.738597] sky2 0000:04:00.0: Yukon-2 EC chip revision 2
[ 1.738673] alloc irq_desc for 30 on node 0
[ 1.738676] alloc kstat_irqs on node 0
[ 1.738690] sky2 0000:04:00.0: irq 30 for MSI/MSI-X
[ 1.739317] sky2 eth0: addr 00:18:f3:cd:83:2c
[ 1.739381] sky2 0000:03:00.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16
[ 1.739392] sky2 0000:03:00.0: setting latency timer to 64
[ 1.739420] sky2 0000:03:00.0: Yukon-2 EC chip revision 2
[ 1.739495] alloc irq_desc for 31 on node 0
[ 1.739497] alloc kstat_irqs on node 0
[ 1.739508] sky2 0000:03:00.0: irq 31 for MSI/MSI-X
[ 1.740214] sky2 eth1: addr 00:18:f3:cd:73:8c

Here we see on dmesg that it does NOT care of what is in udev. It takes the first interface, which here is the one finishing by 83:2c, and gives it eth0 although udev says it should be eth1

The problem is interfaces are presented randomly by my BIOS to the system according to the moment when they are ready. Yesterday, they were reversed, and thus got right... but it is just randomly right.

This is quite bad as I'd like to use this PC as a router, and thus I have to know which interface is eth0 and which is eth1.

I suppose this bug was never spotted (or maybe it's a "feature"!) because you need a stupid BIOS (like mine) that gives the ethernet devices randomly to the O.S.
If the BIOS gave them in a reliable order, there should still be a UDEV bug, but I might not have noticed it. And if I did, it would have been easy to do the workaround.

Best regards.

ProblemType: Bug
Architecture: amd64
Date: Sat Jun 12 10:18:40 2010
DistroRelease: Ubuntu 9.10
ExecutablePath: /usr/bin/yelp
InstallationMedia: Ubuntu 9.10 "Karmic Koala" - Release amd64 (20091027)
NonfreeKernelModules: nvidia
Package: yelp 2.28.0-0ubuntu2
ProcEnviron:
 LANG=fr_FR.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 2.6.31-22.60-generic
SourcePackage: yelp
Uname: Linux 2.6.31-22-generic x86_64
XsessionErrors:
 (gnome-settings-daemon:2338): GLib-CRITICAL **: g_propagate_error: assertion `src != NULL' failed
 (nautilus:2669): Eel-CRITICAL **: eel_preferences_get_boolean: assertion `preferences_is_initialized ()' failed
 (polkit-gnome-authentication-agent-1:2699): GLib-CRITICAL **: g_once_init_leave: assertion `initialization_value != 0' failed
 (thunderbird-bin:2933): GLib-WARNING **: g_set_prgname() called multiple times
 (firefox:3055): GLib-WARNING **: g_set_prgname() called multiple times

[Pedro Villavicencio (pedro)]

not a yelp issue, let's assign it to udev for now.

[Zakhar (alainb06)]

Thank's Pedro!
I confirm, this is not a yelp bug but a udev bug.

My bad for not having checked the category before posting.

[Zakhar (alainb06)]

I have a clue to this bug (and also a workaround).

I tried changing my rules instead of using eth0 and eth1 to using eth1 and eth2

Basically, I didn't change the declaration of eth1, and moved the one of eth0 to eth2.

So now what happens.

83:2C starts first, it gets eth0 (eth1 on udev)
then the other gets eth1 (eth2 on udev).

Then udev comes and says :

[ 19.666153] udev: renamed network interface eth1 to eth2
[ 19.696240] udev: renamed network interface eth0_rename to eth1

If the order is reverse at startup we get :

73:8C starts first and gets eth0 (eth2 on udev)
the other gets eth1 (eth1 on udev, this is already OK then)

So now udev has only one change to make :

[ 19.666153] udev: renamed network interface eth0 to eth2

Problem is (I suppose), when you assign eth0 and eth1 and it started in the reverse order, udev would have to make 3 moves to change the interfaces :

[ 19.666153] udev: renamed network interface eth0 to eth0_rename
[ 19.696240] udev: renamed network interface eth1 to eth0
[ 19.696240] udev: renamed network interface eth0_rename to eth1

Classic exchange of two variables works in 3 moves.
I bet this is why I have an interface that's called now : eth0_rename.

Some sort of bug around this exchange might have happened.

Well, anyway with eth1 and eth2 it seems to work whichever interface starts first.

[Martin Pitt (pitti)]

There have been some fixes in the net device renaming part some months ago. Does this still happen with oneiric?

[Zakhar (alainb06)]

Thanks for the news on this bug.

Although I reported it, the correction became of very low priority for me as I solved the problem with "hardware": bought a Gigabyte Ethernet switch ;-)

-very useful and not that expensive, around 30€-

I would gladly do the test on Oneiric, but Oneiric fails to work on my PC (version 64 bit). In fact I get a complete freeze after say randomly 15min of use or less. Same thing on both my desktop and laptop. Complete freeze means that keayboard is not even responsive, thus I can't properly switch to console (CTRL-ATL-F1) to do clean sudo shutdown, and I have to switch off electrically.
To preserve my hardware, I reverted to Maverick (for video playback) and still on Lucid for my main usage.

I don't have Maverick on my desktop (the one with dual eth), only Lucid.

So if the bug was corrected on Lucid, I can do a test, eitherwise I'll wait for a while that they stabilise Oneiric to do another trial on this.

Thanks again.

and SUMMARY : as far as I am concerned, you can move the bug to "low priority"

[Martin Pitt (pitti)]

Thanks, keeping as "incomplete" for now then.

[Peter Hendrickson (pdh-wiredyne)]

I have one machine which is experiencing this problem and I know of another which is having
it, too. This is a fairly bad problem with troubling security implications. Imagine a router/firewall
setup which allows people access to services internally. If the interfaces are swapped, internal
services would be exposed to the whole world. Can we boost the importance of this bug?

If I disable NetworkManager the problem goes away. I use /etc/network/interfaces to set
the network information statically. (This problem is now solved for my purposes, but it
would be a good idea to track this down for the benefit of others.)

The swap of the interfaces is intermittent. It fails maybe one in ten times. However, when
I had knockd and snort starting, it would fail much more frequently. (Both of which access
the network interface when they start.) Furthermore, I would find messages like this in /var/log/syslog:
> Nov 15 10:48:35 opdubuntu udevd[469]: changing net interface name from 'eth1' to 'eth2'
> Nov 15 10:48:35 opdubuntu udevd[469]: error changing net interface name eth1 to eth2: Device or resource busy

After disabling the start of knockd, snort, and finally NetworkManager only this message appears:
> Nov 16 10:00:33 opdubuntu udevd[384]: changing net interface name from 'eth1' to 'eth0'

I believe that what is happening is that something is seizing one or the other of the network devices
and causing udev to give up on its renaming attempt or to complete it poorly.

I looked in the sources for the udev package and found that the "changing" messages are emitted
by rename_netif() in udev-173/udev/udev-event.c:846. This routine unfortunately aggregates
multiple errors into an "out:" section at the end, so it's hard to tell exactly what went wrong. My
guess is that the ioctl() call on line 867 has failed:
> err = ioctl(sk, SIOCSIFNAME, &ifr);

Suggestion A: maybe udevd could emit a more helpful error? Something along the lines of "ioctl()
failed with device X."

/etc/udev/rules.d/70-persistent-net.rules contains and is apparently what causes the changing
of the interfaces:
> # This file was automatically generated by the /lib/udev/write_net_rules
> # program, run by the persistent-net-generator.rules rules file.
> #
> # You can modify it, as long as you keep each rule on a single
> # line, and change only the value of the NAME= key.
>
> # PCI device 0x10ec:0x8139 (8139too)
> SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:50:fc:3d:42:1e", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth1"
>
> # PCI device 0x14e4:0x170c (b44)
> SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="00:13:72:32:43:e9", ATTR{dev_id}=="0x0", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

That file is generated automatically by /lib/udev/rules.d/75-persistent-net-generator.rules. However,
I find a curious prohibition in udev(7):
> NAME
> What a network interface should be named.
>
> Also, as a temporary workaround, this is what a device node should
> be named; usually the kernel provides the defined node name or
> creates and removes the node before udev even receives any event.
> Changing the node name from ...

[Launchpad Janitor (janitor)]

[Expired for udev (Ubuntu) because there has been no activity for 60 days.]