macvtap creation issue using net namespaces

[zappacor]

Hi there,

There seems to be a bug when trying to create two macvtap links that are contained within different net namespaces. This is what I'm facing and, at the same time, the steps to reproduce the issue:

1 - Create two net namespaces:
#
root@rjz-nuc:~# ip netns add bng1
root@rjz-nuc:~# ip netns add bng2

2 - Create two veth links, one in one net NS and another one in the other net NS:
#
root@rjz-nuc:~# ip link add netns bng1 bng1-111_veth type veth peer name subs-lg111_veth
root@rjz-nuc:~# ip link add netns bng2 bng2-111_veth type veth peer name subs-lg211_veth

3a - Add a macvtap link on top of the 1st veth in the 1st net NS:
#
root@rjz-nuc:~# ip netns exec bng1 ip link add link bng1-111_veth name bng1-111_mcvtp type macvtap mode passthru

3b - This executes correctly and creates a tap device:
#
root@rjz-nuc:~# ip netns exec bng1 ip link show bng1-111_mcvtp
3: bng1-111_mcvtp@bng1-111_veth: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 500
    link/ether ae:c6:bc:d8:d2:e7 brd ff:ff:ff:ff:ff:ff

3c - Note the "3:" at the beginning of above output pointing to "tap3" that the command created:
#
root@rjz-nuc:~# ls -tral /dev/tap*
crw------- 1 root root 241, 1 Jun 7 20:16 /dev/tap3

4a - Try to add another macvtap link but now on top of the 1st veth in the 2nd net NS:
#
root@rjz-nuc:~# ip netns exec bng2 ip link add link bng2-111_veth name bng2-111_mcvtp type macvtap mode passthru
RTNETLINK answers: File exists

4b - As it can be seen, it fails stating "file exists". Checking the output of "dmesg", it can be seen the reason behind this is that instead of choosing as the tap device the next system-wide free one (for instance, say, tap4) it chooses the "tap3" that was already created by the command in step 3a above:
#
[Jun 7 20:17] ------------[ cut here ]------------
[ +0.000032] WARNING: CPU: 3 PID: 1216 at /build/linux-As38az/linux-4.4.0/fs/sysfs/dir.c:31 sysfs_warn_dup+0x62/0x80()
[ +0.000007] sysfs: cannot create duplicate filename '/class/macvtap/tap3'
[ +0.000005] Modules linked in: macvtap macvlan veth drbg ansi_cprng ctr ccm xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp espaf_key xfrm_algo arc4 snd_hda_codec_hdmi 8250_dw snd_soc_skl snd_soc_skl_ipc snd_hda_ext_core snd_soc_sst_ipc snd_soc_sst_dsp snd_hec_realtek snd_hda_codec_generic snd_soc_core snd_compress ac97_bus snd_pcm_dmaengine dw_dmac_core snd_hda_intel snd_hda_codec inte iwlmvm snd_hda_core x86_pkg_temp_thermal intel_powerclamp coretemp snd_hwdep crct10dif_pclmul mac80211 snd_pcm crc32_pclmul ghash_i_intel snd_seq_midi aesni_intel snd_seq_midi_event aes_x86_64 lrw gf128mul snd_rawmidi glue_helper ablk_helper iwlwifi cryptd snd_put_leds snd_seq_device serio_raw cfg80211 snd_timer snd ir_lirc_codec soundcore lirc_dev ir_xmp_decoder ir_mce_kbd_decoder
[ +0.000149] ir_sharp_decoder ir_sanyo_decoder ir_sony_decoder ir_jvc_decoder ir_rc6_decoder btusb hci_uart btrtl btbcm ir_rc5_debtqca ir_nec_decoder idma64 virt_dma mei_me shpchp mei btintel intel_lpss_pci bluetooth rc_rc6_mce ite_cir rc_core intel_lpss_acpi ad mac_hid acpi_als intel_lpss kfifo_buf industrialio kvm_intel kvm irqbypass autofs4 i915_bpo intel_ips i2c_algo_bit drm_kms_helpe0e syscopyarea sysfillrect sysimgblt ptp sdhci_pci fb_sys_fops pps_core drm sdhci ahci libahci video pinctrl_sunrisepoint i2c_hid p_intel hid fjes
[ +0.000124] CPU: 3 PID: 1216 Comm: ip Not tainted 4.4.0-79-generic #100-Ubuntu
[ +0.000007] Hardware name: /NUC6i3SYB, BIOS SYSKLi35.86A.0024.2015.1027.2142 10/27/2015
[ +0.000006] 0000000000000286 00000000ffea66e1 ffff88083a5ef5b8 ffffffff813f94d3
[ +0.000013] ffff88083a5ef600 ffffffff81ce3010 ffff88083a5ef5f0 ffffffff81081322
[ +0.000011] ffff88083a56c000 ffff88083c212900 ffff88083a5fbca8 ffff88083a5fbca8
[ +0.000011] Call Trace:
[ +0.000017] [<ffffffff813f94d3>] dump_stack+0x63/0x90
[ +0.000015] [<ffffffff81081322>] warn_slowpath_common+0x82/0xc0
[ +0.000011] [<ffffffff810813bc>] warn_slowpath_fmt+0x5c/0x80
[ +0.000015] [<ffffffff81290452>] sysfs_warn_dup+0x62/0x80
[ +0.000014] [<ffffffff812907de>] sysfs_do_create_link_sd.isra.2+0x9e/0xb0
[ +0.000014] [<ffffffff81290815>] sysfs_create_link+0x25/0x40
[ +0.000013] [<ffffffff8155f66e>] device_add+0x22e/0x650
[ +0.000012] [<ffffffff8155fc80>] device_create_groups_vargs+0xe0/0xf0
[ +0.000015] [<ffffffff8155fd01>] device_create+0x51/0x70
[ +0.000015] [<ffffffffc090103e>] macvtap_device_event+0xde/0x170 [macvtap]
[ +0.000012] [<ffffffff810a1b1a>] notifier_call_chain+0x4a/0x70
[ +0.000011] [<ffffffff810a1c96>] raw_notifier_call_chain+0x16/0x20
[ +0.000014] [<ffffffff8172f655>] call_netdevice_notifiers_info+0x35/0x60
[ +0.000011] [<ffffffff81739ba2>] register_netdevice+0x342/0x480
[ +0.000015] [<ffffffffc08f6d16>] macvlan_common_newlink+0x166/0x450 [macvlan]
[ +0.000012] [<ffffffffc09001b7>] macvtap_newlink+0x67/0x70 [macvtap]
[ +0.000010] [<ffffffff8174859e>] rtnl_newlink+0x6ee/0x8a0
[ +0.000009] [<ffffffff81748026>] ? rtnl_newlink+0x176/0x8a0
[ +0.000015] [<ffffffff81746ad6>] rtnetlink_rcv_msg+0xe6/0x230
[ +0.000015] [<ffffffff817216b7>] ? __alloc_skb+0x87/0x1f0
[ +0.000008] [<ffffffff817469f0>] ? rtnetlink_rcv+0x30/0x30
[ +0.000014] [<ffffffff81769764>] netlink_rcv_skb+0xa4/0xc0
[ +0.000008] [<ffffffff817469e8>] rtnetlink_rcv+0x28/0x30
[ +0.000011] [<ffffffff8176913a>] netlink_unicast+0x18a/0x240
[ +0.000012] [<ffffffff817694eb>] netlink_sendmsg+0x2fb/0x3a0
[ +0.000014] [<ffffffff8139fc71>] ? aa_sock_msg_perm+0x61/0x150
[ +0.000010] [<ffffffff81718ae8>] sock_sendmsg+0x38/0x50
[ +0.000008] [<ffffffff81719591>] ___sys_sendmsg+0x281/0x290
[ +0.000017] [<ffffffff81202aeb>] ? mem_cgroup_try_charge+0x6b/0x1e0
[ +0.000017] [<ffffffff8119ffd7>] ? lru_cache_add_active_or_unevictable+0x27/0xa0
[ +0.000009] [<ffffffff811c1e2d>] ? handle_mm_fault+0xcad/0x1820
[ +0.000011] [<ffffffff81719ee1>] __sys_sendmsg+0x51/0x90
[ +0.000010] [<ffffffff81719f32>] SyS_sendmsg+0x12/0x20
[ +0.000014] [<ffffffff81840a72>] entry_SYSCALL_64_fastpath+0x16/0x71
[ +0.000125] ---[ end trace fd7a14e4ffe1cf42 ]---

############################
What I expected to happen: the 2nd macvtap should check for the next free tap device in the system instead of in its own net namespace
############################
What happened instead: the 2nd macvtap tries to create a tap device that is already in use in the system and fails
############################
Context info is:
#
root@rjz-nuc:~# lsb_release -rd
Description: Ubuntu 16.04.2 LTS
Release: 16.04
#
root@rjz-nuc:~# uname -a
Linux rjz-nuc 4.4.0-79-generic #100-Ubuntu SMP Wed May 17 19:58:14 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
#
root@rjz-nuc:~# apt-cache policy iproute2
iproute2:
  Installed: 4.3.0-1ubuntu3.16.04.1
  Candidate: 4.3.0-1ubuntu3.16.04.1
  Version table:
 *** 4.3.0-1ubuntu3.16.04.1 500
        500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     4.3.0-1ubuntu3 500
        500 http://us.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
############################

Please don't hesitate to ask me for further information should you need it.

Thanks in advance,
Rolando

[Christian Ehrhardt  (paelzer)]

Hi,
thank you for your report - nicely written as well!

I can confirm the issue on Xenial.
OTOH I can also confirm it working on Artful.

That known I was spawning Yakkety and Zesty guests as well to see where it was fixed.
It is already fixed even back to Yakkety.

[Christian Ehrhardt  (paelzer)]

I upgraded my Xenial test system to linux-virtual-hwe-16.04-edge and with that it is working fine.
So we have a good testcase (thanks Rolando) a crash dmesg on the issue and confirmed that it is solved in a newer kernel.

Comes down to bisecting maybe, but there is a chance the kernel team just knows.

That said I'm rerouting the bug to the kernel Team.

[Joseph Salisbury (jsalisbury)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1696623

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[zappacor (zappacor-l)]

Hi Joseph,

the server where I'm trying to run this is a headless system and have no any VNC or anything else. The apport-collect command mentioned above requires access authorization through a web browser and Lynx (the only thing I could think of) was of no help to successfully authenticate myself.

That said, I think the bug is easy to reproduce, was reproduced by Christian and he also found later versions where this works as expected so setting back to confirmed.

Thanks,
Rolando

[zappacor (zappacor-l)]

Hi guys,

any news for this bug?

TIA,
Rolando.

[Joseph Salisbury (jsalisbury)]

Would it be possible for you to test the latest upstream kernel? Refer to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest v4.12 kernel[0].

If this bug is fixed in the mainline kernel, please add the following tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag: 'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as "Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.12

[zappacor (zappacor-l)]

Still couldn't but I didn't forget, pretty busy and away on business travel lately

________________________________
From: Joseph Salisbury <email address hidden>
Sent: July 11, 2017 10:45:36 AM CDT
To: <email address hidden>
Subject: [Bug 1696623] Re: macvtap creation issue using net namespaces

Would it be possible for you to test the latest upstream kernel? Refer
to https://wiki.ubuntu.com/KernelMainlineBuilds . Please test the latest
v4.12 kernel[0].

If this bug is fixed in the mainline kernel, please add the following
tag 'kernel-fixed-upstream'.

If the mainline kernel does not fix this bug, please add the tag:
'kernel-bug-exists-upstream'.

Once testing of the upstream kernel is complete, please mark this bug as
"Confirmed".

Thanks in advance.

[0] http://kernel.ubuntu.com/~kernel-ppa/mainline/v4.12