regression in 'ip --family bridge neigh' since linux v4.12

[Mauricio Faria de Oliveira]

[Impact]

 * Netlink RTM_GETNEIGH requests for PF_BRIDGE are broken since linux v4.12.

 * Users, tools (e.g., iproute2), and libraries (e.g., go netlink) that use
   such request/family currently receive nothing back in the kernel response.

 * The upstream fix resolves the breakage in the userspace-kernel interface
   by explicitly checking for the old/broken request to ensure it's replied.

[Test Case]

 * The command 'ip --family bridge neigh' returns nothing on broken kernels,
   and matches 'bridge fdb show' on fixed kernels.

 * Before:

    $ ip --family bridge neigh
    $

 * After:

    $ ip --family bridge neigh
    dev ens3 lladdr 33:33:00:00:00:01 PERMANENT
    dev ens3 lladdr 01:00:5e:00:00:01 PERMANENT
    dev ens3 lladdr 33:33:ff:e9:9d:60 PERMANENT

 * Reference:

    $ bridge fdb show
    33:33:00:00:00:01 dev ens3 self permanent
    01:00:5e:00:00:01 dev ens3 self permanent
    33:33:ff:e9:9d:60 dev ens3 self permanent

[Regression Potential]

 * Low, for three reasons:

 * The fix is fairly contained (RTM_GETNEIGH request for PF_BRIDGE family).

 * The checks introduced by the fix are conservative, based on the size
   of the old request (the size of the old/new requests are different),
   and it does nothing different in case the (old) size doesn't match.

 * Given the above, only applications with message length and contents
   specially hand-crafted (and likely not valid nor useful) might fail.
   To the best of my knowledge, this is not the common case out there.

[Other Info]

 * The patch is only applicable to v4.12+ (so not Trusty nor Xenial).

 * The patch is the same for Bionic, Cosmic, and unstable.

 * Upstream commit: bd961c9bc664 ("rtnetlink: fix rtnl_fdb_dump() for ndmsg header")
   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=bd961c9bc66497f0c63f4ba1d02900bb85078366

 * I'll submit the patch shortly to the kernel-team mailing list.

[Mauricio Faria de Oliveira (mfo)]

test on unstable:
-----------------

original:

    $ uname -a
    Linux mfo-sf194734 4.19.0-1-generic #2 SMP Mon Oct 8 14:38:01 -03 2018 x86_64 x86_64 x86_64 GNU/Linux

    $ bridge fdb show
    33:33:00:00:00:01 dev ens3 self permanent
    01:00:5e:00:00:01 dev ens3 self permanent
    33:33:ff:e9:9d:60 dev ens3 self permanent

    $ ip --family bridge neigh
    $

    $ ./neighlist
    $

modified:

    $ uname -a
    Linux mfo-sf194734 4.19.0-1-generic #2+sf1947341 SMP Mon Oct 8 12:57:16 -03 2018 x86_64 x86_64 x86_64 GNU/Linux
    $ bridge fdb show
    33:33:00:00:00:01 dev ens3 self permanent
    01:00:5e:00:00:01 dev ens3 self permanent
    33:33:ff:e9:9d:60 dev ens3 self permanent

    $ ip --family bridge neigh
    dev ens3 lladdr 33:33:00:00:00:01 PERMANENT
    dev ens3 lladdr 01:00:5e:00:00:01 PERMANENT
    dev ens3 lladdr 33:33:ff:e9:9d:60 PERMANENT

    $ ./neighlist
    netlink.Neigh{LinkIndex:2, Family:7, State:128, Type:0, Flags:2, IP:net.IP(nil), HardwareAddr:net.HardwareAddr{0x33, 0x33, 0x0, 0x0, 0x0, 0x1}, LLIPAddr:net.IP(nil), Vlan:0, VNI:0}
    netlink.Neigh{LinkIndex:2, Family:7, State:128, Type:0, Flags:2, IP:net.IP(nil), HardwareAddr:net.HardwareAddr{0x1, 0x0, 0x5e, 0x0, 0x0, 0x1}, LLIPAddr:net.IP(nil), Vlan:0, VNI:0}
    netlink.Neigh{LinkIndex:2, Family:7, State:128, Type:0, Flags:2, IP:net.IP(nil), HardwareAddr:net.HardwareAddr{0x33, 0x33, 0xff, 0xe9, 0x9d, 0x60}, LLIPAddr:net.IP(nil), Vlan:0, VNI:0}

[Mauricio Faria de Oliveira (mfo)]

test on Cosmic
--------------

original:

    $ uname -a
    Linux mfo-sf194734 4.18.0-9-generic #10-Ubuntu SMP Fri Oct 5 16:49:44 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

    $ bridge fdb show
    33:33:00:00:00:01 dev ens3 self permanent
    01:00:5e:00:00:01 dev ens3 self permanent
    33:33:ff:e9:9d:60 dev ens3 self permanent

    $ ip --family bridge neigh
    $

    $ ./neighlist
    $

modified:

    $ uname -a
    Linux mfo-sf194734 4.18.0-9-generic #10+sf1947341 SMP Mon Oct 8 11:46:22 -03 2018 x86_64 x86_64 x86_64 GNU/Linux

    $ bridge fdb show
    33:33:00:00:00:01 dev ens3 self permanent
    01:00:5e:00:00:01 dev ens3 self permanent
    33:33:ff:e9:9d:60 dev ens3 self permanent

    $ ip --family bridge neigh
    dev ens3 lladdr 33:33:00:00:00:01 PERMANENT
    dev ens3 lladdr 01:00:5e:00:00:01 PERMANENT
    dev ens3 lladdr 33:33:ff:e9:9d:60 PERMANENT

    $ ./neighlist
    netlink.Neigh{LinkIndex:2, Family:7, State:128, Type:0, Flags:2, IP:net.IP(nil), HardwareAddr:net.HardwareAddr{0x33, 0x33, 0x0, 0x0, 0x0, 0x1}, LLIPAddr:net.IP(nil), Vlan:0, VNI:0}
    netlink.Neigh{LinkIndex:2, Family:7, State:128, Type:0, Flags:2, IP:net.IP(nil), HardwareAddr:net.HardwareAddr{0x1, 0x0, 0x5e, 0x0, 0x0, 0x1}, LLIPAddr:net.IP(nil), Vlan:0, VNI:0}
    netlink.Neigh{LinkIndex:2, Family:7, State:128, Type:0, Flags:2, IP:net.IP(nil), HardwareAddr:net.HardwareAddr{0x33, 0x33, 0xff, 0xe9, 0x9d, 0x60}, LLIPAddr:net.IP(nil), Vlan:0, VNI:0}

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1796748

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Mauricio Faria de Oliveira (mfo)]

test on Bionic
--------------

original:

    $ uname -a
    Linux mfo-sf194734 4.15.0-37-generic #40-Ubuntu SMP Tue Oct 2 15:36:54 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

    $ bridge fdb show
    33:33:00:00:00:01 dev ens3 self permanent
    01:00:5e:00:00:01 dev ens3 self permanent
    33:33:ff:e9:9d:60 dev ens3 self permanent

    $ ip --family bridge neigh
    $

    $ ./neighlist
    $

modified:

    $ uname -a
    Linux mfo-sf194734 4.15.0-37-generic #40+sf1947341 SMP Mon Oct 8 15:41:38 -03 2018 x86_64 x86_64 x86_64 GNU/Linux

    $ bridge fdb show
    33:33:00:00:00:01 dev ens3 self permanent
    01:00:5e:00:00:01 dev ens3 self permanent
    33:33:ff:e9:9d:60 dev ens3 self permanent

    $ ip --family bridge neigh
    dev ens3 lladdr 33:33:00:00:00:01 PERMANENT
    dev ens3 lladdr 01:00:5e:00:00:01 PERMANENT
    dev ens3 lladdr 33:33:ff:e9:9d:60 PERMANENT

    $ ./neighlist
    netlink.Neigh{LinkIndex:2, Family:7, State:128, Type:0, Flags:2, IP:net.IP(nil), HardwareAddr:net.HardwareAddr{0x33, 0x33, 0x0, 0x0, 0x0, 0x1}, LLIPAddr:net.IP(nil), Vlan:0, VNI:0}
    netlink.Neigh{LinkIndex:2, Family:7, State:128, Type:0, Flags:2, IP:net.IP(nil), HardwareAddr:net.HardwareAddr{0x1, 0x0, 0x5e, 0x0, 0x0, 0x1}, LLIPAddr:net.IP(nil), Vlan:0, VNI:0}
    netlink.Neigh{LinkIndex:2, Family:7, State:128, Type:0, Flags:2, IP:net.IP(nil), HardwareAddr:net.HardwareAddr{0x33, 0x33, 0xff, 0xe9, 0x9d, 0x60}, LLIPAddr:net.IP(nil), Vlan:0, VNI:0}

[Brad Figg (brad-figg)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Brad Figg (brad-figg)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-cosmic' to 'verification-done-cosmic'. If the problem still exists, change the tag 'verification-needed-cosmic' to 'verification-failed-cosmic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Mauricio Faria de Oliveira (mfo)]

Verification successful on Bionic.
---

$ apt-cache madison linux-image-4.15.0-39-generic
linux-image-4.15.0-39-generic | 4.15.0-39.42 | http://archive.ubuntu.com/ubuntu bionic-proposed/main amd64 Packages

$ sudo apt-get install -y linux-image-4.15.0-39-generic=4.15.0-39.42
$ sudo reboot

$ uname -a
Linux mfo-xenial 4.15.0-39-generic #42-Ubuntu SMP Tue Oct 23 15:48:01 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ bridge fdb show
33:33:00:00:00:01 dev ens3 self permanent
01:00:5e:00:00:01 dev ens3 self permanent
33:33:ff:2c:6a:90 dev ens3 self permanent

$ ip --family bridge neigh
dev ens3 lladdr 33:33:00:00:00:01 PERMANENT
dev ens3 lladdr 01:00:5e:00:00:01 PERMANENT
dev ens3 lladdr 33:33:ff:2c:6a:90 PERMANENT

[Mauricio Faria de Oliveira (mfo)]

Verification successful on Cosmic.
---

$ apt-cache madison linux-image-4.18.0-11-generic
linux-image-4.18.0-11-generic | 4.18.0-11.12 | http://archive.ubuntu.com/ubuntu cosmic-proposed/main amd64 Packages

$ sudo apt-get install -y linux-image-4.18.0-11-generic=4.18.0-11.12
$ sudo reboot

$ uname -a
Linux mfo-xenial 4.18.0-11-generic #12-Ubuntu SMP Tue Oct 23 19:22:37 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

$ bridge fdb show
33:33:00:00:00:01 dev ens3 self permanent
01:00:5e:00:00:01 dev ens3 self permanent
33:33:ff:2c:6a:90 dev ens3 self permanent

$ ip --family bridge neigh
dev ens3 lladdr 33:33:00:00:00:01 PERMANENT
dev ens3 lladdr 01:00:5e:00:00:01 PERMANENT
dev ens3 lladdr 33:33:ff:2c:6a:90 PERMANENT

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.15.0-39.42

---------------
linux (4.15.0-39.42) bionic; urgency=medium

  * linux: 4.15.0-39.42 -proposed tracker (LP: #1799411)

  * Linux: insufficient shootdown for paging-structure caches (LP: #1798897)
    - mm: move tlb_table_flush to tlb_flush_mmu_free
    - mm/tlb: Remove tlb_remove_table() non-concurrent condition
    - mm/tlb, x86/mm: Support invalidating TLB caches for RCU_TABLE_FREE
    - [Config] CONFIG_HAVE_RCU_TABLE_INVALIDATE=y

  * Ubuntu18.04: GPU total memory is reduced (LP: #1792102)
    - Revert "powerpc/powernv: Increase memory block size to 1GB on radix"

  * arm64: snapdragon: reduce boot noise (LP: #1797154)
    - [Config] arm64: snapdragon: DRM_MSM=m
    - [Config] arm64: snapdragon: SND*=m
    - [Config] arm64: snapdragon: disable ARM_SDE_INTERFACE
    - [Config] arm64: snapdragon: disable DRM_I2C_ADV7511_CEC
    - [Config] arm64: snapdragon: disable VIDEO_ADV7511, VIDEO_COBALT

  * [Bionic] CPPC bug fixes (LP: #1796949)
    - ACPI / CPPC: Update all pr_(debug/err) messages to log the susbspace id
    - cpufreq: CPPC: Don't set transition_latency
    - ACPI / CPPC: Fix invalid PCC channel status errors

  * regression in 'ip --family bridge neigh' since linux v4.12 (LP: #1796748)
    - rtnetlink: fix rtnl_fdb_dump() for ndmsg header

  * screen displays abnormally on the lenovo M715 with the AMD GPU (Radeon Vega
    8 Mobile, rev ca, 1002:15dd) (LP: #1796786)
    - drm/amd/display: Fix takover from VGA mode
    - drm/amd/display: early return if not in vga mode in disable_vga
    - drm/amd/display: Refine disable VGA

  * arm64: snapdragon: WARNING: CPU: 0 PID: 1 arch/arm64/kernel/setup.c:271
    reserve_memblock_reserved_regions (LP: #1797139)
    - SAUCE: arm64: Fix /proc/iomem for reserved but not memory regions

  * The front MIC can't work on the Lenovo M715 (LP: #1797292)
    - ALSA: hda/realtek - Fix the problem of the front MIC on the Lenovo M715

  * Keyboard backlight sysfs sometimes is missing on Dell laptops (LP: #1797304)
    - platform/x86: dell-smbios: Correct some style warnings
    - platform/x86: dell-smbios: Rename dell-smbios source to dell-smbios-base
    - platform/x86: dell-smbios: Link all dell-smbios-* modules together
    - [Config] CONFIG_DELL_SMBIOS_SMM=y, CONFIG_DELL_SMBIOS_WMI=y

  * rpi3b+: ethernet not working (LP: #1797406)
    - lan78xx: Don't reset the interface on open

  * 87cdf3148b11 was never backported to 4.15 (LP: #1795653)
    - xfrm: Verify MAC header exists before overwriting eth_hdr(skb)->h_proto

  * [Ubuntu18.04][Power9][DD2.2]package installation segfaults inside debian
    chroot env in P9 KVM guest with HTM enabled (kvm) (LP: #1792501)
    - KVM: PPC: Book3S HV: Fix guest r11 corruption with POWER9 TM workarounds

  * Provide mode where all vCPUs on a core must be the same VM (LP: #1792957)
    - KVM: PPC: Book3S HV: Provide mode where all vCPUs on a core must be the same
      VM

  * fscache: bad refcounting in fscache_op_complete leads to OOPS (LP: #1797314)
    - SAUCE: fscache: Fix race in decrementing refcount of op->npages

  * CVE-2018-9363
    - Bluetooth: hidp: buffer overflow in hidp_process_report

  * CVE-20...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.18.0-11.12

---------------
linux (4.18.0-11.12) cosmic; urgency=medium

  * linux: 4.18.0-11.12 -proposed tracker (LP: #1799445)

  * arm64: snapdragon: WARNING: CPU: 0 PID: 1 arch/arm64/kernel/setup.c:271
    reserve_memblock_reserved_regions (LP: #1797139)
    - SAUCE: arm64: Fix /proc/iomem for reserved but not memory regions

  * arm64: snapdragon: WARNING: CPU: 0 PID: 1 at drivers/irqchip/irq-gic.c:1016
    gic_irq_domain_translate (LP: #1797143)
    - SAUCE: arm64: dts: msm8916: camms: fix gic_irq_domain_translate warnings

  * The front MIC can't work on the Lenovo M715 (LP: #1797292)
    - ALSA: hda/realtek - Fix the problem of the front MIC on the Lenovo M715

  * Provide mode where all vCPUs on a core must be the same VM (LP: #1792957)
    - KVM: PPC: Book3S HV: Provide mode where all vCPUs on a core must be the same
      VM

  * fscache: bad refcounting in fscache_op_complete leads to OOPS (LP: #1797314)
    - SAUCE: fscache: Fix race in decrementing refcount of op->npages

  * hns3: autoneg settings get lost on down/up (LP: #1797654)
    - net: hns3: Fix for information of phydev lost problem when down/up

  * not able to unwind the stack from within __kernel_clock_gettime in the Linux
    vDSO (LP: #1797963)
    - powerpc/vdso: Correct call frame information

  * Signal 7 error when running GPFS tracing in cluster (LP: #1792195)
    - powerpc/mm/books3s: Add new pte bit to mark pte temporarily invalid.
    - powerpc/mm/radix: Only need the Nest MMU workaround for R -> RW transition

  * Support Edge Gateway's WIFI LED (LP: #1798330)
    - SAUCE: mwifiex: Switch WiFi LED state according to the device status

  * Support Edge Gateway's Bluetooth LED (LP: #1798332)
    - SAUCE: Bluetooth: Support for LED on Edge Gateways

  * kvm doesn't work on 36 physical bits systems (LP: #1798427)
    - KVM: x86: fix L1TF's MMIO GFN calculation

  * CVE-2018-15471
    - xen-netback: fix input validation in xenvif_set_hash_mapping()

  * regression in 'ip --family bridge neigh' since linux v4.12 (LP: #1796748)
    - rtnetlink: fix rtnl_fdb_dump() for ndmsg header

 -- Stefan Bader <email address hidden> Tue, 23 Oct 2018 18:59:15 +0200