Activity log for bug #1567578

Date Who What changed Old value New value Message
2016-04-07 17:47:57 Lorenzo Cavassa bug added bug
2016-04-07 17:49:31 Launchpad Janitor libnl3 (Ubuntu): status New Confirmed
2016-04-25 10:12:02 Nobuto Murata bug added subscriber Nobuto Murata
2016-05-28 21:50:20 Mathew Hodson tags 14.04.4 libnl nova-compute openstack sriov ubuntu vf nova-compute openstack sriov trusty
2016-05-28 21:50:49 Mathew Hodson libnl3 (Ubuntu): importance Undecided Low
2016-10-19 15:29:32 Joshua Powers bug added subscriber Ubuntu Server Team
2016-10-20 13:13:57 teluka bug added subscriber Mateusz Pawlowski
2016-10-20 18:42:10 Jorge Niedbalski libnl3 (Ubuntu): assignee Jorge Niedbalski (niedbalski)
2016-10-20 18:42:13 Jorge Niedbalski libnl3 (Ubuntu): importance Low Medium
2016-10-20 19:14:28 Jorge Niedbalski nominated for series Ubuntu Precise
2016-10-20 19:14:28 Jorge Niedbalski nominated for series Ubuntu Trusty
2016-10-21 14:09:42 Louis Bouchard bug task added libnl3 (Ubuntu Precise)
2016-10-21 14:09:48 Louis Bouchard bug task added libnl3 (Ubuntu Trusty)
2016-10-21 14:16:04 Jorge Niedbalski libnl3 (Ubuntu): status Confirmed Fix Released
2016-10-21 14:16:10 Jorge Niedbalski libnl3 (Ubuntu): assignee Jorge Niedbalski (niedbalski)
2016-10-21 14:16:13 Jorge Niedbalski libnl3 (Ubuntu Trusty): assignee Jorge Niedbalski (niedbalski)
2016-10-21 14:16:15 Jorge Niedbalski libnl3 (Ubuntu Precise): assignee Jorge Niedbalski (niedbalski)
2016-10-21 17:58:43 Jorge Niedbalski libnl3 (Ubuntu Trusty): status New In Progress
2016-10-21 17:58:45 Jorge Niedbalski libnl3 (Ubuntu Trusty): importance Undecided High
2016-10-24 12:24:34 Jorge Niedbalski bug added subscriber Rafael David Tinoco
2016-10-24 12:27:28 Jorge Niedbalski description Ubuntu 14.04.4 and SRIOV settings. As already documented in https://bugs.launchpad.net/mos/+bug/1501738 there is a bug in the default libnl library release installed on Ubuntu 14.04.4 The library release is the 3.2.21-1 and the bug is impacting on the maximum VFs number that can be enabled (up to 30) even if the PF supports up to 63 VFs in an Openstack SRIOV configuration The workaround is to install a newer library release, the 3.2.24-2: wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-3-200_3.2.24-2_amd64.deb wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-genl-3-200_3.2.24-2_amd64.deb wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-route-3-200_3.2.24-2_amd64.deb dpkg -i libnl-3-200_3.2.24-2_amd64.deb dpkg -i libnl-genl-3-200_3.2.24-2_amd64.deb dpkg -i libnl-route-3-200_3.2.24-2_amd64.deb [Description] Ubuntu 14.04.4 and SRIOV settings. As already documented in https://bugs.launchpad.net/mos/+bug/1501738 there is a bug in the default libnl library release installed on Ubuntu 14.04.4 When trying to enable a guest with more than 30 VFs attached, the following error is returned: error: Failed to start domain guest1 error: internal error: missing IFLA_VF_INFO in netlink response [Impact] The library release is the 3.2.21-1 the bug is impacting on the maximum VFs number that can be enabled (up to 30) even if the PF supports up to 63 VFs in an Openstack SRIOV configuration [Test Case] The sequence to reproduce this bug is: 1) Edit /etc/default/grub GRUB_CMDLINE_LINUX="intel_iommu=on ixgbe.max_vfs=63" 2) $ sudo update-grub ### Reboot the machine. 3) Check that the virtual functions are available: $ sudo lspci|grep -i eth | grep -i virtual | wc -l 126 4) Create a KVM guest $ sudo uvt-kvm create guest1 release=trusty 5) List the VF devices : $ sudo lspci|grep -i eth | grep -i virtual | awk '{print $1}' | sed 's/\:/\_/g' | sed 's/\./\_/g' > devices.txt 6) Get the libvirt node device: $ sudo for device in $(cat ./devices.txt); do virsh nodedev-list | grep $device; done > pci_devices.txt 7) Generate the XML config for each device: $ sudo mkdir devices && for d in $(cat pci_devices.txt); do virsh nodedev-dumpxml $d > devices/$d.xml; done 8) Save and Run the following script (http://pastebin.ubuntu.com/23374186/) $ sudo python generate-interfaces.py |grep address | wc -l 9) Finally attach the devices to the guest. $ sudo for i in $(seq 0 63); do virsh attach-device guest1 ./interfaces/$i.xml --config; done Device attached successfully [...] Device attached successfully Device attached successfully 10) Then destroy/start the guest again, at this point the error is reproduced. $ sudo virsh destroy guest1 Domain guest1 destroyed $ sudo virsh start guest1 error: Failed to start domain guest1 error: internal error: missing IFLA_VF_INFO in netlink response * detailed instructions how to reproduce the bug * these should allow someone who is not familiar with the affected package to reproduce the bug and verify that the updated package fixes the problem. [Regression Potential] ** None identified. [Other Info] - Redhat Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1040626 [Workaround] The workaround is to install a newer library release, the 3.2.24-2: wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-3-200_3.2.24-2_amd64.deb wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-genl-3-200_3.2.24-2_amd64.deb wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-route-3-200_3.2.24-2_amd64.deb dpkg -i libnl-3-200_3.2.24-2_amd64.deb dpkg -i libnl-genl-3-200_3.2.24-2_amd64.deb dpkg -i libnl-route-3-200_3.2.24-2_amd64.deb
2016-10-24 12:27:36 Jorge Niedbalski tags nova-compute openstack sriov trusty nova-compute openstack sriov sts trusty
2016-10-24 13:55:22 Jorge Niedbalski attachment added fix-1567578-trusty.debdiff https://bugs.launchpad.net/ubuntu/+source/libnl3/+bug/1567578/+attachment/4766476/+files/fix-1567578-trusty.debdiff
2016-10-24 14:40:16 Jorge Niedbalski bug added subscriber Ubuntu Stable Release Updates Team
2016-10-24 16:18:37 Louis Bouchard tags nova-compute openstack sriov sts trusty nova-compute openstack sriov sts sts-sru trusty
2016-10-24 16:48:23 Brian Murray libnl3 (Ubuntu Trusty): status In Progress Fix Committed
2016-10-24 16:48:27 Brian Murray bug added subscriber SRU Verification
2016-10-24 16:48:36 Brian Murray tags nova-compute openstack sriov sts sts-sru trusty nova-compute openstack sriov sts sts-sru trusty verification-needed
2016-10-24 22:17:20 Mathew Hodson description [Description] Ubuntu 14.04.4 and SRIOV settings. As already documented in https://bugs.launchpad.net/mos/+bug/1501738 there is a bug in the default libnl library release installed on Ubuntu 14.04.4 When trying to enable a guest with more than 30 VFs attached, the following error is returned: error: Failed to start domain guest1 error: internal error: missing IFLA_VF_INFO in netlink response [Impact] The library release is the 3.2.21-1 the bug is impacting on the maximum VFs number that can be enabled (up to 30) even if the PF supports up to 63 VFs in an Openstack SRIOV configuration [Test Case] The sequence to reproduce this bug is: 1) Edit /etc/default/grub GRUB_CMDLINE_LINUX="intel_iommu=on ixgbe.max_vfs=63" 2) $ sudo update-grub ### Reboot the machine. 3) Check that the virtual functions are available: $ sudo lspci|grep -i eth | grep -i virtual | wc -l 126 4) Create a KVM guest $ sudo uvt-kvm create guest1 release=trusty 5) List the VF devices : $ sudo lspci|grep -i eth | grep -i virtual | awk '{print $1}' | sed 's/\:/\_/g' | sed 's/\./\_/g' > devices.txt 6) Get the libvirt node device: $ sudo for device in $(cat ./devices.txt); do virsh nodedev-list | grep $device; done > pci_devices.txt 7) Generate the XML config for each device: $ sudo mkdir devices && for d in $(cat pci_devices.txt); do virsh nodedev-dumpxml $d > devices/$d.xml; done 8) Save and Run the following script (http://pastebin.ubuntu.com/23374186/) $ sudo python generate-interfaces.py |grep address | wc -l 9) Finally attach the devices to the guest. $ sudo for i in $(seq 0 63); do virsh attach-device guest1 ./interfaces/$i.xml --config; done Device attached successfully [...] Device attached successfully Device attached successfully 10) Then destroy/start the guest again, at this point the error is reproduced. $ sudo virsh destroy guest1 Domain guest1 destroyed $ sudo virsh start guest1 error: Failed to start domain guest1 error: internal error: missing IFLA_VF_INFO in netlink response * detailed instructions how to reproduce the bug * these should allow someone who is not familiar with the affected package to reproduce the bug and verify that the updated package fixes the problem. [Regression Potential] ** None identified. [Other Info] - Redhat Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1040626 [Workaround] The workaround is to install a newer library release, the 3.2.24-2: wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-3-200_3.2.24-2_amd64.deb wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-genl-3-200_3.2.24-2_amd64.deb wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-route-3-200_3.2.24-2_amd64.deb dpkg -i libnl-3-200_3.2.24-2_amd64.deb dpkg -i libnl-genl-3-200_3.2.24-2_amd64.deb dpkg -i libnl-route-3-200_3.2.24-2_amd64.deb [Impact] libnl can only enable up to 30 VFs even if the PF supports up to 63 VFs in an Openstack SRIOV configuration. As already documented in https://bugs.launchpad.net/mos/+bug/1501738 there is a bug in the default libnl library release installed on Ubuntu 14.04.4. When trying to enable a guest with more than 30 VFs attached, the following error is returned: error: Failed to start domain guest1 error: internal error: missing IFLA_VF_INFO in netlink response [Test Case] 1) Edit /etc/default/grub. GRUB_CMDLINE_LINUX="intel_iommu=on ixgbe.max_vfs=63" 2) Update grub and reboot the machine. $ sudo update-grub 3) Check that the virtual functions are available. $ sudo lspci|grep -i eth | grep -i virtual | wc -l 126 4) Create a KVM guest. $ sudo uvt-kvm create guest1 release=trusty 5) List the VF devices. $ sudo lspci|grep -i eth | grep -i virtual | awk '{print $1}' | sed 's/\:/\_/g' | sed 's/\./\_/g' > devices.txt 6) Get the libvirt node device. $ sudo for device in $(cat ./devices.txt); do virsh nodedev-list | grep $device; done > pci_devices.txt 7) Generate the XML config for each device. $ sudo mkdir devices && for d in $(cat pci_devices.txt); do virsh nodedev-dumpxml $d > devices/$d.xml; done 8) Save and Run the following script. (http://pastebin.ubuntu.com/23374186/) $ sudo python generate-interfaces.py |grep address | wc -l 9) Finally attach the devices to the guest. $ sudo for i in $(seq 0 63); do virsh attach-device guest1 ./interfaces/$i.xml --config; done Device attached successfully [...] Device attached successfully Device attached successfully 10) Then destroy/start the guest again, at this point the error is reproduced. $ sudo virsh destroy guest1 Domain guest1 destroyed $ sudo virsh start guest1 error: Failed to start domain guest1 error: internal error: missing IFLA_VF_INFO in netlink response [Regression Potential]  * None identified. [Other Info] * Redhat Bug: https://bugzilla.redhat.com/show_bug.cgi?id=1040626 * A workaround is to install a newer library release. $ wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-3-200_3.2.24-2_amd64.deb $ wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-genl-3-200_3.2.24-2_amd64.deb $ wget https://launchpad.net/ubuntu/+archive/primary/+files/libnl-route-3-200_3.2.24-2_amd64.deb $ dpkg -i libnl-3-200_3.2.24-2_amd64.deb $ dpkg -i libnl-genl-3-200_3.2.24-2_amd64.deb $ dpkg -i libnl-route-3-200_3.2.24-2_amd64.deb
2016-10-24 22:19:37 Mathew Hodson libnl3 (Ubuntu Precise): importance Undecided Medium
2016-10-25 02:55:13 Jorge Niedbalski tags nova-compute openstack sriov sts sts-sru trusty verification-needed nova-compute openstack sriov sts sts-sru trusty verification-done
2016-11-01 05:27:15 Martin Pitt removed subscriber Ubuntu Stable Release Updates Team
2016-11-01 05:28:13 Launchpad Janitor libnl3 (Ubuntu Trusty): status Fix Committed Fix Released
2016-11-02 14:35:09 Jorge Niedbalski bug task deleted libnl3 (Ubuntu Precise)
2016-11-09 12:20:30 Louis Bouchard tags nova-compute openstack sriov sts sts-sru trusty verification-done nova-compute openstack sriov sts trusty verification-done