2017-09-28 12:14:49 |
Pieter Malan |
bug |
|
|
added bug |
2017-09-29 13:30:07 |
Jan Gutter |
bug |
|
|
added subscriber Jan Gutter |
2017-09-29 13:31:03 |
Launchpad Janitor |
iproute2 (Ubuntu): status |
New |
Confirmed |
|
2017-10-02 16:39:04 |
Joshua Powers |
bug |
|
|
added subscriber Ubuntu Server Team |
2017-10-02 16:39:07 |
Joshua Powers |
iproute2 (Ubuntu): importance |
Undecided |
High |
|
2017-10-10 09:17:16 |
Jan Gutter |
attachment added |
|
[PATCH] Fix "Message truncated" issue with many VF's https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4966551/+files/iproute2-fix-netlink-overflow.patch |
|
2017-10-10 11:24:27 |
Jan Gutter |
bug watch added |
|
https://bugzilla.redhat.com/show_bug.cgi?id=1380803 |
|
2017-10-10 11:24:27 |
Jan Gutter |
bug task added |
|
iproute2 (CentOS) |
|
2017-10-10 12:28:55 |
Ubuntu Foundations Team Bug Bot |
tags |
|
patch |
|
2017-10-10 12:29:05 |
Ubuntu Foundations Team Bug Bot |
bug |
|
|
added subscriber Ubuntu Review Team |
2017-10-12 14:35:51 |
Monique van den Berg |
description |
I've ran into an issue when creating a large number of virtual functions on a SR-IOV capable device.
ip link show reports a message truncated error:
ip link show > /dev/null
Message truncated
Message truncated
Message truncated
A likely cause might be that when called in a system where the number of PCIe Virtual Functions are more than 30 for a given Physical Function, the netlink response is larger than 16K, meaning that a message is truncated.
The issue is seen with Ubuntu14.04 and Ubuntu16.04.
A possible solution for the issue is to increase the size of the receive buffer in libnetlink.c
Additional information:
=====================================================================================================
Ubuntu16 system
stack@cluster04:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
stack@cluster04:~$ uname -r
4.4.0-93-generic
stack@cluster04:~$ apt-cache policy iproute2
iproute2:
Installed: 4.3.0-1ubuntu3.16.04.1
Version table:
*** 4.3.0-1ubuntu3.16.04.1 500
500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
=================================================================================================
Ubuntu14 system:
root@boomslang:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
root@boomslang:~# uname -r
4.4.0-96-generic
root@boomslang:~# apt-cache policy iproute2
iproute2:
Installed: 3.12.0-2ubuntu1
Version table:
*** 3.12.0-2ubuntu1 0
500 http://za.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages |
[Impact]
When querying a Physical Function netdev with a large amount of VF's (more than 30), the resulting return message can overflow the 16K netlink message buffer.
This can be fixed by enabling message peeking on the socket and resizing the buffer on receive, or by simply enlarging the receive buffer.
Since there's an upper limit to the number of VF's per PF, it's relatively sane to just enlarge the receive buffer. Please see the attached patch.
[Test Case]
# Set up 60 VF's on an SR-IOV device
ip link show > /dev/null
Message truncated
Message truncated
Message truncated
[Regression Potential]
1) Applications relying on the broken behaviour will need to be updated, but it would be a really dubious use case.
2) Increasing the rx buffer size increases the memory footprint (but realistically, this is tiny).
3) Extra processing time is now needed to parse the larger buffer, in the case that a call to "ip link" is on the critical time path of an application, (called multiple times in a tight loop, for example), it would affect load.
All of the above are mitigated because this workaround is in upstream iproute2 and has been running on different distributions without repercussions.
[Other Info]
Observed on Ubuntu kernel 4.4.0-93-generic on both 14.04 and 16.04
=====================================================================================================
Ubuntu16 system
stack@cluster04:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
stack@cluster04:~$ uname -r
4.4.0-93-generic
stack@cluster04:~$ apt-cache policy iproute2
iproute2:
Installed: 4.3.0-1ubuntu3.16.04.1
Version table:
*** 4.3.0-1ubuntu3.16.04.1 500
500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
=================================================================================================
Ubuntu14 system:
root@boomslang:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
root@boomslang:~# uname -r
4.4.0-96-generic
root@boomslang:~# apt-cache policy iproute2
iproute2:
Installed: 3.12.0-2ubuntu1
Version table:
*** 3.12.0-2ubuntu1 0
500 http://za.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages |
|
2017-10-12 14:36:29 |
Monique van den Berg |
description |
[Impact]
When querying a Physical Function netdev with a large amount of VF's (more than 30), the resulting return message can overflow the 16K netlink message buffer.
This can be fixed by enabling message peeking on the socket and resizing the buffer on receive, or by simply enlarging the receive buffer.
Since there's an upper limit to the number of VF's per PF, it's relatively sane to just enlarge the receive buffer. Please see the attached patch.
[Test Case]
# Set up 60 VF's on an SR-IOV device
ip link show > /dev/null
Message truncated
Message truncated
Message truncated
[Regression Potential]
1) Applications relying on the broken behaviour will need to be updated, but it would be a really dubious use case.
2) Increasing the rx buffer size increases the memory footprint (but realistically, this is tiny).
3) Extra processing time is now needed to parse the larger buffer, in the case that a call to "ip link" is on the critical time path of an application, (called multiple times in a tight loop, for example), it would affect load.
All of the above are mitigated because this workaround is in upstream iproute2 and has been running on different distributions without repercussions.
[Other Info]
Observed on Ubuntu kernel 4.4.0-93-generic on both 14.04 and 16.04
=====================================================================================================
Ubuntu16 system
stack@cluster04:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
stack@cluster04:~$ uname -r
4.4.0-93-generic
stack@cluster04:~$ apt-cache policy iproute2
iproute2:
Installed: 4.3.0-1ubuntu3.16.04.1
Version table:
*** 4.3.0-1ubuntu3.16.04.1 500
500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
=================================================================================================
Ubuntu14 system:
root@boomslang:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
root@boomslang:~# uname -r
4.4.0-96-generic
root@boomslang:~# apt-cache policy iproute2
iproute2:
Installed: 3.12.0-2ubuntu1
Version table:
*** 3.12.0-2ubuntu1 0
500 http://za.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages |
[Impact]
When querying a Physical Function netdev with a large amount of VF's (more than 30), the resulting return message can overflow the 16K netlink message buffer.
This can be fixed by enabling message peeking on the socket and resizing the buffer on receive, or by simply enlarging the receive buffer.
Since there's an upper limit to the number of VF's per PF, it's relatively sane to just enlarge the receive buffer. Please see the attached patch.
[Test Case]
# Set up 60 VF's on an SR-IOV device
ip link show > /dev/null
Observe the following:
Message truncated
Message truncated
Message truncated
[Regression Potential]
1) Applications relying on the broken behaviour will need to be updated, but it would be a really dubious use case.
2) Increasing the rx buffer size increases the memory footprint (but realistically, this is tiny).
3) Extra processing time is now needed to parse the larger buffer, in the case that a call to "ip link" is on the critical time path of an application, (called multiple times in a tight loop, for example), it would affect load.
All of the above are mitigated because this workaround is in upstream iproute2 and has been running on different distributions without repercussions.
[Other Info]
Observed on Ubuntu kernel 4.4.0-93-generic on both 14.04 and 16.04
=====================================================================================================
Ubuntu16 system
stack@cluster04:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
stack@cluster04:~$ uname -r
4.4.0-93-generic
stack@cluster04:~$ apt-cache policy iproute2
iproute2:
Installed: 4.3.0-1ubuntu3.16.04.1
Version table:
*** 4.3.0-1ubuntu3.16.04.1 500
500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
=================================================================================================
Ubuntu14 system:
root@boomslang:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
root@boomslang:~# uname -r
4.4.0-96-generic
root@boomslang:~# apt-cache policy iproute2
iproute2:
Installed: 3.12.0-2ubuntu1
Version table:
*** 3.12.0-2ubuntu1 0
500 http://za.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages |
|
2017-10-12 14:37:34 |
Monique van den Berg |
description |
[Impact]
When querying a Physical Function netdev with a large amount of VF's (more than 30), the resulting return message can overflow the 16K netlink message buffer.
This can be fixed by enabling message peeking on the socket and resizing the buffer on receive, or by simply enlarging the receive buffer.
Since there's an upper limit to the number of VF's per PF, it's relatively sane to just enlarge the receive buffer. Please see the attached patch.
[Test Case]
# Set up 60 VF's on an SR-IOV device
ip link show > /dev/null
Observe the following:
Message truncated
Message truncated
Message truncated
[Regression Potential]
1) Applications relying on the broken behaviour will need to be updated, but it would be a really dubious use case.
2) Increasing the rx buffer size increases the memory footprint (but realistically, this is tiny).
3) Extra processing time is now needed to parse the larger buffer, in the case that a call to "ip link" is on the critical time path of an application, (called multiple times in a tight loop, for example), it would affect load.
All of the above are mitigated because this workaround is in upstream iproute2 and has been running on different distributions without repercussions.
[Other Info]
Observed on Ubuntu kernel 4.4.0-93-generic on both 14.04 and 16.04
=====================================================================================================
Ubuntu16 system
stack@cluster04:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
stack@cluster04:~$ uname -r
4.4.0-93-generic
stack@cluster04:~$ apt-cache policy iproute2
iproute2:
Installed: 4.3.0-1ubuntu3.16.04.1
Version table:
*** 4.3.0-1ubuntu3.16.04.1 500
500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
=================================================================================================
Ubuntu14 system:
root@boomslang:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
root@boomslang:~# uname -r
4.4.0-96-generic
root@boomslang:~# apt-cache policy iproute2
iproute2:
Installed: 3.12.0-2ubuntu1
Version table:
*** 3.12.0-2ubuntu1 0
500 http://za.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages |
[Impact]
When querying a Physical Function netdev with a large amount of VF's (more than 30), the resulting return message can overflow the 16K netlink message buffer.
This can be fixed by enabling message peeking on the socket and resizing the buffer on receive, or by simply enlarging the receive buffer.
Since there's an upper limit to the number of VF's per PF, it's relatively sane to just enlarge the receive buffer. Please see the attached patch.
[Test Case]
# Set up 60 VF's on an SR-IOV device
ip link show > /dev/null
Observe the following:
Message truncated
Message truncated
Message truncated
[Regression Potential]
1) Applications relying on the broken behaviour will need to be updated, but it would be a really dubious use case.
2) Increasing the rx buffer size increases the memory footprint (but realistically, this is tiny).
3) Extra processing time is now needed to parse the larger buffer, in the case that a call to "ip link" is on the critical time path of an application, (called multiple times in a tight loop, for example), it would affect load.
[Other Info]
Observed on Ubuntu kernel 4.4.0-93-generic on both 14.04 and 16.04
=====================================================================================================
Ubuntu16 system
stack@cluster04:~$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.3 LTS
Release: 16.04
Codename: xenial
stack@cluster04:~$ uname -r
4.4.0-93-generic
stack@cluster04:~$ apt-cache policy iproute2
iproute2:
Installed: 4.3.0-1ubuntu3.16.04.1
Version table:
*** 4.3.0-1ubuntu3.16.04.1 500
500 http://us.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
=================================================================================================
Ubuntu14 system:
root@boomslang:~# lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 14.04.3 LTS
Release: 14.04
Codename: trusty
root@boomslang:~# uname -r
4.4.0-96-generic
root@boomslang:~# apt-cache policy iproute2
iproute2:
Installed: 3.12.0-2ubuntu1
Version table:
*** 3.12.0-2ubuntu1 0
500 http://za.archive.ubuntu.com/ubuntu/ trusty-updates/main amd64 Packages |
|
2017-10-12 14:50:03 |
Monique van den Berg |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
2017-10-12 15:23:28 |
Monique van den Berg |
removed subscriber Ubuntu Sponsors Team |
|
|
|
2017-10-12 19:32:14 |
Nish Aravamudan |
nominated for series |
|
Ubuntu Xenial |
|
2017-10-12 19:32:14 |
Nish Aravamudan |
bug task added |
|
iproute2 (Ubuntu Xenial) |
|
2017-10-12 19:32:14 |
Nish Aravamudan |
nominated for series |
|
Ubuntu Zesty |
|
2017-10-12 19:32:14 |
Nish Aravamudan |
bug task added |
|
iproute2 (Ubuntu Zesty) |
|
2017-10-12 19:32:21 |
Nish Aravamudan |
iproute2 (Ubuntu): status |
Confirmed |
Fix Released |
|
2017-10-12 19:32:25 |
Nish Aravamudan |
iproute2 (Ubuntu Xenial): status |
New |
Confirmed |
|
2017-10-12 19:32:28 |
Nish Aravamudan |
iproute2 (Ubuntu Zesty): status |
New |
Fix Released |
|
2017-10-12 19:32:32 |
Nish Aravamudan |
iproute2 (Ubuntu Xenial): importance |
Undecided |
High |
|
2017-10-12 19:32:34 |
Nish Aravamudan |
iproute2 (Ubuntu): importance |
High |
Undecided |
|
2017-10-12 19:32:55 |
Nish Aravamudan |
nominated for series |
|
Ubuntu Trusty |
|
2017-10-12 19:32:55 |
Nish Aravamudan |
bug task added |
|
iproute2 (Ubuntu Trusty) |
|
2017-10-12 19:53:27 |
Nish Aravamudan |
bug |
|
|
added subscriber Nish Aravamudan |
2017-10-13 08:19:41 |
Monique van den Berg |
bug |
|
|
added subscriber Monique van den Berg |
2017-10-13 12:45:01 |
Monique van den Berg |
attachment added |
|
Changelog https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4969908/+files/changelog |
|
2017-10-13 12:46:26 |
Monique van den Berg |
attachment added |
|
Patch https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4969909/+files/99-increase-receive-buffer-size.patch |
|
2017-10-13 12:47:50 |
Monique van den Berg |
bug |
|
|
added subscriber Ubuntu Sponsors Team |
2017-10-13 12:52:07 |
Monique van den Berg |
attachment removed |
Changelog https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4969908/+files/changelog |
|
|
2017-10-13 12:52:40 |
Monique van den Berg |
attachment added |
|
Changelog https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4969928/+files/changelog |
|
2017-10-13 13:11:19 |
Monique van den Berg |
bug |
|
|
added subscriber Ubuntu Stable Release Updates Team |
2017-10-13 14:31:13 |
Monique van den Berg |
attachment removed |
Changelog https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4969928/+files/changelog |
|
|
2017-10-13 14:32:45 |
Monique van den Berg |
attachment added |
|
Change https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4970030/+files/changelog |
|
2017-10-16 08:05:13 |
Monique van den Berg |
attachment added |
|
iproute2_3.12.0-2ubuntu1.debdiff https://bugs.launchpad.net/ubuntu/xenial/+source/iproute2/+bug/1720126/+attachment/4972550/+files/iproute2_3.12.0-2ubuntu1.debdiff |
|
2017-10-16 08:15:50 |
Monique van den Berg |
attachment removed |
iproute2_3.12.0-2ubuntu1.debdiff https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4972550/+files/iproute2_3.12.0-2ubuntu1.debdiff |
|
|
2017-10-16 08:17:37 |
Monique van den Berg |
attachment added |
|
iproute2_3.12.0-2ubuntu1.debdiff https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4972558/+files/iproute2_3.12.0-2ubuntu1.debdiff |
|
2017-10-16 08:19:07 |
Monique van den Berg |
attachment removed |
iproute2_3.12.0-2ubuntu1.debdiff https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4972558/+files/iproute2_3.12.0-2ubuntu1.debdiff |
|
|
2017-10-16 08:22:17 |
Monique van den Berg |
attachment added |
|
iproute2_3.12.0-2ubuntu1.debdiff https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4972559/+files/iproute2_3.12.0-2ubuntu1.debdiff |
|
2017-10-17 10:12:45 |
Monique van den Berg |
attachment added |
|
Xenial https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4973449/+files/iproute2_4.3.0-1ubuntu3.debdiff |
|
2017-10-19 06:53:26 |
Monique van den Berg |
attachment added |
|
iproute2_3.12.0-2ubuntu1.1.debdiff (Trusty) https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4975629/+files/iproute2_3.12.0-2ubuntu1.debdiff |
|
2017-10-19 06:53:32 |
Monique van den Berg |
attachment added |
|
iproute2_4.3.0-1ubuntu3.16.0.4.2.debdiff (Xenial) https://bugs.launchpad.net/ubuntu/+source/iproute2/+bug/1720126/+attachment/4975630/+files/iproute2_4.3.0-1ubuntu3.debdiff |
|
2017-10-19 23:57:00 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~nacc/ubuntu/+source/iproute2/+git/iproute2/+merge/332530 |
|
2017-10-19 23:57:41 |
Launchpad Janitor |
merge proposal linked |
|
https://code.launchpad.net/~nacc/ubuntu/+source/iproute2/+git/iproute2/+merge/332531 |
|
2017-10-25 12:58:29 |
Chris J Arges |
iproute2 (Ubuntu Xenial): status |
Confirmed |
Fix Committed |
|
2017-10-25 12:58:33 |
Chris J Arges |
bug |
|
|
added subscriber SRU Verification |
2017-10-25 12:58:39 |
Chris J Arges |
tags |
patch |
patch verification-needed verification-needed-xenial |
|
2017-10-25 12:58:48 |
Chris J Arges |
removed subscriber Ubuntu Sponsors Team |
|
|
|
2017-10-25 12:59:26 |
Chris J Arges |
iproute2 (Ubuntu Trusty): status |
New |
Fix Committed |
|
2017-10-25 12:59:37 |
Chris J Arges |
tags |
patch verification-needed verification-needed-xenial |
patch verification-needed verification-needed-trusty verification-needed-xenial |
|
2017-10-26 11:58:05 |
Jan Gutter |
tags |
patch verification-needed verification-needed-trusty verification-needed-xenial |
patch verification-done-trusty verification-needed verification-needed-xenial |
|
2017-10-26 11:58:33 |
Monique van den Berg |
tags |
patch verification-done-trusty verification-needed verification-needed-xenial |
patch verification-done-xenial verification-needed verification-needed-trusty |
|
2017-10-26 11:59:55 |
Monique van den Berg |
tags |
patch verification-done-xenial verification-needed verification-needed-trusty |
patch verification-done-trusty verification-done-xenial verification-needed |
|
2017-10-26 12:04:00 |
Monique van den Berg |
tags |
patch verification-done-trusty verification-done-xenial verification-needed |
patch verification-done-trusty verification-done-xenial verification-needed verification-needed-trusty verification-needed-xenial |
|
2017-10-26 12:04:52 |
Jan Gutter |
tags |
patch verification-done-trusty verification-done-xenial verification-needed verification-needed-trusty verification-needed-xenial |
patch verification-done verification-done-trusty verification-done-xenial verification-needed-trusty verification-needed-xenial |
|
2017-10-26 12:05:29 |
Jan Gutter |
tags |
patch verification-done verification-done-trusty verification-done-xenial verification-needed-trusty verification-needed-xenial |
patch verification-done verification-done-trusty verification-done-xenial |
|
2017-10-27 00:06:48 |
Bug Watch Updater |
iproute2 (CentOS): status |
Unknown |
In Progress |
|
2017-10-27 00:06:48 |
Bug Watch Updater |
iproute2 (CentOS): importance |
Unknown |
High |
|
2017-10-27 00:06:53 |
Bug Watch Updater |
bug watch added |
|
https://bugzilla.redhat.com/show_bug.cgi?id=1086512 |
|
2017-11-02 16:00:39 |
Launchpad Janitor |
iproute2 (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
2017-11-02 16:00:42 |
Brian Murray |
removed subscriber Ubuntu Stable Release Updates Team |
|
|
|
2017-11-09 09:38:37 |
Bug Watch Updater |
iproute2 (CentOS): status |
In Progress |
Fix Committed |
|
2017-12-01 20:29:29 |
Mathew Hodson |
iproute2 (Ubuntu): importance |
Undecided |
High |
|
2017-12-01 20:29:33 |
Mathew Hodson |
iproute2 (Ubuntu Trusty): importance |
Undecided |
High |
|
2017-12-01 20:29:35 |
Mathew Hodson |
iproute2 (Ubuntu Zesty): importance |
Undecided |
High |
|
2017-12-01 20:32:25 |
Mathew Hodson |
bug watch removed |
https://bugzilla.redhat.com/show_bug.cgi?id=1086512 |
|
|
2017-12-06 20:53:05 |
Launchpad Janitor |
iproute2 (Ubuntu Trusty): status |
Fix Committed |
Fix Released |
|