| 2019-08-19 08:34:15 |
Matthew Ruffell |
bug |
|
|
added bug |
| 2019-08-19 08:34:28 |
Matthew Ruffell |
nominated for series |
|
Ubuntu Xenial |
|
| 2019-08-19 08:34:28 |
Matthew Ruffell |
bug task added |
|
linux (Ubuntu Xenial) |
|
| 2019-08-19 08:41:11 |
Matthew Ruffell |
linux (Ubuntu Xenial): status |
New |
In Progress |
|
| 2019-08-19 08:41:15 |
Matthew Ruffell |
linux (Ubuntu Xenial): importance |
Undecided |
Medium |
|
| 2019-08-19 08:41:19 |
Matthew Ruffell |
linux (Ubuntu Xenial): assignee |
|
Matthew Ruffell (mruffell) |
|
| 2019-08-19 08:41:32 |
Matthew Ruffell |
description |
BugLink: https://bugs.launchpad.net/bugs/
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule:
-A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handeling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the warning printed to dmesg and no crashes occur.
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
BugLink: https://bugs.launchpad.net/bugs/1840619
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule:
-A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handeling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the warning printed to dmesg and no crashes occur.
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
|
| 2019-08-19 08:41:54 |
Matthew Ruffell |
tags |
|
sts |
|
| 2019-08-19 09:00:06 |
Ubuntu Kernel Bot |
linux (Ubuntu): status |
New |
Incomplete |
|
| 2019-08-19 09:00:08 |
Ubuntu Kernel Bot |
tags |
sts |
sts xenial |
|
| 2019-08-21 00:43:41 |
Matthew Ruffell |
description |
BugLink: https://bugs.launchpad.net/bugs/1840619
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule:
-A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handeling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the warning printed to dmesg and no crashes occur.
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
BugLink: https://bugs.launchpad.net/bugs/1840619
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule to the mangle table:
-A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80 with incorrect checksums in the ip header.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handeling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the warning printed to dmesg and no crashes occur.
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
|
| 2019-08-21 03:11:59 |
Matthew Ruffell |
description |
BugLink: https://bugs.launchpad.net/bugs/1840619
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule to the mangle table:
-A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80 with incorrect checksums in the ip header.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handeling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the warning printed to dmesg and no crashes occur.
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
BugLink: https://bugs.launchpad.net/bugs/1840619
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule to the mangle table:
-A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80 with incorrect checksums in the ip header.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handeling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the below warning printed to dmesg and no crashes occur.
xt_CHECKSUM: CHECKSUM should be avoided. If really needed, restrict with "-p udp" and only use in OUTPUT
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
|
| 2019-08-21 03:14:44 |
Matthew Ruffell |
description |
BugLink: https://bugs.launchpad.net/bugs/1840619
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule to the mangle table:
-A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80 with incorrect checksums in the ip header.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handeling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the below warning printed to dmesg and no crashes occur.
xt_CHECKSUM: CHECKSUM should be avoided. If really needed, restrict with "-p udp" and only use in OUTPUT
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
BugLink: https://bugs.launchpad.net/bugs/1840619
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule to the mangle table:
-t mangle -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80 with incorrect checksums in the ip header.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handeling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the below warning printed to dmesg and no crashes occur.
xt_CHECKSUM: CHECKSUM should be avoided. If really needed, restrict with "-p udp" and only use in OUTPUT
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
|
| 2019-08-21 03:16:26 |
Matthew Ruffell |
description |
BugLink: https://bugs.launchpad.net/bugs/1840619
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule to the mangle table:
-t mangle -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80 with incorrect checksums in the ip header.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handeling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the below warning printed to dmesg and no crashes occur.
xt_CHECKSUM: CHECKSUM should be avoided. If really needed, restrict with "-p udp" and only use in OUTPUT
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
BugLink: https://bugs.launchpad.net/bugs/1840619
[Impact]
In environments which have CHECKSUM iptables rules set, the following kernel call trace will be created when a GSO skb is processed by the CHECKSUM target:
WARNING: CPU: 34 PID: 806048 at /build/linux-zdslHp/linux-4.4.0/net/core/dev.c:2456 skb_warn_bad_offload+0xcf/0x110()
qr-f78bfdf7-fe: caps=(0x000000000fdb58e9, 0x000000000fdb58e9) len=1955 data_len=479 gso_size=1448 gso_type=1 ip_summed=3
CPU: 34 PID: 806048 Comm: haproxy Tainted: G W OE 4.4.0-138-generic #164-Ubuntu
Call Trace:
dump_stack+0x63/0x90
warn_slowpath_common+0x82/0xc0
warn_slowpath_fmt+0x5c/0x80
? ___ratelimit+0xa2/0xe0
skb_warn_bad_offload+0xcf/0x110
skb_checksum_help+0x185/0x1a0
checksum_tg+0x22/0x29 [xt_CHECKSUM]
ipt_do_table+0x301/0x730 [ip_tables]
? ipt_do_table+0x349/0x730 [ip_tables]
iptable_mangle_hook+0x39/0x107 [iptable_mangle]
nf_iterate+0x68/0x80
nf_hook_slow+0x73/0xd0
ip_output+0xcf/0xe0
? __ip_flush_pending_frames.isra.43+0x90/0x90
ip_local_out+0x3b/0x50
ip_queue_xmit+0x154/0x390
__tcp_transmit_skb+0x52b/0x9b0
tcp_write_xmit+0x1dd/0xf50
__tcp_push_pending_frames+0x31/0xd0
tcp_push+0xec/0x110
tcp_sendmsg+0x749/0xba0
inet_sendmsg+0x6b/0xa0
sock_sendmsg+0x3e/0x50
SYSC_sendto+0x101/0x190
? __sys_sendmsg+0x51/0x90
SyS_sendto+0xe/0x10
entry_SYSCALL_64_fastpath+0x22/0xc1
The CHECKSUM target does not support GSO skbs, and when a GSO skb is passed to skb_checksum_help(), it errors out and skb_warn_bad_offload() is called.
The above call trace was found in a customer environment which has an Openstack deployment, with the following sorts of iptables rules set:
-A neutron-l3-agent-POSTROUTING -o qr-+ -p tcp -m tcp --sport 9697 -j CHECKSUM --checksum-fill
-A neutron-dhcp-age-POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM --checksum-fill
This was causing haproxy running on the node to crash and restart every time a GSO skb was processed by the CHECKSUM target.
I recommend reading the netdev mailing list thread for more details:
https://www.spinics.net/lists/netdev/msg517366.html
[Fix]
This was fixed in 4.19 upstream with the below commit:
commit 10568f6c5761db24249c610c94d6e44d5505a0ba
Author: Florian Westphal <fw@strlen.de>
Date: Wed Aug 22 11:33:27 2018 +0200
Subject: netfilter: xt_checksum: ignore gso skbs
This commit adds a check to see if the current skb is a gso skb, and if it is, skips skb_checksum_help(). It then continues on to check if the packet uses udp, and if it does, exits early. Otherwise it prints a single warning that CHECKSUM should be avoided, and if really needed, only for use with outbound udp.
Note, 10568f6c5761db24249c610c94d6e44d5505a0ba was included in upstream stable version 4.18.13, and was backported to bionic in 4.15.0-58.64 by LP #1836426.
This patch required minor backporting for 4.4, by slightly adjusting the context in the final patch hunk.
[Testcase]
You can reproduce this by adding the following iptables rule to the mangle table:
-t mangle -A POSTROUTING -p tcp -m tcp --sport 80 -j CHECKSUM --checksum-fill
and running traffic over port 80 with incorrect checksums in the ip header.
I built a test kernel, which is available here:
https://launchpad.net/~mruffell/+archive/ubuntu/sf216537-test
For unpatched kernels, this causes the process which was handling the socket to crash, as seen by haproxy crashing on a node in production which hits this issue.
On patched kernels you see the below warning printed to dmesg and no crashes occur.
xt_CHECKSUM: CHECKSUM should be avoided. If really needed, restrict with "-p udp" and only use in OUTPUT
[Regression Potential]
The changes are limited only to users which have CHECKSUM rules enabled in their iptables configs. Openstack commonly configures such rules on deployment, even though they are not necessary, as almost all packets have their checksum calculated by NICs these days, and CHECKSUM is only around to service old dhcp clients which would discard UDP packets with empty checksums.
This commit was selected for upstream -stable 4.18.13, and has made its way into bionic 4.15.0-58.64 by LP #1836426. There have been no reported problems and those kernels would have had sufficient testing with Openstack and its configured iptables rules.
If any users are affected by regression, then they can simply delete any CHECKSUM entries in their iptables configs. |
|
| 2019-08-26 14:59:10 |
Stefan Bader |
linux (Ubuntu): status |
Incomplete |
Fix Released |
|
| 2019-09-03 13:18:55 |
Kleber Sacilotto de Souza |
linux (Ubuntu Xenial): status |
In Progress |
Fix Committed |
|
| 2019-09-12 11:07:06 |
Ubuntu Kernel Bot |
tags |
sts xenial |
sts verification-needed-xenial xenial |
|
| 2019-09-12 23:00:26 |
Matthew Ruffell |
tags |
sts verification-needed-xenial xenial |
sts verification-done-xenial xenial |
|
| 2019-09-22 15:47:34 |
Alvaro Uria |
bug |
|
|
added subscriber Alvaro Uria |
| 2019-09-30 22:09:25 |
Launchpad Janitor |
linux (Ubuntu Xenial): status |
Fix Committed |
Fix Released |
|
| 2019-09-30 22:09:25 |
Launchpad Janitor |
cve linked |
|
2016-10905 |
|
| 2019-09-30 22:09:25 |
Launchpad Janitor |
cve linked |
|
2018-20961 |
|
| 2019-09-30 22:09:25 |
Launchpad Janitor |
cve linked |
|
2018-20976 |
|
| 2019-09-30 22:09:25 |
Launchpad Janitor |
cve linked |
|
2019-0136 |
|
| 2019-09-30 22:09:25 |
Launchpad Janitor |
cve linked |
|
2019-11487 |
|
