Activity log for bug #1811094

Date Who What changed Old value New value Message
2019-01-09 13:52:04 Mauricio Faria de Oliveira bug added bug
2019-01-09 13:52:10 Mauricio Faria de Oliveira linux (Ubuntu): assignee Mauricio Faria de Oliveira (mfo)
2019-01-09 13:52:14 Mauricio Faria de Oliveira linux (Ubuntu): status New Confirmed
2019-01-10 03:22:48 Mauricio Faria de Oliveira description The following iptables connlimit rule can be breached with a multithreaded client and network device driver, due to a race in the conncount/connlimit code: # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \ -m connlimit --connlimit-above 2000 --connlimit-mask 0 \ -j DROP NOTE: Patches will be sent to the kernel-team mailing list and more details/testing will be provided later today. [Impact] * The iptables connection count/limit rules can be breached with multithreaded network driver/server/client (common) due to a race in the conncount/connlimit code. * For example: # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \ -m connlimit --connlimit-above 2000 --connlimit-mask 0 \ -j DROP * The fix is a backport from an upstream commit that resolves the problem (plus dependencies for a cleaner backport) that address the race condition: commit b36e4523d4d5 ("netfilter: nf_conncount: fix garbage collection confirm race"). [Test Case] * Server-side: (relevant kernel side) (limit TCP port 7777 to only 2000 connections) # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \ -m connlimit --connlimit-above 2000 --connlimit-mask 0 \ -j DROP # ulimit -SHn 65000 # increase number of open files # ruby server.rb # multi-threaded server * Client-side: # ulimit -SHn 65000 # ruby client.rb <server ip> <port> <target # connections> <# threads> <test output> * Results with Original kernel: (client achieves target of 6000 connections > limit of 2000 connections) # ruby client.rb 10.230.56.100 7777 6000 3 1 2 3 <...> 6000 Target reached. Thread finishing 6001 Target reached. Thread finishing 6002 Target reached. Thread finishing Threads done. 6002 connections press enter to exit * Results with Modified kernel: (client is limited to 2000 connections, and times out afterward) # ruby client.rb 10.230.56.100 7777 6000 3 1 2 3 <...> 2000 <... blocks for a few minutes ...> failed to create connection: Connection timed out - connect(2) for "10.230.56.100" port 7777 failed to create connection: Connection timed out - connect(2) for "10.230.56.100" port 7777 failed to create connection: Connection timed out - connect(2) for "10.230.56.100" port 7777 Threads done. 2000 connections press enter to exit * Test cases possibly available upon request, depending on original author's permission. [Regression Potential] * The patchset has been reviewed by a netfilter maintainer [1] in stable mailing list, and was considered OK for 4.14, and that's essentially the same backport for 4.15 and 4.4. * The changes are limited to netfilter conncount/connlimit (names change between older/newer kernel versions). [Other Info] * The backport for 4.14 [2] is applied as of 4.14.92. [1] https://www.spinics.net/lists/stable/msg276883.html [2] https://www.spinics.net/lists/stable/msg276910.html
2019-01-10 03:23:37 Mauricio Faria de Oliveira description [Impact] * The iptables connection count/limit rules can be breached with multithreaded network driver/server/client (common) due to a race in the conncount/connlimit code. * For example: # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \ -m connlimit --connlimit-above 2000 --connlimit-mask 0 \ -j DROP * The fix is a backport from an upstream commit that resolves the problem (plus dependencies for a cleaner backport) that address the race condition: commit b36e4523d4d5 ("netfilter: nf_conncount: fix garbage collection confirm race"). [Test Case] * Server-side: (relevant kernel side) (limit TCP port 7777 to only 2000 connections) # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \ -m connlimit --connlimit-above 2000 --connlimit-mask 0 \ -j DROP # ulimit -SHn 65000 # increase number of open files # ruby server.rb # multi-threaded server * Client-side: # ulimit -SHn 65000 # ruby client.rb <server ip> <port> <target # connections> <# threads> <test output> * Results with Original kernel: (client achieves target of 6000 connections > limit of 2000 connections) # ruby client.rb 10.230.56.100 7777 6000 3 1 2 3 <...> 6000 Target reached. Thread finishing 6001 Target reached. Thread finishing 6002 Target reached. Thread finishing Threads done. 6002 connections press enter to exit * Results with Modified kernel: (client is limited to 2000 connections, and times out afterward) # ruby client.rb 10.230.56.100 7777 6000 3 1 2 3 <...> 2000 <... blocks for a few minutes ...> failed to create connection: Connection timed out - connect(2) for "10.230.56.100" port 7777 failed to create connection: Connection timed out - connect(2) for "10.230.56.100" port 7777 failed to create connection: Connection timed out - connect(2) for "10.230.56.100" port 7777 Threads done. 2000 connections press enter to exit * Test cases possibly available upon request, depending on original author's permission. [Regression Potential] * The patchset has been reviewed by a netfilter maintainer [1] in stable mailing list, and was considered OK for 4.14, and that's essentially the same backport for 4.15 and 4.4. * The changes are limited to netfilter conncount/connlimit (names change between older/newer kernel versions). [Other Info] * The backport for 4.14 [2] is applied as of 4.14.92. [1] https://www.spinics.net/lists/stable/msg276883.html [2] https://www.spinics.net/lists/stable/msg276910.html [Impact]  * The iptables connection count/limit rules can be breached    with multithreaded network driver/server/client (common)    due to a race in the conncount/connlimit code.  * For example:    # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \      -m connlimit --connlimit-above 2000 --connlimit-mask 0 \      -j DROP  * The fix is a backport from an upstream commit that resolves    the problem (plus dependencies for a cleaner backport) that    address the race condition:    commit b36e4523d4d5 ("netfilter: nf_conncount: fix garbage    collection confirm race"). [Test Case]  * Server-side: (relevant kernel side)    (limit TCP port 7777 to only 2000 connections)    # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \      -m connlimit --connlimit-above 2000 --connlimit-mask 0 \      -j DROP    # ulimit -SHn 65000 # increase number of open files    # ruby server.rb # multi-threaded server  * Client-side:    # ulimit -SHn 65000    # ruby client.rb <server ip> <port> <target # connections> <# threads>    <test output>  * Results with Original kernel:    (client achieves target of 6000 connections > limit of 2000 connections)    # ruby client.rb 10.230.56.100 7777 6000 3    1    2    3    <...>    6000    Target reached. Thread finishing    6001    Target reached. Thread finishing    6002    Target reached. Thread finishing    Threads done. 6002 connections    press enter to exit  * Results with Modified kernel:    (client is limited to 2000 connections, and times out afterward)    # ruby client.rb 10.230.56.100 7777 6000 3    1    2    3    <...>    2000    <... blocks for a few minutes ...>    failed to create connection: Connection timed out - connect(2) for "10.230.56.100" port 7777    failed to create connection: Connection timed out - connect(2) for "10.230.56.100" port 7777    failed to create connection: Connection timed out - connect(2) for "10.230.56.100" port 7777    Threads done. 2000 connections    press enter to exit  * Test cases possibly available upon request,    depending on original author's permission. [Regression Potential]  * The patchset has been reviewed by a netfilter maintainer [1] in    stable mailing list, and was considered OK for 4.14, and that's    essentially the same backport for 4.15 and 4.4.  * The changes are limited to netfilter connlimit/conncount (names    change between older/newer kernel versions). [Other Info]  * The backport for 4.14 [2] is applied as of 4.14.92. [1] https://www.spinics.net/lists/stable/msg276883.html [2] https://www.spinics.net/lists/stable/msg276910.html
2019-01-10 08:52:08 Stefan Bader nominated for series Ubuntu Cosmic
2019-01-10 08:52:08 Stefan Bader bug task added linux (Ubuntu Cosmic)
2019-01-10 08:52:16 Stefan Bader linux (Ubuntu): importance Undecided Medium
2019-01-10 08:52:20 Stefan Bader linux (Ubuntu Cosmic): importance Undecided Medium
2019-01-10 08:57:18 Stefan Bader nominated for series Ubuntu Bionic
2019-01-10 08:57:18 Stefan Bader bug task added linux (Ubuntu Bionic)
2019-01-10 08:57:18 Stefan Bader nominated for series Ubuntu Xenial
2019-01-10 08:57:18 Stefan Bader bug task added linux (Ubuntu Xenial)
2019-01-10 08:57:28 Stefan Bader linux (Ubuntu Bionic): importance Undecided Medium
2019-01-10 08:57:31 Stefan Bader linux (Ubuntu Xenial): importance Undecided Medium
2019-01-10 15:26:11 Kleber Sacilotto de Souza linux (Ubuntu Bionic): status New Fix Committed
2019-01-10 15:37:38 Kleber Sacilotto de Souza linux (Ubuntu Cosmic): status New Fix Committed
2019-01-10 16:24:32 Kleber Sacilotto de Souza linux (Ubuntu Xenial): status New Fix Committed
2019-01-11 17:12:23 Seth Forshee linux (Ubuntu): status Confirmed Fix Committed
2019-01-15 10:33:10 Brad Figg tags verification-needed-cosmic
2019-01-15 10:37:17 Brad Figg tags verification-needed-cosmic verification-needed-bionic verification-needed-cosmic
2019-01-15 18:45:27 Mauricio Faria de Oliveira tags verification-needed-bionic verification-needed-cosmic verification-done-bionic verification-done-cosmic
2019-01-17 14:21:49 Brad Figg tags verification-done-bionic verification-done-cosmic verification-done-bionic verification-done-cosmic verification-needed-xenial
2019-01-17 14:56:08 Mauricio Faria de Oliveira tags verification-done-bionic verification-done-cosmic verification-needed-xenial verification-done-bionic verification-done-cosmic verification-done-xenial
2019-01-28 17:12:01 Launchpad Janitor linux (Ubuntu Bionic): status Fix Committed Fix Released
2019-01-28 17:12:01 Launchpad Janitor cve linked 2018-14625
2019-01-28 17:12:01 Launchpad Janitor cve linked 2018-16882
2019-01-28 17:12:01 Launchpad Janitor cve linked 2018-17972
2019-01-28 17:12:01 Launchpad Janitor cve linked 2018-18281
2019-01-28 17:12:01 Launchpad Janitor cve linked 2018-19407
2019-01-29 07:05:48 Stefan Bader nominated for series Ubuntu Trusty
2019-01-29 07:05:48 Stefan Bader bug task added linux (Ubuntu Trusty)
2019-01-29 07:05:57 Stefan Bader linux (Ubuntu Trusty): importance Undecided Medium
2019-01-29 07:06:02 Stefan Bader linux (Ubuntu Trusty): status New Fix Committed
2019-02-04 08:47:55 Launchpad Janitor linux (Ubuntu Xenial): status Fix Committed Fix Released
2019-02-04 08:47:55 Launchpad Janitor cve linked 2000-1134
2019-02-04 08:47:55 Launchpad Janitor cve linked 2007-3852
2019-02-04 08:47:55 Launchpad Janitor cve linked 2008-0525
2019-02-04 08:47:55 Launchpad Janitor cve linked 2009-0416
2019-02-04 08:47:55 Launchpad Janitor cve linked 2011-4834
2019-02-04 08:47:55 Launchpad Janitor cve linked 2015-1838
2019-02-04 08:47:55 Launchpad Janitor cve linked 2015-7442
2019-02-04 08:47:55 Launchpad Janitor cve linked 2016-7489
2019-02-04 08:47:55 Launchpad Janitor cve linked 2017-5715
2019-02-04 08:48:45 Launchpad Janitor linux (Ubuntu Cosmic): status Fix Committed Fix Released
2019-02-04 14:46:37 Launchpad Janitor linux (Ubuntu): status Fix Committed Fix Released
2019-02-11 20:03:36 Brad Figg tags verification-done-bionic verification-done-cosmic verification-done-xenial verification-done-bionic verification-done-cosmic verification-done-xenial verification-needed-trusty
2019-02-11 21:07:44 Mauricio Faria de Oliveira tags verification-done-bionic verification-done-cosmic verification-done-xenial verification-needed-trusty verification-done-bionic verification-done-cosmic verification-done-trusty verification-done-xenial
2019-03-12 18:24:27 Launchpad Janitor linux (Ubuntu Trusty): status Fix Committed Fix Released
2019-03-12 18:24:27 Launchpad Janitor cve linked 2019-6133