procps outdated network options, old syncookies, new ecn update please.

[Simon Iremonger]

The ubuntu version of procps carries it's own /etc/sysctl.d/10-network-security.conf file explicitly that appears not to be part of debian procps version.

Firstly, the section about "# Turn on SYN-flood protections." (came from LP #57091 ) is now entirely outdated, upstream kernel has long since turned on syncookies by default, so setting this flag explicitly in 10-network-security.conf is entirely redundant likely since before ubuntu-14.04 .
I would like the ubuntu-maintainer to remove that section entirely in cosmic onwards.

[I am going to report debian the similarly outdated syncookies comments in sysctl.conf itself].

Secondly, I propose a new 10-network-tuning.conf with:-
==============================================================================
# Allow ECN for outgoing connections. Starting with 4.2, there is an adaptive
# fallback [enabled by default tcp_ecn_fallback option] preventing connection
# loss even with ecn enabled, also ecn-intolerance is increasingly very rare.
net.ipv4.tcp_ecn=1
==============================================================================

I know there is a (small) chance of issues/regressions with ECN enabled by default on outgoing but I'm quite sure the issue is very rare, like others notice [ref: 1 and 2 below]. Apple's selective enablements etc. show this works just as much as my own use for years and many similar reports.

ECN actually being used for outgoing connections really helps with latency-reduction with modern routers (both core and edge) using queuing disciplines fq_codel or otherwise, able to mark rather than drop packets on ECN-enabled flows [helps latency and realtime applications]. Now we are just past LTS release is in my view the 'right time' to finally enable ECN [and obviously easy to revert!]. If this is disputed, in ANY case I strongly suggest at the very least a commented-out ECN section should be included, but 'defaults matter'!.

I was going to suggest a non-default section about net.core.default_qdisc [ LP #1436945 ] but this appears to have been fixed upstream similarly.

[1] https://www.ietf.org/proceedings/98/slides/slides-98-maprg-tcp-ecn-experience-with-enabling-ecn-on-the-internet-padma-bhooma-00.pdf
[2] http://seclists.org/nanog/2015/Jun/675

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Toke Høiland-Jørgensen (tohojo)]

I've been running with tcp_ecn=1 for years with no issue. Since ECN marking is making its way into more and more places, enabling it by default makes sense. In particular, it would go well with the recent addition of FQ-CoDel as the default queue management algorithm :)

[Simon Iremonger (ubuntu-iremonger)]

(fwiw, fq_codel queuing is now being turned-on in bionic (at least) by systemd, confusingly!).
https://github.com/systemd/systemd/commit/e6c253e363dee77ef7e5c5f44c4ca55cded3fd47
Possibly, turning on ECN might more likely happen there first, but I would like the procps updated for those using upstart or otherwise). This seems to be the last piece of bufferbloat puzzle (see LP bug #1436945 ).

[Balint Reczey (rbalint)]

I'm dropping the redundant setting of syncookies from the delta but IMO ECN should be enabled upstream or in Debian instead of adding a delta for it in Ubuntu.

Please try to get ECN enabled at upstream or at Debian, we have plenty of time before the next (LTS) release in case we have to fall back to adding a delta if upstream and Debian don't want to enable ECN but it would still be beneficial for Ubuntu to enable it.

[Simon Iremonger (ubuntu-iremonger)]

It would appear that the path-of-least-resistance at present, is systemd, poettering which is what is (for systemd-booters) where fq_codel is getting turned-on in ubuntu.

This raises a wider-issue about bringing systemd-provided sysctl-defaults into procps more widely [systemd has introduced many of these in its' own repository, but version in ubuntu-bionic has few, see /usr/lib/sysctl.d/ on a bionic system...

ALSO I have discovered there are facts to be checked about "BBR" as default TCP congestion-control, which will also be desirable, but MAY still have immature/issues when ECN is used on a TCP connection as well [one suggestion BBR doesn't react to ECN notifications]... I'm trying to get 'evidence' and 'facts' in that regard, which seem to be sparse and hard-to-find ...

I'm going to (try) to get more facts before suggesting patches with reasons/evidence a few places.
Agree entirely debian and upstream worth trying to ask, etc.
HOWEVER its' often very useful to have had a change introduced in a 'non-lts' or 'testing' distibution like ubuntu-non-LTS releases so you can say how it works and had some testing/exposure somewhere first... It may be I come back to you and suggest a delta in ubuntu "for now" for good reason. We will see.

Thankyou for helpful and promising-sounding response!.

[Simon Iremonger (ubuntu-iremonger)]

Right -- systemd have just-now agreed to set the change in their upstream systemd sysctl files :-
https://github.com/systemd/systemd/commit/6f130e85c76cfc2c58ba31f90d2ac3800866c1dd

I notice, however, that ubuntu's systemd pkg 'strips most those settings out', in 18.04 currently only carrying the 18.04 fq_codel switch-on in their sysctl.d....

I think, given what has been said, I would like to propose that I :-

* Make a suggested text for a 10-network-bufferbloat.conf here in procps in 18.10 (hopefully-onwards, including suitable references/comments about BBR (which should be there but commented/not-enabled yet unless we are sure its' been fixed to respond to ECN notifications.). This text shall explain clearly these are deliberately being tested into 18.10 and where to report bugs.

* Look at what ubuntu's systemd package towards 18.10 is importing in sysctl.d -- and likely suggest ubuntu 'taken out' entirely so procps is the 'one' location for these settings (i.e. no duplicate setting of qdisc=fq_codel in 2 different places). Some will want to boot ubuntu with OpenRC or upstart for whatever reasons and consistent-behaviour would be helpful...

* Ask in the BBR community about tcp_congestion_control goings-on there, when they are ready for ECN-compatible BBR wider-deployment.

* Then, as/when seems appropriate, suggest changes into upstream-debian and upstream-kernel on the defaults.

@rbalint -- what do you think on this plan for the interim?

[Launchpad Janitor (janitor)]

This bug was fixed in the package procps - 2:3.3.15-2ubuntu1

---------------
procps (2:3.3.15-2ubuntu1) cosmic; urgency=medium

  * Merge from Debian unstable. Remaining changes:
    - debian/sysctl.d (Ubuntu-specific):
      + 10-console-messages.conf: stop low-level kernel messages on console.
      + 10-kernel-hardening.conf: add the kptr_restrict setting
      + 10-keyboard.conf.powerpc: mouse button emulation on PowerPC.
      + 10-ipv6-privacy.conf: add a file to sysctl.d to apply the defaults
        for IPv6 privacy extensions for interfaces. (LP: #176125, #841353)
      + 10-link-restrictions.conf: even though the Ubuntu
        kernel is built with these defaults in place, we want to make sure
        that people running stock kernels don't miss out.
      + 10-magic-sysrq.conf: Disable most magic sysrq by default, allowing
        critical sync, remount, reboot functions. (LP: #194676, LP: #1025467)
      + 10-network-security.conf: enable rp_filter.
      + 10-ptrace.conf: describe new PTRACE setting.
      + 10-zeropage.conf: safe mmap_min_addr value for graceful fall-back.
        for armhf, and arm64.
      + 10-qemu.conf.s390x for qemu.
      + README: describe how this directory is supposed to work.
    - debian/rules: Fix cross build
    - ignore_eaccess.patch: If we get eaccess when opening a sysctl file for
      writing, don't error out. Otherwise package upgrades can fail,
      especially in containers.
    - ignore_erofs.patch: Same as ignore_eaccess but for the case where
      part of /proc is read/only.

procps (2:3.3.15-2) unstable; urgency=medium

  * Fix link in libprocps-dev Closes: 900239
  * Fix typo in license Closes: #899346

 -- Balint Reczey <email address hidden> Tue, 05 Jun 2018 11:20:00 -0700