kernel 4.15 breaks xl2tpd

[Douglas Kosovic]

[Impact]

Kernel 4.15 breaks xl2tpd, please see following upstream issue for more details :
   https://github.com/xelerance/xl2tpd/issues/147

The following commit/patch fixes the issue:
  https://github.com/xelerance/xl2tpd/commit/9c2cd4933478a83075df5b10f24af7589e90abc3.patch

As Ubuntu 18.04 (Bionic Beaver) is no longer accepting Debian packages, I'm guessing then that the patch would need to be added and applied to the existing xl2tpd-1.3.10-1 package.

The linux-image kernel package on Ubuntu 18.04 is currently linux-image-4.15.0-13

[Test Case]

* apt install network-manager-l2tp-gnome
* Add new VPN connection via network manager
* establish vpn connection

[Regression Potential]

* For Bionic, there is none. The xl2tpd daemon does not work without the patch on the standard bionic kernel.
* For Xenial w/ the 4.15 HWE kernel there is no risk.
* For Xenial w/ the stock 4.4 kernel there is minimal risk as the code is pulling values that are not valid on the 4.4 kernel by default.

Note that a test pkg (Pre-SRU) has been tested for both Bionic and Xenial by a user affected by this problem and the package has been proven to work as expected on the different supported Ubuntu kernel series (v4.4, v4.13, v4.15).

For more details, please have a look at comment #17 and #23.

[Douglas (douglas-redhat-bugs)]

kernels 4.16 release candidates and 4.15 have broken xl2tpd with a ENODEV (No such device) error, see:
   https://github.com/xelerance/xl2tpd/issues/147

As mentioned in that github xl2tpd issue, I've submitted a pull request that fixes the bug. It is based on xl2tpd-1.3.8-saref.patch in Fedora's xl2tpd RPM spec file, except it doesn't attempt to obtain the ancillary IP_PKTINFO data, so does pretty much what xl2tpd running on non-linux operating systems do.

I've requested a new xl2tpd release which has the pull request, but don't know how long it will be for the new release.

It would be great if Fedora 26, 27 and EPEL7 xl2tpd packages are updated to the next new release or have a patch based on the pull request.

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-3.el7 has been submitted as an update to Fedora EPEL 7. https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-8beb831755

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-7.fc28 has been submitted as an update to Fedora 28. https://bodhi.fedoraproject.org/updates/FEDORA-2018-96c9c68f33

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-7.fc27 has been submitted as an update to Fedora 27. https://bodhi.fedoraproject.org/updates/FEDORA-2018-91225f9e39

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-7.fc26 has been submitted as an update to Fedora 26. https://bodhi.fedoraproject.org/updates/FEDORA-2018-3095b294d1

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-3.el7 has been pushed to the Fedora EPEL 7 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-EPEL-2018-8beb831755

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-7.fc26 has been pushed to the Fedora 26 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-3095b294d1

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-7.fc27 has been pushed to the Fedora 27 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-91225f9e39

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-7.fc28 has been pushed to the Fedora 28 testing repository. If problems still persist, please make note of it in this bug report.
See https://fedoraproject.org/wiki/QA:Updates_Testing for
instructions on how to install test updates.
You can provide feedback for this update here: https://bodhi.fedoraproject.org/updates/FEDORA-2018-96c9c68f33

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-7.fc26 has been pushed to the Fedora 26 stable repository. If problems still persist, please make note of it in this bug report.

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-7.fc27 has been pushed to the Fedora 27 stable repository. If problems still persist, please make note of it in this bug report.

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-7.fc28 has been pushed to the Fedora 28 stable repository. If problems still persist, please make note of it in this bug report.

[Fedora (fedora-redhat-bugs)]

xl2tpd-1.3.8-3.el7 has been pushed to the Fedora EPEL 7 stable repository. If problems still persist, please make note of it in this bug report.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Billy Olsen (billy-olsen)]

I've built a test package containing the identified patch and put it in a PPA [0] for test purposes. Any feedback is welcome. If I get positive feedback, I'll propose this for SRU into Ubuntu but will require help from affected users to verify the package.

Since the 4.15 kernel can be installed on Xenial I'll take a look at that backport as well, as it will become relevant when the hwe-edge kernel becomes the hwe kernel for xenial.

[0] https://launchpad.net/~billy-olsen/+archive/ubuntu/xl2tpd-lp1760796

[Eric Desrochers (slashd)]

If the patch is backported in Xenial, I would strongly suggest users to not only test on 4.15 kernel but on 4.4 (Xenial LTS kernel) too to make sure it doesn't break things on kernel version <4.15.

On confirmation of positive feedback for Bionic (4.15) and for Xenial (4.4 LTS kernel and 4.15). I'll gladly sponsor it.

Extra note:
Setting Artful to "Won't fix" as Ubuntu 17.10 (Artful Aardvark) reaches End of Life on July 19 2018

[Douglas Kosovic (dkosovic)]

I can confirm I am able to establish a L2TP/IPsec connection with xl2tpd_1.3.10-1+lp1760796_amd64.deb test package with Bionic's latest 4.15 kernel.

I'll need to bring up a VM for xenial, but happy to test with kernel 4.4 and 4.15 on xenial for any backport. The version of xl2tpd in xenial updates is currently 1.3.6+dfsg-4ubuntu0.16.04.1 and suspect the patch should apply cleanly to it.

[Simon Déziel (sdeziel)]

@Billy, I also tested xl2tpd_1.3.10-1+lp1760796 and it fixes the issue, many thanks!

[Billy Olsen (billy-olsen)]

I've attached the bionic SRU patch to start the processing for the bionic release.

@Douglas & @Simon - I've also backported the patch to 16.04 and loaded it into the same PPA previously used [0]. I'll upload the Xenial patch for for SRU when I get some feedback on the patch itself. Its the same patch so I expect it to work fine, but would like your input on the 4.4 kernel, the 4.13 kernel (the current hwe kernel), and the 4.15 kernel (the current hwe-edge). Soon the 4.15 kernel will become the hwe kernel, but I don't want to break any users still on 4.13.

Thanks for your help & feedback testing the patches!

[0] - https://launchpad.net/~billy-olsen/+archive/ubuntu/xl2tpd-lp1760796

[Billy Olsen (billy-olsen)]

Note: uploaded the patch for Xenial in the off-chance I get delayed

[Douglas Kosovic (dkosovic)]

@Billy thanks for the Xenial xl2tpd test package.

I setup an Ubuntu 16.04.4 VM which came with kernel 4.13.0-36-generic and did an apt update followed by an apt upgrade and it installed kernel 4.15.0-29-generic. I didn't know the proper way to downgrade to kernel 4.4, so manually downloaded and installed the following :

    linux-headers-4.4.0-124_4.4.0-124.148_all.deb
    linux-headers-4.4.0-124-generic_4.4.0-124.148_amd64.deb
    linux-image-4.4.0-124-generic_4.4.0-124.148_amd64.deb

Following is a summary of the tests performed, first with unpatched xl2tpd followed by the test PPA xl2tpd.

Ubuntu 16.04.4 with xl2tpd-1.3.6+dfsg-4ubuntu0.16.04.1 :

* didn't check kernel 4.4.0-124-generic.

* kernel 4.13.0-36-generic - works as expected.

* kernel 4.15.0-29-generic - results in following error as expected :

  xl2tpd[2189]: udp_xmit failed ... with err=-1:No such device

Ubuntu 16.04.4 upgrade to xl2tpd-1.3.6+dfsg-4ubuntu0.16.04.1+lp1760796 :

* kernel 4.4.0-124-generic - works!

* kernel 4.13.0-36-generic - works!

* kernel 4.15.0-29-generic - works!

So things look okay to me in regards to xenial.

[Eric Desrochers (slashd)]

Thanks for the testing.

Based on the positive feedback we received from Douglas who have tested the patch in Bionic w/ 4.15 and Xenial w/ 4.4, 4.13 & 4.15.

I have sponsored the patch for Bionic and Xenial. It is now in the upload queue waiting for the SRU verification team to approve the upload and for the pkgs to start building in -proposed for the official test phase before the pkgs land into their final destination -updates.

- Eric

[Łukasz Zemczak (sil2100)]

Hello Douglas, or anyone else affected,

Accepted xl2tpd into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/xl2tpd/1.3.10-1ubuntu1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Łukasz Zemczak (sil2100)]

Hello Douglas, or anyone else affected,

Accepted xl2tpd into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/xl2tpd/1.3.6+dfsg-4ubuntu0.16.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

[Eric Desrochers (slashd)]

Hi Douglas,

Would you be amenable to redo the same exact testing one last time with the packages found in $RELEASE-proposed w/ the supported kernel series (4.4, 4.13, 4.15) ?

This package is exactly what the archive package of xl2tpd will look like at the end of this SRU.

If the feedback are still positive for each packages, the packages will then be copy into their final destination, $RELEASE-updates, and this will mark the bug as 'Fix Released'.

- Eric

[Eric Desrochers (slashd)]

** Additional details **

There is no blocker I could find (regression or else).

The packages doesn't seems to belong to any specific package automatic testing (autopkgtest - DEP8)

For more details:
https://people.canonical.com/~ubuntu-archive/pending-sru.html

- Eric

[Douglas Kosovic (dkosovic)]

Hi Eric and Łukasz,

I uninstalled existing xl2tpd from test PPA on xenial and bionic before installing xl2tpd from respective proposed repository.

On xenial I installed and tested xl2tpd_1.3.6+dfsg-4ubuntu0.16.04.2_amd64.deb and can confirm I'm able to establish L2TP/IPsec VPN connection with following kernels :

* kernel 4.4.0-124-generic.
* kernel 4.13.0-36-generic.
* kernel 4.15.0-29-generic.

Similarly on bionic with xl2tpd_1.3.10-1ubuntu1_amd64.deb, I can confirm VPN connection with following kernels :
* kernel 4.13.0-36-generic.
* kernel 4.15.0-29-generic.

I've changed verification-needed-bionic and verification-needed-xenial to verification-done-bionic and verification-done-xenial respectively.

I didn't change the verification-needed tag, but guess I should have.

--
Doug

[Eric Desrochers (slashd)]

Perfect thanks !

As an FYI.

As per SRU policy, packages have to stay for a minimum period of 7 days in $RELEASE-proposed to
give the chance to others to test the packages and report feedback too.

If everything is good we should be able to release the package next mi-week.

- Eric

[Eric Desrochers (slashd)]

Please do not change the status, the SRU machinery will take care of this when the package will be officially release (copied from -proposed to -updates)

Switching back to "Fix Commited"

[Launchpad Janitor (janitor)]

This bug was fixed in the package xl2tpd - 1.3.6+dfsg-4ubuntu0.16.04.2

---------------
xl2tpd (1.3.6+dfsg-4ubuntu0.16.04.2) xenial; urgency=medium

  * Fix ENODEV errors with linux kernel 4.15 (LP: #1760796)
    - d/p/fix-for-enodev.patch: cherry-pick patch from upstream to no
      longer attempt to obtain IP_PKTINFO by default.

 -- Billy Olsen <email address hidden> Sat, 21 Jul 2018 07:29:54 -0700

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for xl2tpd has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package xl2tpd - 1.3.10-1ubuntu1

---------------
xl2tpd (1.3.10-1ubuntu1) bionic; urgency=medium

  * Fix ENODEV errors with linux kernel 4.15 (LP: #1760796)
    - d/p/fix-for-enodev.patch: cherry-pick patch from upstream to no
      longer attempt to obtain IP_PKTINFO by default.

 -- Billy Olsen <email address hidden> Thu, 12 Jul 2018 12:07:29 -0700