Ubuntu 18.04.3 LTS bump Glibc 2.27 to the latest stable

[Romain Naour]

[Impact]

* Ubuntu 18.04 is missing various stability and performance fixes that have been added to upstream's 2.27 branch. The accumulated changes are known to fix various issues already reported to Launchpad.

[Test Case]
* Observe that debian/patches/git-updates-2.diff contains the missing upstream commits intended to be backported.
* Observe the patch being applied at build time.

* All triggered autopkgtests were run in the Bileto PPA before the SRU upload took place and no reggressions were found.
* Several issues fixed in git-updates-2.diff were reported on Launchpad and the ones having reproducers were and will be verified separately.

[Regression Potential]
* Any form of regression is possible including hangs, live locks and crashes due to the broad range of fixes to be backported. In addition to the standard autopkgtests it is recommended to keep the packages in bionic-proposed longer and call for testing on additional public channels, such as on the ubuntu-devel mailing list.

The update seem to have triggered an existing, but hiding bug in lftp:
LP: #1902832.
The update caused a regression originally observed with GLibc 2.28 in 18.10 and later releases: LP: #1821677.

[Original Bug Text]

Hi,

I updated from ubuntu 14.04 to 18.04 and installed a custom (old) application.

When starting the application it stop immediately with this error message:
"glibc detected an invalid stdio handle"

This error message was added by commit [1] "libio: Implement vtable verification [BZ #20191]" to fix a security issue [2].

I tested with several Linux distribution (so different libc version) and the application is working fine with Fedora 30 (Glibc 2.29).

There is an interesting patch [3] from Glibc 2.28 which was backported to Glibc 2.27 [4] "libio: Disable vtable validation in case of interposition [BZ #23313]"

But Ubuntu 18.04 is still using an old Glibc 2.27 version (from 02-2018).
Here is the Glibc version used in 18.04:
$ dpkg -s libc6
[...]
Version: 2.27-3ubuntu1

Looking at the changelog, ubuntu updated Glibc 2.27 the 16 Apr 2018 but there is a lot of fix from upstream Glibc 2.27 stable branch. The one I'm looking for was merged the 07-2018.

It would be great if Ubuntu 18.04 can update Glibc to the latest stable version.

Best regards,
Romain

[1] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=db3476aff19b75c4fdefbe65fcd5f0a90588ba51
[2] https://dhavalkapil.com/blogs/FILE-Structure-Exploitation
[3] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=c402355dfa7807b8e0adb27c009135a7e2b9f1b0
[4] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=3bb748257405e94e13de76573a4e9da1cfd961d0

[Loïc Minier (lool)]

Hi Romain and thanks for your report,

In general, we don't make large upstream updates after release, especially for LTS and even more so for a critical piece like glibc. If you've identified the single patch that would address your issue, it can be considered for the SRU process:
https://wiki.ubuntu.com/StableReleaseUpdates

[Florian Weimer (fw)]

I think what Romain is asking is to rebase along release/2.27/master, and not a different upstream version. If you don't do that, you have to evaluate each upstream commit individually for backporting.

[Romain Naour (kubu44)]

Hi Loïc,

You're welcome.

Yes, I understand that glibc is a critical piece and we need to do the upgrade carefully.

With my customer, we tested with several ubuntu versions:
ubuntu 14.04: glibc 2.19: OK
ubuntu 16.04: glibc 2.23: OK
ubuntu 18.04: glibc 2.27: KO
ubuntu 18.10: glibc 2.28: OK

Only the LTS 18.04 is affected.

The patch I'm looking for has been backported [1] by Glibc upstream project (for good reason) and they are certainly more competent than me to complete the SRU process.

Maybe other patches related to libio can be necessary [2] and there are some patches related to CVE.
I only tested up to the last Debian glibc 2.27 version packaged [3].

[1] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=3bb748257405e94e13de76573a4e9da1cfd961d0
[2] https://sourceware.org/git/?p=glibc.git;a=commitdiff;h=0262507918cfad7223bf81b8f162b7adc7a2af01
[3] https://salsa.debian.org/glibc-team/glibc/commit/0c8d271ac59dc2e4ee6bd509d59049080bd87f76

Best regards,
Romain

[Adam Conrad (adconrad)]

There is an intent to rebase to 2.27/master, yes.

[Romain Naour (kubu44)]

Hi Adam,

Thanks for your feedback.
Do you have any info (date) about the work in progress to update Glibc 2.27 for Ubuntu 18.04?

Best regards,
Romain

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Balint Reczey (rbalint)]

The SRU is being pre-tested in https://bileto.ubuntu.com/#/ticket/4217 .
There are a low number of test failures which could indicate regressions, those are being checked and fixed if needed before the upload to the archive. Additional testing is welcome.

[Balint Reczey (rbalint)]

There is a new Bileto PPA with the most likely final SRU:

https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4242/+packages

[Romain Naour (kubu44)]

Hello,

Le 02/09/2020 à 17:14, Balint Reczey a écrit :
> There is a new Bileto PPA with the most likely final SRU:
>
> https://launchpad.net/~ci-train-ppa-
> service/+archive/ubuntu/4242/+packages
>

I installed the ppa and tested the glibc 2.27-3 package.
I can confirm the issue #1851263 can't be reproduced.

Thanks for the update!

Best regards,
Romain

[Steve Langasek (vorlon)]

Hello Romain, or anyone else affected,

Accepted glibc into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/glibc/2.27-3ubuntu1.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted glibc (2.27-3ubuntu1.3) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

mysql-5.7/5.7.31-0ubuntu0.18.04.1 (armhf)
libsys-utmp-perl/1.8-1 (armhf)
libscope-upper-perl/0.30-1 (armhf)
octave-miscellaneous/1.2.1-4 (armhf, arm64, s390x, amd64, i386, ppc64el)
libsocket-multicast6-perl/unknown (armhf)
octave-strings/1.2.0-3 (armhf, arm64, s390x, amd64, i386, ppc64el)
libgnatcoll/unknown (armhf)
octave-econometrics/1:1.1.1-5 (armhf, arm64, s390x, amd64, i386, ppc64el)
octave-secs2d/0.0.8-9 (armhf, arm64, s390x, amd64, i386, ppc64el)
libb-hooks-parser-perl/unknown (armhf)
octave-general/2.0.0-3 (armhf, arm64, s390x, amd64, i386, ppc64el)
libcompress-raw-bzip2-perl/2.074-1build2 (armhf)
libunicode-casefold-perl/unknown (armhf)
mod-wsgi/4.5.17-1ubuntu1 (ppc64el)
libdata-alias-perl/unknown (armhf)
libdata-clone-perl/unknown (armhf)
libsort-key-perl/unknown (armhf)
linux-raspi-5.4/5.4.0-1018.20~18.04.1 (armhf)
ann/unknown (armhf)
icecast2/unknown (i386)
python-maxminddb/1.3.0-1 (armhf)
lua-torch-sundown/unknown (armhf)
libkf5mailcommon/4:17.12.3-0ubuntu1 (arm64, i386)
apport/2.20.9-0ubuntu7.17 (amd64)
linux-hwe-5.0/5.0.0-61.65 (armhf)
ffmpeg/7:3.4.8-0ubuntu0.2 (armhf, arm64, s390x, amd64, i386, ppc64el)
glibc/2.27-3ubuntu1.3 (armhf)
nut/2.7.4-5.1ubuntu2 (amd64)
mbuffer/unknown (armhf)
linux-aws-edge/5.0.0-1019.21~18.04.1 (amd64, arm64)
octave-ocs/0.1.5-6 (armhf, arm64, s390x, amd64, i386, ppc64el)
libx11-xcb-perl/unknown (armhf)
pgbouncer/1.8.1-1build1 (amd64)
indicator-session/17.3.20+17.10.20171006-0ubuntu1 (armhf)
gcc-6/6.5.0-2ubuntu1~18.04 (armhf)
vmtouch/unknown (armhf)
libhtml-gumbo-perl/0.17-1build1 (ppc64el)
octave-sparsersb/1.0.5-3 (armhf, arm64, s390x, amd64, i386, ppc64el)
octave-mpi/1.2.0-4 (armhf, arm64, s390x, amd64, i386, ppc64el)
libalgorithm-svm-perl/0.13-2build2 (s390x)
libconvert-binary-c-perl/0.78-1build2 (amd64)
kauth/5.44.0-0ubuntu1 (i386)
libkdegames-kde4/unknown (amd64)
openssh/1:7.6p1-4ubuntu0.3 (armhf, arm64, s390x, amd64, i386, ppc64el)
keditbookmarks/17.12.3-0ubuntu1 (ppc64el)
jovie/unknown (armhf)
kdepim-runtime/4:17.12.3-0ubuntu2 (armhf)
libscalar-util-numeric-perl/0.40-1build3 (s390x)
pgpdump/unknown (armhf)
libdevice-cdio-perl/0.4.0-3 (armhf)
octave-sockets/1.2.0-3 (armhf, arm64, s390x, amd64, i386, ppc64el)
octave-gsl/2.1.0-3 (armhf, arm64, s390x, amd64, i386, ppc64el)
libdbd-odbc-perl/1.56-1build1 (armhf)
libnet-dbus-perl/1.1.0-4build2 (armhf)
linux-aws-5.3/unknown (arm64)
libalgorithm-permute-perl/0.16-1 (s390x)
xdg-desktop-portal/1.0.3-0ubuntu0.2 (i386, ppc64el)
octave-ltfat/2.2.0+dfsg-7 (s390x, amd64, i386, ppc64el)
octave-geometry/3.0.0-6 (armhf, arm64, s390x, amd64, i386, ppc64el)
octave-linear-algebra/2.2.2-4 (armhf, arm64, s390x, amd64, i386, ppc64el)
octave-nurbs/1.3.13-4 (armhf, arm64, s390x, amd64, i386, ppc64el)
devscripts/2.17.12ubuntu1.1 (armhf, arm64, s390x, amd64, i386, ppc64el)
meliae/0.4.0+bzr199-3build1 (ppc64el)
libocas/unknown (armhf)
k3d/unknown (armhf)
firefox/80.0.1+build1-0ubuntu0.18.04.1 (armhf)
libb-hooks-op-check-perl/unknown (armhf)
octave-quaternion/2.4.0-4 (armhf, arm64, s390x, amd64, i38...

[Mohammed Naser (mnaser)]

I've verified that at least my issue is resolved by this:

```
docker run -it --rm ubuntu:18.04
apt update
apt install g++ wget
wget https://sourceware.org/bugzilla/attachment.cgi?id=11382 -O bug23861.c
sed -i 's/do_exit = 0/do_exit(0)/' bug23861.c
g++ bug23861.c -lpthread -o bug23861
for ((x=1;x<100;x++)) ; do echo $x;date;./bug23861 --prefer-writer-nonrecursive;done
```

The above hands, now if we try to update to the newer proposed version.

```
apt install lsb-release
cat <<EOF >/etc/apt/sources.list.d/ubuntu-$(lsb_release -cs)-proposed.list
# Enable Ubuntu proposed archive
deb http://archive.ubuntu.com/ubuntu/ $(lsb_release -cs)-proposed restricted main multiverse universe
EOF
apt update
apt install libc6
for ((x=1;x<100;x++)) ; do echo $x;date;./bug23861 --prefer-writer-nonrecursive;done
```

Now it keeps going without crashing.

[Balint Reczey (rbalint)]

Verified 2.27-3ubuntu1.3 on Bionic:

* Observe that debian/patches/git-updates-2.diff contains the missing upstream commits intended to be backported.

Done.

* Observe the patch being applied at build time.

https://launchpadlibrarian.net/497109365/buildlog_ubuntu-bionic-arm64.glibc_2.27-3ubuntu1.3_BUILDING.txt.gz :
...
dpkg-source: info: applying git-updates.diff
dpkg-source: info: applying git-updates-2.diff
dpkg-source: info: applying locale/check-unknown-symbols.diff
...

* All triggered autopkgtests were run in the Bileto PPA before the SRU upload took place and no reggressions were found.

No regressions were found while testing the package in bionic-proposed either.

* Several issues fixed in git-updates-2.diff were reported on Launchpad and the ones having reproducers were and will be verified separately.

All other bugs expected to being fixed by the upload have been verified.

[Balint Reczey (rbalint)]

Setting block-proposed to pick the right time to land the update.

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for glibc has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package glibc - 2.27-3ubuntu1.3

---------------
glibc (2.27-3ubuntu1.3) bionic; urgency=medium

  [ Balint Reczey ]
  * debian/gbp.conf: Add initial configuration
  * debian/control.in/main: Add Vcs-* pointing to Ubuntu packaging repository
  * arm64: Enable searching shared libraries in atomics/ on LSE HW
  * Ship arm64 variant with LSE support in libc6-lse (LP: #1885012)
  * Run tests of libc6-lse on HW supporting LSE
  * debian/patches/git-updates.diff: update from upstream stable branch
    - pthread_cond_broadcast: Fix waiters-after-spinning case
    - Fix SSe2-based memmove corrupting memory (CVE-2017-18269)
    - Fix strstr() performance regression on Haswell processors
    - Support Japanese new era "令和 (Reiwa)"
    - io: Remove copy_file_range emulation
    (LP: #1851263, #1858203, #1838327, #1797335, #1756209, #1853193)
  * XFAIL stdlib/tst-getrandom (LP: #1891403)
  * debian/testsuite-xfail-debian.mk: XFAIL new tst-support_descriptors

  [ Thadeu Lima de Souza Cascardo ]
  * tests: Make preadwritev2 invalid flags tests unsupported (LP: #1770480)

  [ Andreas Hasenack ]
  * branch-pthread_rwlock_trywrlock-hang-23844.patch:
    nptl: Fix pthread_rwlock_try*lock stalls (Bug 23844) (LP: #1864864)

 -- Balint Reczey <email address hidden> Wed, 02 Sep 2020 11:18:37 +0200