Upgrade glibc 2.32 by required upstream patches

[bugproxy]

SRU Justification:

[Impact]

* The glibc version 2.32 in groovy has some regressions with the following impact:

* A regression that got introduced by commit ffd178c651b827f24acead02284abbb12f3f723b can lead to a crash, because __shmctl calls shmid_to_shmid64 on the input buffer even when cmd is IPC_INFO. If SHM_INFO is immediately followed by unmapped memory, shmid_to_shmid64 will read past its end into unmapped memory and will crash.

* Starting with glibc-2.31.9000-687-g3283f71113 (glibc-2.32~83) IPC_INFO and MSG_INFO commands of __msgctl and __msgctl64 return garbage because a pointer to an internal buffer on the stack is passed to the kernel. The buffer specified by the user remains unchanged after IPC_INFO and MSG_INFO commands.

* semctl SEM_STAT_ANY fails to pass the buffer specified by the caller to the kernel. The kernel receives garbage instead of union semun.buf address specified by the caller.

[ Fix ]

* a49d7fd4f764e97ccaf922e433046590ae52fce9 "32-bit shmctl(IPC_INFO) crashes when shminfo struct is at the end of a memory mapping"

* 20a00dbefca5695cccaa44846a482db8ccdd85ab "msgctl IPC_INFO and MSG_INFO return garbage"

* 574500a108be1d2a6a0dc97a075c9e0a98371aba "semctl SEM_STAT_ANY fails to pass the buffer specified by the caller to the kernel"

[ Test Case ]

Execute test script available upstream (1) here:
https://sourceware.org/bugzilla/show_bug.cgi?id=26636#c0

And run the test suite with the newly introduced test-cases that came with the commits.

[ Where problems could occur ]

* glibc modification are usually quite sensitive.

* Erroneous modifications (1) in the area of IPC and SHM (i.e. IPC_INFO and MSG_INFO) and it's control may lead to an even bigger impact - and in worst case the crashes go beyond the case where shminfo is immediately followed by unmapped memory and could happen always (which would break the system entirely).

* Returning (3) or passing over and pointing to wrong buffers (2), or in worst case to other unwanted areas, can cause virtually any unforeseen consequences.
Returning garbage is only one aspect, returning wrong data and even modifying it would be even worse.

[Other Info]

* All fixes are upstream accepted and are part of glibc 2.33.

__________

The current libc6-2.32-0ubuntu3 package lacks some of the upstream glibc commits on the "release/2.32/master" branch (see http://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.32/master). Does Ubuntu automatically pick the commits from this release branch?

Otherwise, please update to the latest commits on this branch, especially for:
- 2dfa659a66 resolv: Handle transaction ID collisions in parallel queries (bug 26600)
- 0b9460d22e sysvipc: Fix IPC_INFO and SHM_INFO handling [BZ #26636]
- c4aeedea59 sysvipc: Fix IPC_INFO and MSG_INFO handling [BZ #26639]
- 9b139b6b81 sysvipc: Fix SEM_STAT_ANY kernel argument pass [BZ #26637]
- ...

[Balint Reczey (rbalint)]

Those are already included in 2.32-0ubuntu5 in hirsute-proposed.

[Balint Reczey (rbalint)]

Is this a request for SRUs as well?

[Frank Heimes (fheimes)]

What I've heard, yes (to all 2.32 - so H <done> and G).
But I've asked IBM to confirm here ...

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-11-17 02:52 EDT-------
affects 2.32, hence SRU to G requested, yes

[Balint Reczey (rbalint)]

Regarding the original question we don't automatically follow the release branches in stable releases especially not in interim releases which have only 9 months of support.

I'm collecting patches for an upcoming SRU for Focal and Groovy, and the patches listed here can be included as well, but to go through the SRU process this bug needs more information. Could you please fill out the SRU template for the fixes? https://wiki.ubuntu.com/StableReleaseUpdates

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-12-10 10:23 EDT-------
The sysvipc related bugs were introduced with glibc 2.32 and are fixed upstream and will be included in upcoming glibc 2.33 release. In addition those commits were also cherry-picked to the release/2.32/master branch. Therefore no updates required for focal. As mentioned earlier, the hirsute libc6 2.32-0ubuntu5 package contains those commits (see debian/patches/git-updates.diff). Only the groovy libc6 2.32-0ubuntu3 package is affected. Those bugs are observable with the strace testsuite or the rr testsuite:
commit "sysvipc: Fix IPC_INFO and SHM_INFO handling [BZ #26636]"
(https://sourceware.org/git/?p=glibc.git;a=commit;h=0b9460d22e285432d232f42c7442a3226e1bf830)
"Bug 26636 - 32-bit shmctl(IPC_INFO) crashes when shminfo struct is at the end of a memory mapping "
(https://sourceware.org/bugzilla/show_bug.cgi?id=26636)

commit "sysvipc: Fix IPC_INFO and MSG_INFO handling [BZ #26639]"
(https://sourceware.org/git/?p=glibc.git;a=commit;h=c4aeedea598a1bd80f52ca9ebd07fe447680d491)
"Bug 26639 - msgctl IPC_INFO and MSG_INFO return garbage "
(https://sourceware.org/bugzilla/show_bug.cgi?id=26639)

commit "sysvipc: Fix SEM_STAT_ANY kernel argument pass [BZ #26637]"
(https://sourceware.org/git/?p=glibc.git;a=commit;h=9b139b6b81a5def91bec01f30301acc95fbf0289)
"Bug 26637 - semctl SEM_STAT_ANY fails to pass the buffer specified by the caller to the kernel "
(https://sourceware.org/bugzilla/show_bug.cgi?id=26637)

The DNS lookup can be slow downed as the A and AAAA queries times out after 5s before retrying. This is fixed upstream and will be included in upcoming glibc 2.33 release and it is also cherry-picked to the release/2.32/master branch. As mentioned earlier, the hirsute libc6 2.32-0ubuntu5 package contains this commit (see debian/patches/git-updates.diff).
commit "resolv: Handle transaction ID collisions in parallel queries (bug 26600)"
(https://sourceware.org/git/gitweb.cgi?p=glibc.git;h=2dfa659a66f20facc4082207884c20e986ddecee)
"Bug 26600 - Transaction ID collisions cause slow DNS lookups in getaddrinfo"
(https://sourceware.org/bugzilla/show_bug.cgi?id=26600)
"Bug 1868106 - glibc: Transaction ID collisions cause slow DNS lookups in getaddrinfo"
(https://bugzilla.redhat.com/show_bug.cgi?id=1868106)

[Dimitri John Ledkov (xnox)]

But we don't use nss_resolve from libc6 by default, we use systemd-resolved. Do we need the resolv fixes? (i.e. transaction IDs are handled by resolved)

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-12-11 03:37 EDT-------
Yes you are right, in this case send_dg() from glibc is not invoked.
Thus from my side, you can exclude commit "resolv: Handle transaction ID collisions in parallel queries (bug 26600)" for groovy.
Thanks.

[Frank Heimes (fheimes)]

Ok, so the following two topics remain:

1)
"Fixing unneeded extra conversion of IPC_INFO and SHM_INFO for __IPC_TIME64, since their information does not contain any time related fields."
Fixed with:

commit "sysvipc: Fix IPC_INFO and SHM_INFO handling [BZ #26636]"
(https://sourceware.org/git/?p=glibc.git;a=commit;h=0b9460d22e285432d232f42c7442a3226e1bf830)
"Bug 26636 - 32-bit shmctl(IPC_INFO) crashes when shminfo struct is at the end of a memory mapping "
(https://sourceware.org/bugzilla/show_bug.cgi?id=26636)
and
commit "sysvipc: Fix IPC_INFO and MSG_INFO handling [BZ #26639]"
(https://sourceware.org/git/?p=glibc.git;a=commit;h=c4aeedea598a1bd80f52ca9ebd07fe447680d491)
"Bug 26639 - msgctl IPC_INFO and MSG_INFO return garbage "
(https://sourceware.org/bugzilla/show_bug.cgi?id=26639)

2)
With SEM_STAT_ANY the buffer argument is not properly passed over (in contrast to just SEM_STAT).
Fixed with:

commit "sysvipc: Fix SEM_STAT_ANY kernel argument pass [BZ #26637]"
(https://sourceware.org/git/?p=glibc.git;a=commit;h=9b139b6b81a5def91bec01f30301acc95fbf0289)
"Bug 26637 - semctl SEM_STAT_ANY fails to pass the buffer specified by the caller to the kernel "
(https://sourceware.org/bugzilla/show_bug.cgi?id=26637)

(skipping resolv)

Since the integration of such patches in Ubuntu require that an SRU justification is provided;
https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template
I'm wondering what the real impact is? And if there is a known reproducer available?

For 1) is it just performance improvements due to the unneeded conversion or did something broke (particularly on s390x) ?

For 2) again is it just an improved performance due to the fact that the buffer was not properly passed over or did the erroneous passing over of the buffer led to errors and broke something?

How were these issues identified - due to code analysis or by users that raised bugs?

From the commits I noticed that there are regression test cases mentioned.
But I'm not sure if that's sufficient for the needed SRU TestCase section (https://wiki.ubuntu.com/StableReleaseUpdates#SRU_Bug_Template).

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-12-11 10:43 EDT-------
It has nothing to do with a performance improvement. It's just broken. Thus your Ubuntu 20.10 has a regression compared to Ubuntu 20.04! The issues were reported via glibc-upstream bugzillas:
- "Bug 26636 - 32-bit shmctl(IPC_INFO) crashes when shminfo struct is at the end of a memory mapping ":
...
Compile attached test program:
gcc -m32 -o ~/tmp/test ~/tmp/test.c
It should print something like "max=2147483647". With the above glibc commit, it crashes.
...
This causes an rr test failure: https://github.com/mozilla/rr/issues/2681

- "Bug 26639 - msgctl IPC_INFO and MSG_INFO return garbage "
Starting with commit glibc-2.31.9000-687-g3283f71113 aka glibc-2.32~83,
both IPC_INFO and MSG_INFO commands of msgctl return garbage because their argument of type "struct msginfo" is converted using kmsqid64_to_msqid64.
This bug was found by strace test suite.

- "Bug 26637 - semctl SEM_STAT_ANY fails to pass the buffer specified by the caller to the kernel"
The kernel receives garbage instead of union semun.buf address specified by the caller.
...
(As Dimitry has reported this bug on the same day as Bug 26639, I assume he also found it by strace test suite.)

The glibc-commits have added new testcaes which are run with "make check" while building the libc6 package. I assume you are monitoring those testfails.

[Frank Heimes (fheimes)]

Ok, just wanted to be sure - this will justify a glibc SRU.
(clicking the bug number earlier opened it as LP bug, which is <known to be> wrong and is usually an indicator for me that it's an internal IBM BZ number - but I've found the glibc upstream BZs now
 thx ...)
I've created a first SRU justification based on upstream info and added it to the bug description above.
And I've also marked this bug as 'regression' (add it as tag) and such regressions are usually marked (in LP) as critical as well.

[Łukasz Zemczak (sil2100)]

Hello bugproxy, or anyone else affected,

Accepted glibc into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/glibc/2.32-0ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2020-12-17 08:50 EDT-------
I've updated to libc6/groovy-proposed,now 2.32-0ubuntu3.1 (s390x and amd64) and have successfully run the strace-testsuite (especially ipc_msg/ipc_shm tests), glibc-tests which were added by the mentioned commits and the test attached to glibc-Bugzilla 26636 for 64bit and -m32 / -m31. Thanks.

[Frank Heimes (fheimes)]

Thx for the verification, I'm adjusting the tags accordingly.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted glibc (2.32-0ubuntu3.1) for groovy have finished running.
The following regressions have been reported in tests triggered by the package:

snapd-glib/1.58-0ubuntu0.20.10.0 (armhf)
libcompress-raw-bzip2-perl/unknown (armhf)
node-ws/7.3.0+~cs24.0.3-1build1 (s390x, armhf, ppc64el, amd64, arm64)
austin/unknown (armhf)
r-bioc-rhdf5lib/1.10.1+dfsg-1 (armhf)
tup/0.7.8-3 (arm64)
r-cran-seurat/unknown (armhf)
python-pomegranate/unknown (armhf)
ruby-mysql2/0.5.2-1ubuntu3 (armhf)
libdevel-cover-perl/1.36-1build1 (armhf)
undbx/unknown (armhf)
libtime-warp-perl/0.54-1build1 (ppc64el)
statsprocessor/unknown (armhf)
pycurl/7.43.0.2-7 (armhf)
libhash-fieldhash-perl/unknown (armhf)
reprotest/0.7.15 (s390x)
r-cran-httpuv/1.5.4+dfsg-1 (armhf)
libosinfo/unknown (armhf)
r-cran-batchtools/unknown (armhf)
chiark-tcl/1.3.4ubuntu3 (armhf)
libdbd-mariadb-perl/1.11-3ubuntu2 (s390x, ppc64el, amd64, arm64)
firefox/84.0+build3-0ubuntu0.20.10.1 (arm64)
hilive/2.0a-3build2 (amd64)
etcd/3.2.26+dfsg-8 (amd64)
libdbd-mariadb-perl/unknown (armhf)
prometheus/unknown (armhf)
libticonv/unknown (armhf)
notary/0.6.1~ds2-6 (armhf)
firebird3.0/unknown (armhf)
burrow/unknown (armhf)
ruby-stackprof/0.2.15-2 (arm64)
gnutls28/3.6.15-4ubuntu2 (i386, amd64, s390x, arm64, ppc64el, armhf)
postgresql-plproxy/2.9-2 (armhf)
librg-blast-parser-perl/0.03-6build2 (armhf)
binutils/2.35.1-1ubuntu1 (armhf)
clutter-1.0/1.26.4+dfsg-1 (arm64)
transtermhp/2.09-5 (armhf)
genext2fs/1.5.0-1 (s390x)
python-cmarkgfm/unknown (armhf)
libpff/20180714-2 (s390x, armhf, ppc64el, amd64, arm64)
debsig-verify/unknown (armhf)
bosh/unknown (armhf)
casync/2+20190213-1 (amd64)
golang-github-spf13-cobra/unknown (armhf)
tpm2-tools/unknown (armhf)
umockdev/0.14.3-1 (armhf)
crrcsim/unknown (armhf)
postgis/3.0.2+dfsg-2ubuntu2 (amd64)
r-cran-proc/unknown (armhf)
mercurial/5.5.1-1 (armhf)
r-cran-dqrng/unknown (armhf)
frobby/unknown (armhf)
libcrypt-cast5-perl/unknown (armhf)
drumkv1/0.9.17-1 (ppc64el)
nsf/unknown (armhf)
python3-lxc/1:3.0.4-1ubuntu6 (s390x)
hyphy/2.5.1+dfsg-3build1 (amd64)
fatrace/0.16-1 (arm64)
bio-rainbow/unknown (armhf)
python-fabio/0.10.2+dfsg-2 (armhf)
hhsuite/3.2.0-3 (amd64)
libperlio-layers-perl/0.012-1 (armhf)
postgresql-common/unknown (armhf)
menhir/20200624-1 (s390x)
notify-osd/0.9.35+20.04.20191129-0ubuntu1 (ppc64el)
r-cran-xfun/unknown (armhf)
gifsicle/1.92-2 (armhf)
libunix-processors-perl/unknown (armhf)
freebayes/unknown (armhf)
alertmanager-irc-relay/0.1.0-3 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/groovy/update_excuses.html#glibc

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Łukasz Zemczak (sil2100)]

Hello bugproxy, or anyone else affected,

Accepted glibc into groovy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/glibc/2.32-0ubuntu3.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-groovy to verification-done-groovy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-groovy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2021-04-08 06:20 EDT-------
Already verified on groovy by IBM

[Frank Heimes (fheimes)]

Thx, I'm updating the tags accordingly ...

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted glibc (2.32-0ubuntu3.2) for groovy have finished running.
The following regressions have been reported in tests triggered by the package:

cyrus-imapd/3.2.3-2ubuntu1 (armhf)
pyqt5/5.15.0+dfsg-1 (s390x)
libsass/3.6.4-3 (s390x)
network-manager/1.26.2-1ubuntu1 (arm64)
reprotest/0.7.15 (s390x)
flatpak/1.8.2-1ubuntu0.1 (arm64)
puma/3.12.4-1ubuntu2 (arm64, s390x)
libtext-charwidth-perl/0.04-10 (ppc64el)
systemd/246.6-1ubuntu1.3 (arm64)
taptempo/1.4.5-1 (arm64)
uftrace/0.9.3-1ubuntu1 (arm64)
libgdata/0.17.12-1 (armhf)
cysignals/1.10.2+ds-4 (amd64)
hyphy/2.5.1+dfsg-3build1 (amd64)
glibc/2.32-0ubuntu3.2 (amd64)
libdate-simple-perl/3.0300-3 (arm64)
netplan.io/0.101-0ubuntu3~20.10.1 (amd64)
libastro-fits-cfitsio-perl/1.14-1 (ppc64el)
fwlogwatch/1.4-2 (arm64)
pandas/1.0.5+dfsg-3 (ppc64el, armhf, s390x, amd64, arm64)
rhonabwy/0.9.12-2build1 (s390x)
jailkit/2.21-2 (ppc64el)
openjdk-lts/11.0.10+9-0ubuntu1~20.10 (s390x)
dbus/1.12.20-1ubuntu1 (arm64)
pymca/5.5.5+dfsg-2build2 (arm64)
euslisp/9.27+dfsg-6 (armhf, amd64)
libflame/5.2.0-2 (amd64)
samtools/1.10-4 (arm64)
syncthing/1.10.0~ds1-1 (amd64, s390x)
endlessh/1.1-4 (armhf)
r-cran-amore/0.2-16-1build1 (ppc64el)
crrcsim/0.9.13-3.2build1 (ppc64el)
samplv1/0.9.17-1 (ppc64el)
hugo/0.74.3-1 (armhf)
libcsfml/2.5-1build1 (ppc64el)
firefox/87.0+build3-0ubuntu0.20.10.1 (armhf)
etcd/3.2.26+dfsg-8 (amd64)
libnxml/0.18.3-8 (s390x)
google-osconfig-agent/20210219.00-0ubuntu1~20.10.0 (armhf)
combblas/1.6.2-5build1 (arm64)
libbio-db-hts-perl/3.01-3 (amd64)
libterm-readkey-perl/2.38-1build1 (s390x)
ruby2.7/2.7.1-3ubuntu1.2 (armhf)
hkl/5.0.0.2620-1build1 (ppc64el)
liblinux-inotify2-perl/1:2.2-2 (ppc64el)
datefudge/1.24 (ppc64el)
libpgplot-perl/1:2.24-1build1 (s390x)
healpy/1.14.0-1 (arm64)
udisks2/2.9.1-2ubuntu1 (amd64)
kopanocore/8.7.0-7ubuntu4 (arm64)
gyoto/1.4.4-3build1 (s390x)
cpdb-libs/1.2.0-0ubuntu8 (armhf)
ruby-concurrent/1.1.6+dfsg-3 (amd64)
postgis/3.0.2+dfsg-2ubuntu2 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/groovy/update_excuses.html#glibc

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Frank Heimes (fheimes)]

Ubuntu 20.10 (Groovy Gorilla) reached its End of Life on July 22 2021, hence I'm updating the groovy entry of this ticket - and with that the overall ticket status.