FTBFS with glibc 2.35

[Christian Ehrhardt ]

FTBFS found when building in jammy-proposed:

https://launchpadlibrarian.net/583951613/buildlog_ubuntu-jammy-amd64.open-vm-tools_2%3A11.3.5-1ubuntu3~jammyppa1_BUILDING.txt.gz

libtool: compile: gcc -DPACKAGE_NAME=\"open-vm-tools\" -DPACKAGE_TARNAME=\"open-vm-tools\" -DPACKAGE_VERSION=\"11.3.5\" "-DPACKAGE_STRING=\"open-vm-tools 11.3.5\"" -DPACKAGE_BUGREPORT=\"<email address hidden>\" -DPACKAGE_URL=\"\" -DHAVE_STDIO_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STRINGS_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_UNISTD_H=1 -DHAVE_SYS_TIME_H=1 -DSTDC_HEADERS=1 -DPACKAGE=\"open-vm-tools\" -DVERSION=\"11.3.5\" -DHAVE_DLFCN_H=1 -DLT_OBJDIR=\".libs/\" -DHAVE_FUSE3=1 -DFUSE_USE_VERSION=35 -DHAVE_X11_SM_SMLIB_H=1 -DHAVE_X11_ICE_ICELIB_H=1 -DHAVE_X11_EXTENSIONS_XCOMPOSITE_H=1 -DHAVE_DLOPEN=1 -DHAVE_ECVT=1 -DHAVE_FCVT=1 -DNO_DNET=1 -DHAVE_CRYPT_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_WCHAR_H=1 -DHAVE_SYS_IO_H=1 -DHAVE_SYS_PARAM_H=1 -DHAVE_SYS_SYSINFO_H=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_USER_H=1 -DHAVE_SYS_VFS_H=1 -DHAVE_UNWIND_H=1 -DHAVE__BOOL=1 -DHAVE_STDBOOL_H=1 -DHAVE_STRUCT_STAT_ST_RDEV=1 -DTIME_WITH_SYS_TIME=1 -I. -I/<<PKGBUILDDIR>>/open-vm-tools/lib/include -I/<<PKGBUILDDIR>>/open-vm-tools/lib/include -Wdate-time -D_FORTIFY_SOURCE=2 -DUSING_AUTOCONF=1 -DOPEN_VM_TOOLS -DUSE_VGAUTH -DNO_ICU -DVMX86_TOOLS -D_FILE_OFFSET_BITS=64 -D_LARGEFILE64_SOURCE -D_XOPEN_SOURCE=500 -D_BSD_SOURCE -D_SVID_SOURCE -D_DEFAULT_SOURCE -DENABLE_RESOLUTIONKMS -DUSE_SSL_DIRECT -fPIC -Wno-error=deprecated-declarations -Wno-error=address-of-packed-member -Wno-nonnull -g -O2 -ffile-prefix-map=/<<PKGBUILDDIR>>=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -Wall -Werror -Wno-pointer-sign -Wno-unused-value -fno-strict-aliasing -Wno-unknown-pragmas -Wno-uninitialized -Wno-deprecated-declarations -Wno-unused-const-variable -Wno-unused-but-set-variable -c asyncSocketInterface.c -fPIC -DPIC -o asyncSocketInterface.o >/dev/null 2>&1
In file included from /usr/include/features.h:486,
                 from /usr/include/x86_64-linux-gnu/bits/libc-header-start.h:33,
                 from /usr/include/stdio.h:27,
                 from asyncsocket.c:45:
In function ‘poll’,
    inlined from ‘AsyncTCPSocketPollWork’ at asyncsocket.c:2861:19:
/usr/include/x86_64-linux-gnu/bits/poll2.h:39:10: error: ‘__poll_chk’ specified size between 18446744071562067968 and 18446744073709551615 exceeds maximum object size 9223372036854775807 [-Werror=stringop-overflow=]
   39 | return __glibc_fortify (poll, __nfds, sizeof (*__fds),
      | ^~~~~~~~~~~~~~~
In file included from /usr/include/x86_64-linux-gnu/sys/poll.h:87,
                 from asyncsocket.c:74:
/usr/include/x86_64-linux-gnu/bits/poll2.h: In function ‘AsyncTCPSocketPollWork’:
/usr/include/x86_64-linux-gnu/bits/poll2.h:28:12: note: in a call to function ‘__poll_chk’ declared with attribute ‘access (write_only, 1, 2)’
   28 | extern int __poll_chk (struct pollfd *__fds, nfds_t __nfds, int __timeout,
      | ^~~~~~~~~~
cc1: all warnings being treated as errors

Recently libc6 2.35 was uploaded and poll is implemented there.
 libc6 | 2.34-0ubuntu3 | jammy | amd64, arm64, armhf, i386, ppc64el, riscv64, s390x
 libc6 | 2.35-0ubuntu1 | jammy-proposed | amd64, arm64, armhf, i386, ppc64el, riscv64, s390x

We know that internally to vmware there is a 12.0 version prepared,
but public github master still is at 11.3.5.

Checking combinations:
11.3.5 as in Jammy - Jammyi (via d/rules): working
11.3.5 as in Jammy - Jammy-proposed (via d/rules): failing
open-vm-tools latest master - Jammy-proposed (autoconf/conf/make): working
tag stable-11.3.5 from git (autoconf/conf/make) - Jammy proposed: working
tag stable-11.3.5 from git (autoconf/conf/make Ubuntu options) - Jammy proposed: failing

1. Ubuntu build "d/rules build"

2. Upstream build:
autoreconf -i
./configure
make
sudo make install
sudo ldconfig

3. upstream build with Ubuntu options
git clean -x -f -d
autoreconf -i
./configure --without-kernel-modules --with-pic --enable-xmlsec1 --with-gtk3 --enable-servicediscovery --with-fuse=2 CFLAGS="-fPIC -Wno-error=deprecated-declarations -Wno-error=address-of-packed-member -Wno-nonnull -g -O2 -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security" CUSTOM_PROCPS_NAME=procps
make

I've tried various config options, this issue seems to happen only if the following conditions align:

1. build against glibc 2.35: 2.35-0ubuntu1
2. LTO is enabled: -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects
3. Optimizing with gcc: -O2
4. gcc 11.2.0

This isn't a new or show-stopping issue compared to before, therefore in Ubuntu
we will ignore this particular error for now.
In addition I have reported it upstream to eventually be resolved for real.

Working in tests for now is the following mitigation which I'll propose:

diff --git a/open-vm-tools/lib/asyncsocket/asyncsocket.c b/open-vm-tools/lib/asyncsocket/asyncsocket.c
index 05147d2e..546932ec 100644
--- a/open-vm-tools/lib/asyncsocket/asyncsocket.c
+++ b/open-vm-tools/lib/asyncsocket/asyncsocket.c
@@ -2850,6 +2850,10 @@ AsyncTCPSocketPollWork(AsyncTCPSocket **asock, // IN:
          pfd[i].events = read ? POLLIN : POLLOUT;
       }

+#if defined(__GNUC__) && (__GNUC__ >= 11)
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wstringop-overflow"
+#endif
       if (parentSock != NULL) {
          AsyncTCPSocketUnlock(parentSock);
          retval = poll(pfd, numSock, timeoutMS);
@@ -2863,6 +2867,9 @@ AsyncTCPSocketPollWork(AsyncTCPSocket **asock, // IN:
             AsyncTCPSocketLock(asock[i]);
          }
       }
+#if defined(__GNUC__) && (__GNUC__ >= 11)
+#pragma GCC diagnostic pop
+#endif
 #else
       tv.tv_sec = timeoutMS / 1000;
       tv.tv_usec = (timeoutMS % 1000) * 1000;

[Christian Ehrhardt  (paelzer)]

Reported upstream:
https://github.com/vmware/open-vm-tools/issues/570

[Launchpad Janitor (janitor)]

This bug was fixed in the package open-vm-tools - 2:11.3.5-1ubuntu3

---------------
open-vm-tools (2:11.3.5-1ubuntu3) jammy; urgency=medium

  * d/control, d/rules: other dependencies seem to be all ready, switch
    to fuse3 again (LP 1956949 LP 1935665)
  * d/p/libc-2.35-compat-FTBFS-fix.patch: fix FTBFS with glibc 2.35
    (LP: #1960224)

 -- Christian Ehrhardt <email address hidden> Tue, 01 Feb 2022 16:29:32 +0100