linux v4.15 ftbfs on a newer host kernel (e.g. hwe)

[Matthias Klose]

[Impact]

 * linux v4.15.x fails to build on newer host machine kernels (i.e. hwe /
   v4.18+ )

[Fix]

 * Instead of relying on a potentially incompatible host system/kernel definitions of PF_MAX, use the one from the source tree.

 * Upstream commit in v5.1-rc1-2-gdfbd199a7cfe:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=master&id=dfbd199a7cfe3e3cd8531e1353cdbd7175bfbc5e

which was cherrypicked to stable as:
https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?h=linux-4.19.y&id=145f6a70bb9b422a6df5fb8b9408046151f3e4f9

Submitted to the kernel team mailing list (whilst spamming too many people in CC) at
https://lists.ubuntu.com/archives/kernel-team/2019-June/101190.html

[Test Case]

 * Install linux-hwe kernel, reboot, try to rebuild linux GA kernel.

[Regression Potential]

 * This is a FTBFS fix only, to use matching definitions from the target kernel at build time.

[Other Info]

 * Original bug report, autopkgtest regression rebuild test case of the bionic's linux-oracle kernel on newer host kernels:

https://launchpadlibrarian.net/417924791/buildlog_ubuntu-disco-amd64.linux-oracle_4.15.0-1010.12_BUILDING.txt.gz

  HOSTLD scripts/mod/modpost
  HOSTCC scripts/selinux/genheaders/genheaders
In file included from /<<PKGBUILDDIR>>/scripts/selinux/genheaders/genheaders.c:19:
/<<PKGBUILDDIR>>/security/selinux/include/classmap.h:247:2: error: #error New address family defined, please update secclass_map.
 #error New address family defined, please update secclass_map.
  ^~~~~
make[6]: *** [scripts/Makefile.host:90: scripts/selinux/genheaders/genheaders] Error 1
make[5]: *** [/<<PKGBUILDDIR>>/scripts/Makefile.build:606: scripts/selinux/genheaders] Error 2
make[4]: *** [/<<PKGBUILDDIR>>/scripts/Makefile.build:606: scripts/selinux] Error 2
make[3]: *** [/<<PKGBUILDDIR>>/Makefile:589: scripts] Error 2
make[2]: *** [/<<PKGBUILDDIR>>/Makefile:278: __build_one_by_one] Error 2
make[2]: Leaving directory '/<<PKGBUILDDIR>>/debian/build/build-oracle'
make[1]: *** [Makefile:146: sub-make] Error 2
make[1]: Leaving directory '/<<PKGBUILDDIR>>'

[Matthias Klose (doko)]

also in cosmic:
https://launchpadlibrarian.net/418308401/buildlog_ubuntu-cosmic-amd64.linux-oracle_4.15.0-1009.11_BUILDING.txt.gz

[Dan Streetman (ddstreet)]

the same 4.15 kernel is getting built on b/c/d/e, and scripts/selinux/genheaders/genheaders.c includes the local header files, specifically <sys/socket.h>:
#include <sys/socket.h>
which define PF_MAX (via AF_MAX) as 45 for cosmic and later. however genheaders.c also:
#include "classmap.h"
which is security/selinux/include/classmap.h and that does:
#if PF_MAX > 44
#error New address family defined, please update secclass_map.
#endif

$ git grep AF_MAX ubuntu/bionic/master-next socket.h
ubuntu/bionic/master-next:socket.h:#define AF_MAX 44 /* For now.. */
ubuntu/bionic/master-next:socket.h:#define PF_MAX AF_MAX
$ git grep AF_MAX ubuntu/cosmic/master-next socket.h
ubuntu/cosmic/master-next:socket.h:#define AF_MAX 45 /* For now.. */
ubuntu/cosmic/master-next:socket.h:#define PF_MAX AF_MAX
$ git grep AF_MAX ubuntu/disco/master-next socket.h
ubuntu/disco/master-next:socket.h:#define AF_MAX 45 /* For now.. */
ubuntu/disco/master-next:socket.h:#define PF_MAX AF_MAX
$ git grep AF_MAX ubuntu/eoan/master-next socket.h
ubuntu/eoan/master-next:socket.h:#define AF_MAX 45 /* For now.. */
ubuntu/eoan/master-next:socket.h:#define PF_MAX AF_MAX

This is probably a bug that should be fixed in genheaders.c upstream, although I don't quite understand why the 4.15 linux-oracle kernel is getting built on cosmic, disco, or eoan.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Dan Streetman (ddstreet)]

This is difficult because security/selinux/ doesn't use the *actual* kernel headers for the kernel being built to get the AF_MAX/PF_MAX defines; those are not included in the kernel uapi headers. Instead, those defines come from /usr/include/x86_64-linux-gnu/bits/socket.h which is provided by:
$ dpkg -S bits/socket.h
libc6-dev:amd64: /usr/include/x86_64-linux-gnu/bits/socket.h

[Dimitri John Ledkov (xnox)]

It's up to the kernel team to manage kernel bugs.

Are you linux-oracle maintainer?

On Tue, 4 Jun 2019, 21:35 Dan Streetman, <email address hidden>
wrote:

> This is difficult because security/selinux/ doesn't use the *actual*
> kernel headers for the kernel being built to get the AF_MAX/PF_MAX defines;
> those are not included in the kernel uapi headers. Instead, those defines
> come from /usr/include/x86_64-linux-gnu/bits/socket.h which is provided by:
> $ dpkg -S bits/socket.h
> libc6-dev:amd64: /usr/include/x86_64-linux-gnu/bits/socket.h
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1831645).
> https://bugs.launchpad.net/bugs/1823429
>
> Title:
> linux-oracle ftbfs in disco
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/linux-oracle/+bug/1823429/+subscriptions
>

[Dan Streetman (ddstreet)]

> Are you linux-oracle maintainer?

I only play one on TV.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1823429

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-bionic' to 'verification-done-bionic'. If the problem still exists, change the tag 'verification-needed-bionic' to 'verification-failed-bionic'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Dimitri John Ledkov (xnox)]

linux_4.15.0-54.58.dsc -> cannot be built on eoan. and failed very quckily.

linux_4.15.0-55.60.dsc -> builds for a very long time until my machine runs out of disk space. And does get past the beginning of the build where it fails.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 4.15.0-55.60

---------------
linux (4.15.0-55.60) bionic; urgency=medium

  * linux: 4.15.0-55.60 -proposed tracker (LP: #1834954)

  * Request backport of ceph commits into bionic (LP: #1834235)
    - ceph: use atomic_t for ceph_inode_info::i_shared_gen
    - ceph: define argument structure for handle_cap_grant
    - ceph: flush pending works before shutdown super
    - ceph: send cap releases more aggressively
    - ceph: single workqueue for inode related works
    - ceph: avoid dereferencing invalid pointer during cached readdir
    - ceph: quota: add initial infrastructure to support cephfs quotas
    - ceph: quota: support for ceph.quota.max_files
    - ceph: quota: don't allow cross-quota renames
    - ceph: fix root quota realm check
    - ceph: quota: support for ceph.quota.max_bytes
    - ceph: quota: update MDS when max_bytes is approaching
    - ceph: quota: add counter for snaprealms with quota
    - ceph: avoid iput_final() while holding mutex or in dispatch thread

  * QCA9377 isn't being recognized sometimes (LP: #1757218)
    - SAUCE: USB: Disable USB2 LPM at shutdown

  * hns: fix ICMP6 neighbor solicitation messages discard problem (LP: #1833140)
    - net: hns: fix ICMP6 neighbor solicitation messages discard problem
    - net: hns: fix unsigned comparison to less than zero

  * Fix occasional boot time crash in hns driver (LP: #1833138)
    - net: hns: Fix probabilistic memory overwrite when HNS driver initialized

  * use-after-free in hns_nic_net_xmit_hw (LP: #1833136)
    - net: hns: fix KASAN: use-after-free in hns_nic_net_xmit_hw()

  * hns: attempt to restart autoneg when disabled should report error
    (LP: #1833147)
    - net: hns: Restart autoneg need return failed when autoneg off

  * systemd 237-3ubuntu10.14 ADT test failure on Bionic ppc64el (test-seccomp)
    (LP: #1821625)
    - powerpc: sys_pkey_alloc() and sys_pkey_free() system calls
    - powerpc: sys_pkey_mprotect() system call

  * [UBUNTU] pkey: Indicate old mkvp only if old and curr. mkvp are different
    (LP: #1832625)
    - pkey: Indicate old mkvp only if old and current mkvp are different

  * [UBUNTU] kernel: Fix gcm-aes-s390 wrong scatter-gather list processing
    (LP: #1832623)
    - s390/crypto: fix gcm-aes-s390 selftest failures

  * System crashes on hot adding a core with drmgr command (4.15.0-48-generic)
    (LP: #1833716)
    - powerpc/numa: improve control of topology updates
    - powerpc/numa: document topology_updates_enabled, disable by default

  * Kernel modules generated incorrectly when system is localized to a non-
    English language (LP: #1828084)
    - scripts: override locale from environment when running recordmcount.pl

  * [UBUNTU] kernel: Fix wrong dispatching for control domain CPRBs
    (LP: #1832624)
    - s390/zcrypt: Fix wrong dispatching for control domain CPRBs

  * CVE-2019-11815
    - net: rds: force to destroy connection if t_sock is NULL in
      rds_tcp_kill_sock().

  * Sound device not detected after resume from hibernate (LP: #1826868)
    - drm/i915: Force 2*96 MHz cdclk on glk/cnl when audio power is enabled
    - drm/i915: Save the old CDCLK atomic state
...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-oracle - 4.15.0-1018.20

---------------
linux-oracle (4.15.0-1018.20) bionic; urgency=medium

  * linux-oracle: 4.15.0-1018.20 -proposed tracker (LP: #1834943)

  [ Ubuntu: 4.15.0-55.60 ]

  * linux: 4.15.0-55.60 -proposed tracker (LP: #1834954)
  * Request backport of ceph commits into bionic (LP: #1834235)
    - ceph: use atomic_t for ceph_inode_info::i_shared_gen
    - ceph: define argument structure for handle_cap_grant
    - ceph: flush pending works before shutdown super
    - ceph: send cap releases more aggressively
    - ceph: single workqueue for inode related works
    - ceph: avoid dereferencing invalid pointer during cached readdir
    - ceph: quota: add initial infrastructure to support cephfs quotas
    - ceph: quota: support for ceph.quota.max_files
    - ceph: quota: don't allow cross-quota renames
    - ceph: fix root quota realm check
    - ceph: quota: support for ceph.quota.max_bytes
    - ceph: quota: update MDS when max_bytes is approaching
    - ceph: quota: add counter for snaprealms with quota
    - ceph: avoid iput_final() while holding mutex or in dispatch thread
  * QCA9377 isn't being recognized sometimes (LP: #1757218)
    - SAUCE: USB: Disable USB2 LPM at shutdown
  * hns: fix ICMP6 neighbor solicitation messages discard problem (LP: #1833140)
    - net: hns: fix ICMP6 neighbor solicitation messages discard problem
    - net: hns: fix unsigned comparison to less than zero
  * Fix occasional boot time crash in hns driver (LP: #1833138)
    - net: hns: Fix probabilistic memory overwrite when HNS driver initialized
  * use-after-free in hns_nic_net_xmit_hw (LP: #1833136)
    - net: hns: fix KASAN: use-after-free in hns_nic_net_xmit_hw()
  * hns: attempt to restart autoneg when disabled should report error
    (LP: #1833147)
    - net: hns: Restart autoneg need return failed when autoneg off
  * systemd 237-3ubuntu10.14 ADT test failure on Bionic ppc64el (test-seccomp)
    (LP: #1821625)
    - powerpc: sys_pkey_alloc() and sys_pkey_free() system calls
    - powerpc: sys_pkey_mprotect() system call
  * [UBUNTU] pkey: Indicate old mkvp only if old and curr. mkvp are different
    (LP: #1832625)
    - pkey: Indicate old mkvp only if old and current mkvp are different
  * [UBUNTU] kernel: Fix gcm-aes-s390 wrong scatter-gather list processing
    (LP: #1832623)
    - s390/crypto: fix gcm-aes-s390 selftest failures
  * System crashes on hot adding a core with drmgr command (4.15.0-48-generic)
    (LP: #1833716)
    - powerpc/numa: improve control of topology updates
    - powerpc/numa: document topology_updates_enabled, disable by default
  * Kernel modules generated incorrectly when system is localized to a non-
    English language (LP: #1828084)
    - scripts: override locale from environment when running recordmcount.pl
  * [UBUNTU] kernel: Fix wrong dispatching for control domain CPRBs
    (LP: #1832624)
    - s390/zcrypt: Fix wrong dispatching for control domain CPRBs
  * CVE-2019-11815
    - net: rds: force to destroy connection if t_sock is NULL in
      rds_tcp_kill_sock().
  * Sound device not detected after resume from hibernate (LP: #1826868)
    - drm/i915: Force 2*9...

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is awaiting verification that the kernel in -proposed solves the problem. Please test the kernel and update this bug with the results. If the problem is solved, change the tag 'verification-needed-xenial' to 'verification-done-xenial'. If the problem still exists, change the tag 'verification-needed-xenial' to 'verification-failed-xenial'.

If verification is not done by 5 working days from today, this fix will be dropped from the source code, and this bug will be closed.

See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you!

[Po-Hsu Lin (cypressyew)]

Patch available in Focal, mark this as Fix released.