CONFIG_BASE_SMALL=1 restricts pid space, which conflicts with systemd default sysctl

[David Monro]

[Impact]
systemd-systemctl will fail to set kernel.pid_max, leading to a degraded boot.

[Fix]
Set CONFIG_BASE_FULL=y, CONFIG_BASE_SMALL=0.

[Test case]
Write 419304 to /proc/sys/kernel/pid_max.

[Potential regression]
Boot time may be affected.

====================================================================

I'm not completely sure which package to log this against.

I'm running the kvm focal minimal cloud image from 20200302. I noticed on boot that there was an error complaining that systemd-systemctl couldn't update pid_max to the value it wanted:

systemd-sysctl[117]: Couldn't write '4194304' to 'kernel/pid_max': Invalid argument

Digging into it a bit more, this comes from /usr/lib/sysctl.d/50-pid-max.conf:
# Bump the numeric PID range to its maximum of 2^22 (from the in-kernel default
# of 2^16), to make PID collisions less likely.
kernel.pid_max = 4194304

However, the linux-image-kvm kernel is compiled with
CONFIG_BASE_SMALL=1

and this triggers the following code in include/linux/threads.h

#define PID_MAX_LIMIT (CONFIG_BASE_SMALL ? PAGE_SIZE * 8 : \
 (sizeof(long) > 4 ? 4 * 1024 * 1024 : PID_MAX_DEFAULT))

which means that if CONFIG_BASE_SMALL is set we get a maximum limit of PAGE_SIZE * 8, which on x86 would be 32768.

As a workaround I can override it with a file in /etc/sysctl.d/ but this shouldn't be needed.

I really don't know if CONFIG_BASE_SMALL makes any sense on x86 cloud images, they really aren't small machines in the scheme of things!

Cheers

David

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Paride Legovini (paride)]

This is causing LXD VMs (Focal, possibly others) to boot in "degraded" state, see [1]. The underlying cause is:

root@paride-f-delme:~# systemctl status systemd-sysctl.service
● systemd-sysctl.service - Apply Kernel Variables
     Loaded: loaded (/lib/systemd/system/systemd-sysctl.service; static; vendor preset: enabled)
     Active: failed (Result: exit-code) since Fri 2020-11-06 16:43:35 UTC; 14min ago
       Docs: man:systemd-sysctl.service(8)
             man:sysctl.d(5)
   Main PID: 301 (code=exited, status=1/FAILURE)

root@paride-f-delme:~# /lib/systemd/systemd-sysctl
Couldn't write '176' to 'kernel/sysrq', ignoring: No such file or directory
Not setting net/ipv4/conf/all/promote_secondaries (explicit setting exists).
Not setting net/ipv4/conf/default/promote_secondaries (explicit setting exists).
Couldn't write 'fq_codel' to 'net/core/default_qdisc', ignoring: No such file or directory
Couldn't write '4194304' to 'kernel/pid_max': Invalid argument

[1] https://paste.ubuntu.com/p/TQhpFWYzK7/

[Christian Ehrhardt  (paelzer)]

Linux KVM is meant for VM guests and probably therefore was set to CONFIG_BASE_SMALL=1.
The discussion "if this makes sense or not" is up to the kernel team but it seems this bug has got no attention by them yet :-/.

I was quickly assessing the case and found:
- Linux-generic is on CONFIG_BASE_SMALL=0 in B/F/G/H as one would expect.
- linux-kvm in B/F/G/H was all the time set to CONFIG_BASE_SMALL=0
- the option we talk about here is [1][2]
- IMHO "speed loss for saving a few KB" does not sound right. IIRC linux-kvm was meant more for
  speed than density. We are virtual, but not as small as an embedded controller for example.
- This goes back to [3] when linux-kvm was introduced
- I didn't find a commit in [4] that listed any extended reasoning/discussion when enabling [5] it.

IMHO I agree and this should be switched in the kernel config.
I don't think that changing the pid-max was wrong. Even older Xenial linux-kvm running in 2020 can be considered bigger than what CONFIG_BASE_SMALL was meant for.

But further actions are for the kernel Team to consider and make. Therefore I'm adding a src:linux task, subscribing some more people and pinging the kernel team internally to get some attention.

[1]: https://mirrors.edge.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.11-rc5/2.6.11-rc5-mm1/broken-out/base-small-introduce-the-config_base_small-flag.patch
[2]: https://github.com/torvalds/linux/blob/master/init/Kconfig#L1518
[3]: https://kernel.ubuntu.com/~kernel-ppa/config/xenial/linux-kvm/4.4.0-1004.9/amd64-config.flavour.kvm
[5]: https://git.launchpad.net/~canonical-kernel/ubuntu/+source/linux-kvm/tree/debian.kvm/config/config.common.ubuntu#n157
[4]: https://git.launchpad.net/~canonical-kernel/ubuntu/+source/linux-kvm

[Christian Ehrhardt  (paelzer)]

Obviously
- linux-kvm in B/F/G/H was all the time set to CONFIG_BASE_SMALL=0
was some copy-pasta and should be
- linux-kvm in X/B/F/G/H was all the time set to CONFIG_BASE_SMALL=1

[Ubuntu Kernel Bot (ubuntu-kernel-bot)]

This bug is missing log files that will aid in diagnosing the problem. While running an Ubuntu kernel (not a mainline or third-party kernel) please enter the following command in a terminal window:

apport-collect 1866149

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Thadeu Lima de Souza Cascardo (cascardo)]

There is no other effect of disabling CONFIG_BASE_FULL save for changing CONFIG_BASE_SMALL.

CONFIG_BASE_SMALL won't have much effect save for other memory savings that really would be able to reduce some runtime performance. As for boot time, changing this setting should not be expected to cause any significant changes.

At focal, we have:

arch/x86/include/asm/mpspec.h: changes MAX_MP_BUSSES only on 32-bit systems.
drivers/tty/vt/vc_screen.c: change is not effective on memory reducing as of commit fcdba07ee390d
include/linux/udp.h:
include/linux/xarray.h:
kernel/futex.c:
kernel/user.c:
These are all reducing the initial size of hash structures, causing more list traversals to happen when there are hash collisions. From here, there would be some negative effect on runtime execution, but memory allocation of these hashes should not introduce significant boot time regression as I mentioned.

So, all in all, there is really no reason to keep CONFIG_BASE_SMALL=1 in any of the kernels we ship.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-kvm - 4.15.0-1084.86

---------------
linux-kvm (4.15.0-1084.86) bionic; urgency=medium

  * bionic/linux-kvm: 4.15.0-1084.86 -proposed tracker (LP: #1911288)

  * CONFIG_BASE_SMALL=1 restricts pid space, which conflicts with systemd
    default sysctl (LP: #1866149)
    - [Config]: set CONFIG_BASE_FULL

  [ Ubuntu: 4.15.0-133.137 ]

  * bionic/linux: 4.15.0-133.137 -proposed tracker (LP: #1911295)
  * [drm:qxl_enc_commit [qxl]] *ERROR* head number too large or missing monitors
    config: (LP: #1908219)
    - qxl: remove qxl_io_log()
    - qxl: move qxl_send_monitors_config()
    - qxl: hook monitors_config updates into crtc, not encoder.
  * Touchpad not detected on ByteSpeed C15B laptop (LP: #1906128)
    - Input: i8042 - add ByteSpeed touchpad to noloop table
  * vmx_nm_test in ubuntu_kvm_unit_tests interrupted on X-oracle-4.15 /
    B-oracle-4.15 / X-KVM / B-KVM (LP: #1872401)
    - KVM: nVMX: Always reflect #NM VM-exits to L1
  * stack trace in kernel (LP: #1903596)
    - net: napi: remove useless stack trace
  * CVE-2020-27777
    - [Config]: Set CONFIG_PPC_RTAS_FILTER
  * Bionic update: upstream stable patchset 2020-12-04 (LP: #1906875)
    - regulator: defer probe when trying to get voltage from unresolved supply
    - ring-buffer: Fix recursion protection transitions between interrupt context
    - time: Prevent undefined behaviour in timespec64_to_ns()
    - nbd: don't update block size after device is started
    - btrfs: sysfs: init devices outside of the chunk_mutex
    - btrfs: reschedule when cloning lots of extents
    - genirq: Let GENERIC_IRQ_IPI select IRQ_DOMAIN_HIERARCHY
    - hv_balloon: disable warning when floor reached
    - net: xfrm: fix a race condition during allocing spi
    - perf tools: Add missing swap for ino_generation
    - ALSA: hda: prevent undefined shift in snd_hdac_ext_bus_get_link()
    - can: rx-offload: don't call kfree_skb() from IRQ context
    - can: dev: can_get_echo_skb(): prevent call to kfree_skb() in hard IRQ
      context
    - can: dev: __can_get_echo_skb(): fix real payload length return value for RTR
      frames
    - can: can_create_echo_skb(): fix echo skb generation: always use skb_clone()
    - can: peak_usb: add range checking in decode operations
    - can: peak_usb: peak_usb_get_ts_time(): fix timestamp wrapping
    - can: peak_canfd: pucan_handle_can_rx(): fix echo management when loopback is
      on
    - xfs: flush new eof page on truncate to avoid post-eof corruption
    - Btrfs: fix missing error return if writeback for extent buffer never started
    - ath9k_htc: Use appropriate rs_datalen type
    - usb: gadget: goku_udc: fix potential crashes in probe
    - gfs2: Free rd_bits later in gfs2_clear_rgrpd to fix use-after-free
    - gfs2: Add missing truncate_inode_pages_final for sd_aspace
    - gfs2: check for live vs. read-only file system in gfs2_fitrim
    - scsi: hpsa: Fix memory leak in hpsa_init_one()
    - drm/amdgpu: perform srbm soft reset always on SDMA resume
    - mac80211: fix use of skb payload instead of header
    - cfg80211: regulatory: Fix inconsistent format argument
    - scsi: scsi_dh_alua: Avoid crash during al...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-kvm - 5.4.0-1032.33

---------------
linux-kvm (5.4.0-1032.33) focal; urgency=medium

  * focal/linux-kvm: 5.4.0-1032.33 -proposed tracker (LP: #1911326)

  * CONFIG_BASE_SMALL=1 restricts pid space, which conflicts with systemd
    default sysctl (LP: #1866149)
    - [Config]: set CONFIG_BASE_FULL

  * Missing module nfsv4 in linux-kvm (LP: #1907266)
    - [Config] kvm: Enable NFSv4

  [ Ubuntu: 5.4.0-63.71 ]

  * focal/linux: 5.4.0-63.71 -proposed tracker (LP: #1911333)
  * overlay: permission regression in 5.4.0-51.56 due to patches related to
    CVE-2020-16120 (LP: #1900141)
    - ovl: do not fail because of O_NOATIME
  * Focal update: v5.4.79 upstream stable release (LP: #1907151)
    - net/mlx5: Use async EQ setup cleanup helpers for multiple EQs
    - net/mlx5: poll cmd EQ in case of command timeout
    - net/mlx5: Fix a race when moving command interface to events mode
    - net/mlx5: Add retry mechanism to the command entry index allocation
  * Kernel 5.4.0-56 Wi-Fi does not connect (LP: #1906770)
    - mt76: fix fix ampdu locking
  * [Ubuntu 21.04 FEAT] mpt3sas: Request to include the patch set which supports
    topology where zoning is enabled in expander (LP: #1899802)
    - scsi: mpt3sas: Define hba_port structure
    - scsi: mpt3sas: Allocate memory for hba_port objects
    - scsi: mpt3sas: Rearrange _scsih_mark_responding_sas_device()
    - scsi: mpt3sas: Update hba_port's sas_address & phy_mask
    - scsi: mpt3sas: Get device objects using sas_address & portID
    - scsi: mpt3sas: Rename transport_del_phy_from_an_existing_port()
    - scsi: mpt3sas: Get sas_device objects using device's rphy
    - scsi: mpt3sas: Update hba_port objects after host reset
    - scsi: mpt3sas: Set valid PhysicalPort in SMPPassThrough
    - scsi: mpt3sas: Handling HBA vSES device
    - scsi: mpt3sas: Add bypass_dirty_port_flag parameter
    - scsi: mpt3sas: Handle vSES vphy object during HBA reset
    - scsi: mpt3sas: Add module parameter multipath_on_hba
    - scsi: mpt3sas: Bump driver version to 35.101.00.00

  [ Ubuntu: 5.4.0-62.70 ]

  * focal/linux: 5.4.0-62.70 -proposed tracker (LP: #1911144)
  * CVE-2020-28374
    - SAUCE: target: fix XCOPY NAA identifier lookup
  * Packaging resync (LP: #1786013)
    - update dkms package versions

  [ Ubuntu: 5.4.0-60.67 ]

  * Packaging resync (LP: #1786013)
    - [Packaging] update variants
    - update dkms package versions
  * CVE-2021-1052 // CVE-2021-1053
    - [Packaging] NVIDIA -- Add the NVIDIA 460 driver

 -- Kelsey Skunberg <email address hidden> Thu, 14 Jan 2021 15:25:42 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-kvm - 5.8.0-1016.18

---------------
linux-kvm (5.8.0-1016.18) groovy; urgency=medium

  * groovy/linux-kvm: 5.8.0-1016.18 -proposed tracker (LP: #1911344)

  * CONFIG_BASE_SMALL=1 restricts pid space, which conflicts with systemd
    default sysctl (LP: #1866149)
    - [Config]: set CONFIG_BASE_FULL

  * Missing module nfsv4 in linux-kvm (LP: #1907266)
    - [Config] kvm: Enable NFSv4

  [ Ubuntu: 5.8.0-39.44 ]

  * groovy/linux: 5.8.0-39.44 -proposed tracker (LP: #1911350)
  * overlay: permission regression in 5.4.0-51.56 due to patches related to
    CVE-2020-16120 (LP: #1900141)
    - ovl: do not fail because of O_NOATIME

  [ Ubuntu: 5.8.0-38.43 ]

  * groovy/linux: 5.8.0-38.43 -proposed tracker (LP: #1911143)
  * CVE-2020-28374
    - SAUCE: target: fix XCOPY NAA identifier lookup
  * Packaging resync (LP: #1786013)
    - update dkms package versions

  [ Ubuntu: 5.8.0-36.40 ]

  * debian/scripts/file-downloader does not handle positive failures correctly
    (LP: #1878897)
    - [Packaging] file-downloader not handling positive failures correctly

  [ Ubuntu: 5.8.0-35.39 ]

  * Packaging resync (LP: #1786013)
    - update dkms package versions
  * CVE-2021-1052 // CVE-2021-1053
    - [Packaging] NVIDIA -- Add the NVIDIA 460 driver

 -- Kelsey Skunberg <email address hidden> Thu, 14 Jan 2021 16:31:26 -0700

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-kvm - 4.4.0-1087.96

---------------
linux-kvm (4.4.0-1087.96) xenial; urgency=medium

  * xenial/linux-kvm: 4.4.0-1087.96 -proposed tracker (LP: #1911257)

  * CONFIG_BASE_SMALL=1 restricts pid space, which conflicts with systemd
    default sysctl (LP: #1866149)
    - [Config]: set CONFIG_BASE_FULL

  [ Ubuntu: 4.4.0-201.233 ]

  * xenial/linux: 4.4.0-201.233 -proposed tracker (LP: #1911265)
  * Touchpad not detected on ByteSpeed C15B laptop (LP: #1906128)
    - Input: i8042 - add ByteSpeed touchpad to noloop table
  * stack trace in kernel (LP: #1903596)
    - net: napi: remove useless stack trace
  * CVE-2020-27777
    - powerpc/rtas: Restrict RTAS requests from userspace
    - [Config]: Set CONFIG_PPC_RTAS_FILTER
  * Xenial update: v4.4.247 upstream stable release (LP: #1906703)
    - btrfs: tree-checker: Enhance chunk checker to validate chunk profile
    - btrfs: inode: Verify inode mode to avoid NULL pointer dereference
    - HID: cypress: Support Varmilo Keyboards' media hotkeys
    - Input: i8042 - allow insmod to succeed on devices without an i8042
      controller
    - HID: hid-sensor-hub: Fix issue with devices with no report ID
    - x86/xen: don't unbind uninitialized lock_kicker_irq
    - proc: don't allow async path resolution of /proc/self components
    - dmaengine: pl330: _prep_dma_memcpy: Fix wrong burst size
    - scsi: libiscsi: Fix NOP race condition
    - scsi: target: iscsi: Fix cmd abort fabric stop race
    - scsi: ufs: Fix race between shutdown and runtime resume flow
    - bnxt_en: fix error return code in bnxt_init_board()
    - video: hyperv_fb: Fix the cache type when mapping the VRAM
    - bnxt_en: Release PCI regions when DMA mask setup fails during probe.
    - IB/mthca: fix return value of error branch in mthca_init_cq()
    - nfc: s3fwrn5: use signed integer for parsing GPIO numbers
    - efivarfs: revert "fix memory leak in efivarfs_create()"
    - perf probe: Fix to die_entrypc() returns error correctly
    - USB: core: Change %pK for __user pointers to %px
    - x86/speculation: Fix prctl() when spectre_v2_user={seccomp,prctl},ibpb
    - USB: core: add endpoint-blacklist quirk
    - USB: core: Fix regression in Hercules audio card
    - btrfs: fix lockdep splat when reading qgroup config on mount
    - Linux 4.4.247
  * Xenial update: v4.4.246 upstream stable release (LP: #1906700)
    - ah6: fix error return code in ah6_input()
    - atm: nicstar: Unmap DMA on send error
    - net: b44: fix error return code in b44_init_one()
    - net: bridge: add missing counters to ndo_get_stats64 callback
    - netlabel: fix our progress tracking in netlbl_unlabel_staticlist()
    - netlabel: fix an uninitialized warning in netlbl_unlabel_staticlist()
    - net/mlx4_core: Fix init_hca fields offset
    - net: x25: Increase refcnt of "struct x25_neigh" in x25_rx_call_request
    - qlcnic: fix error return code in qlcnic_83xx_restart_hw()
    - sctp: change to hold/put transport for proto_unreach_timer
    - net: usb: qmi_wwan: Set DTR quirk for MR400
    - net: Have netpoll bring-up DSA management interface
    - pinctrl: rockchip: enable gpio pclk for rockchip_gpio_to_irq
 ...

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-kvm - 5.8.0-1018.20+21.04.1

---------------
linux-kvm (5.8.0-1018.20+21.04.1) hirsute; urgency=medium

  * hirsute/linux-kvm: 5.8.0-1018.20+21.04.1 -proposed tracker (LP: #1914678)

  * Boot fails: failed to validate module [nls_iso8859_1] BTF: -22
    (LP: #1911359)
    - SAUCE: x86/entry: build thunk_$(BITS) only if CONFIG_PREEMPTION=y

  * Packaging resync (LP: #1786013)
    - update dkms package versions

  * Missing module nfsv4 in linux-kvm (LP: #1907266)
    - [Config] kvm-21.04: Enable NFSv4

  * Miscellaneous Ubuntu changes
    - sync dkms nvidia-server 418 and 450 to -release

  [ Ubuntu: 5.8.0-1018.20 ]

  * groovy/linux-kvm: 5.8.0-1018.20 -proposed tracker (LP: #1914679)
  * groovy/linux: 5.8.0-43.49 -proposed tracker (LP: #1914689)
  * Packaging resync (LP: #1786013)
    - update dkms package versions
  * Exploitable vulnerabilities in AF_VSOCK implementation (LP: #1914668)
    - vsock: fix the race conditions in multi-transport support
  * groovy/linux: 5.8.0-41.46 -proposed tracker (LP: #1912219)
  * Groovy update: upstream stable patchset 2020-12-17 (LP: #1908555) // nvme
    drive fails after some time (LP: #1910866)
    - Revert "nvme-pci: remove last_sq_tail"
  * initramfs unpacking failed (LP: #1835660)
    - SAUCE: lib/decompress_unlz4.c: correctly handle zero-padding around initrds.
  * overlay: permission regression in 5.4.0-51.56 due to patches related to
    CVE-2020-16120 (LP: #1900141)
    - ovl: do not fail because of O_NOATIME
  * Packaging resync (LP: #1786013)
    - update dkms package versions

  [ Ubuntu: 5.8.0-1016.18 ]

  * groovy/linux-kvm: 5.8.0-1016.18 -proposed tracker (LP: #1911344)
  * CONFIG_BASE_SMALL=1 restricts pid space, which conflicts with systemd
    default sysctl (LP: #1866149)
    - [Config]: set CONFIG_BASE_FULL
  * Missing module nfsv4 in linux-kvm (LP: #1907266)
    - [Config] kvm: Enable NFSv4
  * groovy/linux: 5.8.0-39.44 -proposed tracker (LP: #1911350)
  * overlay: permission regression in 5.4.0-51.56 due to patches related to
    CVE-2020-16120 (LP: #1900141)
    - ovl: do not fail because of O_NOATIME
  * groovy/linux: 5.8.0-38.43 -proposed tracker (LP: #1911143)
  * CVE-2020-28374
    - SAUCE: target: fix XCOPY NAA identifier lookup
  * Packaging resync (LP: #1786013)
    - update dkms package versions
  * debian/scripts/file-downloader does not handle positive failures correctly
    (LP: #1878897)
    - [Packaging] file-downloader not handling positive failures correctly
  * Packaging resync (LP: #1786013)
    - update dkms package versions
  * CVE-2021-1052 // CVE-2021-1053
    - [Packaging] NVIDIA -- Add the NVIDIA 460 driver

 -- Paolo Pisati <email address hidden> Tue, 09 Feb 2021 11:27:33 +0100