IP_SET modules not included in kernel build, prevents container functionality

[Jeffrey Forman]

I would like to run Ubuntu Minimal Cloud Image as the base OS on which I run Kubernetes and various other containers, but it requires ipset and various kernel modules included in the build. Is there a way to have those added by default, or a documented process for me to modify the kernel on my Minimal Cloud images?

I tried to use kube-router as a routing platform for connecting the containers between Ubuntu hosts, but this requires ipset.

The error from kube-router:
I0921 00:19:45.773157 1 kube-router.go:207] Running /usr/local/bin/kube-router version v0.2.0, built on 2018-09-12T08:03:59-0400, go1.10.3
Failed to run kube-router: Failed to create network routing controller: Failed to create ipset set on system: ipset v6.34: Kernel error received: set type not supported

After some digging, it seems I need the following modules at least compiled into the kernel as modules:
root@deva-controller0:~# cat "/boot/config-`uname -r`" | grep IP_SET
CONFIG_IP_SET=y
CONFIG_IP_SET_MAX=256
# CONFIG_IP_SET_BITMAP_IP is not set
# CONFIG_IP_SET_BITMAP_IPMAC is not set
# CONFIG_IP_SET_BITMAP_PORT is not set
# CONFIG_IP_SET_HASH_IP is not set
# CONFIG_IP_SET_HASH_IPMARK is not set
# CONFIG_IP_SET_HASH_IPPORT is not set
# CONFIG_IP_SET_HASH_IPPORTIP is not set
# CONFIG_IP_SET_HASH_IPPORTNET is not set
# CONFIG_IP_SET_HASH_IPMAC is not set
# CONFIG_IP_SET_HASH_MAC is not set
# CONFIG_IP_SET_HASH_NETPORTNET is not set
# CONFIG_IP_SET_HASH_NET is not set
# CONFIG_IP_SET_HASH_NETNET is not set
# CONFIG_IP_SET_HASH_NETPORT is not set
# CONFIG_IP_SET_HASH_NETIFACE is not set
# CONFIG_IP_SET_LIST_SET is not set

Ubuntu Release:
~# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.1 LTS"

Any help is greatly appreciated.

[Kamal Mostafa (kamalmostafa)]

Note: The IP_SET_* modules are already enabled in linux-kvm-xenial.

[Jeffrey Forman (jeffreyforman)]

To clarify your comment #1, I could install linux-kvm-xenial now to get the modules, but your work will eventually add them to the default kernel used by Ubuntu Minimal Cloud images?

Thanks in advance

[Kamal Mostafa (kamalmostafa)]

Here's a test kernel with the IP_SET_* modules enabled:

http://kernel.ubuntu.com/~kamal/lp1793841/

Please confirm that it enables your workflow.

[Jeffrey Forman (jeffreyforman)]

ubuntu@deva-controller0:~$ uname -a
Linux deva-controller0 4.15.0-1022-kvm #22+ipset.0 SMP Mon Sep 24 15:12:21 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

ubuntu@deva-controller0:~$ cd /boot

ubuntu@deva-controller0:/boot$ grep SET config-4.15.0-1022-kvm
CONFIG_ARCH_HAS_SET_MEMORY=y
CONFIG_CPUSETS=y
CONFIG_FAT_DEFAULT_IOCHARSET="iso8859-1"
CONFIG_HAVE_SETUP_PER_CPU_AREA=y
CONFIG_IP_SET=y
CONFIG_IP_SET_BITMAP_IP=m
CONFIG_IP_SET_BITMAP_IPMAC=m
CONFIG_IP_SET_BITMAP_PORT=m
CONFIG_IP_SET_HASH_IP=m
CONFIG_IP_SET_HASH_IPMAC=m
CONFIG_IP_SET_HASH_IPMARK=m
CONFIG_IP_SET_HASH_IPPORT=m
CONFIG_IP_SET_HASH_IPPORTIP=m
CONFIG_IP_SET_HASH_IPPORTNET=m
CONFIG_IP_SET_HASH_MAC=m
CONFIG_IP_SET_HASH_NET=m
CONFIG_IP_SET_HASH_NETIFACE=m
CONFIG_IP_SET_HASH_NETNET=m
CONFIG_IP_SET_HASH_NETPORT=m
CONFIG_IP_SET_HASH_NETPORTNET=m
CONFIG_IP_SET_LIST_SET=m
CONFIG_IP_SET_MAX=256
# CONFIG_NETFILTER_XT_SET is not set
# CONFIG_POWER_RESET is not set
CONFIG_PROC_PID_CPUSET=y
# CONFIG_RESET_CONTROLLER is not set

root@deva-controller0:~# lsmod | grep ip_set
ip_set_hash_net 28672 0
ip_set_hash_ip 24576 0

------------------------

This looks good to me Kamal, Thanks!

[Kamal Mostafa (kamalmostafa)]

Queued up for linux-kvm (4.15.0-1023.23) bionic.

[Kamal Mostafa (kamalmostafa)]

Correction: This fix is queued up for linux-kvm (4.15.0-1024.24) bionic.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-kvm - 4.18.0-1002.2

---------------
linux-kvm (4.18.0-1002.2) cosmic; urgency=medium

  * linux-kvm: 4.18.0-1001.1 -proposed tracker (LP: #1795413)

  * Miscellaneous Ubuntu changes
    - kvm: [Config] CONFIG_HARDENED_USERCOPY=y
    - kvm: [Config] CONFIG_DEBUG_WX=y

 -- Seth Forshee <email address hidden> Mon, 01 Oct 2018 09:27:19 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux-kvm - 4.15.0-1025.25

---------------
linux-kvm (4.15.0-1025.25) bionic; urgency=medium

  * linux-kvm: 4.15.0-1025.25 -proposed tracker (LP: #1797065)

  [ Ubuntu: 4.15.0-38.41 ]

  * linux: 4.15.0-38.41 -proposed tracker (LP: #1797061)
  * Silent data corruption in Linux kernel 4.15 (LP: #1796542)
    - block: add a lower-level bio_add_page interface
    - block: bio_iov_iter_get_pages: fix size of last iovec
    - blkdev: __blkdev_direct_IO_simple: fix leak in error case
    - block: bio_iov_iter_get_pages: pin more pages for multi-segment IOs

linux-kvm (4.15.0-1024.24) bionic; urgency=medium

  * linux-kvm: 4.15.0-1024.24 -proposed tracker (LP: #1795572)

  * kvm kernel missing nbd module (LP: #1793976)
    - kvm: [Config] enable BLK_DEV_NBD

  * IP_SET modules not included in kernel build, prevents container
    functionality (LP: #1793841)
    - kvm: [Config] enable IP_SET_* modules

  [ Ubuntu: 4.15.0-37.40 ]

  * linux: 4.15.0-37.40 -proposed tracker (LP: #1795564)
  * hns3: enable ethtool rx-vlan-filter on supported hw (LP: #1793394)
    - net: hns3: Add vlan filter setting by ethtool command -K
  * hns3: Modifying channel parameters will reset ring parameters back to
    defaults (LP: #1793404)
    - net: hns3: Fix desc num set to default when setting channel
  * hisi_sas: Add SATA FIX check for v3 hw (LP: #1794151)
    - scsi: hisi_sas: Add SATA FIS check for v3 hw
  * Fix potential corruption using SAS controller on HiSilicon arm64 boards
    (LP: #1794156)
    - scsi: hisi_sas: add memory barrier in task delivery function
  * hisi_sas: Reduce unnecessary spin lock contention (LP: #1794165)
    - scsi: hisi_sas: Tidy hisi_sas_task_prep()
  * Add functional level reset support for the SAS controller on HiSilicon D06
    systems (LP: #1794166)
    - scsi: hisi_sas: tidy host controller reset function a bit
    - scsi: hisi_sas: relocate some common code for v3 hw
    - scsi: hisi_sas: Implement handlers of PCIe FLR for v3 hw
  * HiSilicon SAS controller doesn't recover from PHY STP link timeout
    (LP: #1794172)
    - scsi: hisi_sas: tidy channel interrupt handler for v3 hw
    - scsi: hisi_sas: Fix the failure of recovering PHY from STP link timeout
  * getxattr: always handle namespaced attributes (LP: #1789746)
    - getxattr: use correct xattr length
  * Fix unusable NVIDIA GPU after S3 (LP: #1793338)
    - PCI: Reprogram bridge prefetch registers on resume
  * Fails to boot under Xen PV: BUG: unable to handle kernel paging request at
    edc21fd9 (LP: #1789118)
    - x86/EISA: Don't probe EISA bus for Xen PV guests
  * qeth: use vzalloc for QUERY OAT buffer (LP: #1793086)
    - s390/qeth: use vzalloc for QUERY OAT buffer
  * SRU: Enable middle button of touchpad on ThinkPad P72 (LP: #1793463)
    - Input: elantech - enable middle button of touchpad on ThinkPad P72
  * Dell new AIO requires a new uart backlight driver (LP: #1727235)
    - SAUCE: platform/x86: dell-uart-backlight: new backlight driver for DELL AIO
    - updateconfigs for Dell UART backlight driver
  * [Ubuntu] s390/crypto: Fix return code checking in cbc_paes_crypt.
    (LP: #1794294)
    - s390/crypto: Fix return code ch...