cpuset support no longer enabled in kernel

[Chris Samuel]

Binary package hint: linux-source-2.6.24

In Gutsy CONFIG_CPUSETS was enabled:

/boot/config-2.6.22-14-generic:CONFIG_CPUSETS=y

allowing you to mount the /dev/cpuset virtual file system with:

mkdir /dev/cpuset
mount -t cpuset - /dev/cpuset

This is no longer the case in Hardy.

The cpuset functionality allows binding of processes to cores and memory nodes on NUMA systems such as AMD Opteron, and can be vital to getting good performance out of things like scientific HPC codes where memory bandwidth is critical.

I found this as I'm one of the people involved in the development of Torque (an open source, PBS
based, batch queuing system that is often used on large HPC clusters) and was about to test the cpuset code that has just arrived in our trunk. Not having cpusets any more makes it a bit hard to see if it works.. :-)

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 2.6.24-5.8

---------------
linux (2.6.24-5.8) hardy; urgency=low

  [Alessio Igor Bogani]

  * rt: Update to 2.6.24-rc8-rt1
  * rt: Update configuration files

  [Amit Kucheria]

  * Asix: fix breakage caused in 2.6.24-rc7
  * Add CONFIG_CPUSETS to server-related flavours
    - LP: #182434

  [Chuck Short]

  * SAUCE: ata: blacklist FUJITSU MHW2160BH PL
    - LP: #175834

  [Kees Cook]

  * AppArmor: updated patch series to upstream SVN 1079.

  [Soren Hansen]

  * Updated configs to enable virtio stuff Ignore: yes

  [Stefan Bader]

  * Enabled CONFIG_BSD_PROCESS_ACCT=y for sparc.
    - LP: #176587
  * Enable CONFIG_AUDITSYSCALL=y.
    - LP: #140784
  * Added CONFIG_AUDIT_SYSCALL=y to custom lpia(compat)
  * Enabled CONFIG_HUGETLBFS=y for i386/server amd64/server and ia64.
  * Lower priority of pnpacpi resource messages to warning level.
    - LP: #159241
  * Fix the messed up message level of pnpacpi parser.

  [Tim Gardner]

  * Start new release, bump ABI to -5
  * Disabled iwlwifi preperatory to moving it to l-u-m.
  * Enabled CONFIG_USB_SERIAL_KEYSPAN
  * Disabled CONFIG_CGROUPS.
  * Virtio config settings for -rt.
  * Re-enable IWLWIFI in the kernel.
  * Fixed -rt saa7134-core.c FTBS

  [Upstream Kernel Changes]

  * Input: Handle EV_PWR type of input caps in input_set_capability.
  * Input: jornada680_kbd - fix default keymap
  * increase PNP_MAX_PORT to 40 from 24
  * sched: fix gcc warnings
  * leds: Fix leds_list_lock locking issues
  * leds: Fix locomo LED driver oops
  * x86: fix asm-x86/byteorder.h for userspace export
  * x86: fix asm-x86/msr.h for user-space export
  * ACPI: EC: Enable boot EC before bus_scan
  * ACPI: Make sysfs interface in ACPI power optional.
  * fix lguest rmmod "bad pgd"
  * slub: provide /proc/slabinfo
  * [POWERPC] Fix build failure on Cell when CONFIG_SPU_FS=y
  * slub: register slabinfo to procfs
  * [SCSI] scsi_sysfs: restore prep_fn when ULD is removed
  * Unify /proc/slabinfo configuration
  * scsi: revert "[SCSI] Get rid of scsi_cmnd->done"
  * restrict reading from /proc/<pid>/maps to those who share ->mm or can
    ptrace pid
  * Fix kernel/ptrace.c compile problem (missing "may_attach()")
  * hwmon: (w83627ehf) Be more careful when changing VID input level
  * NFS: Fix a possible Oops in fs/nfs/super.c
  * NFSv4: Fix circular locking dependency in nfs4_kill_renewd
  * NFS: add newline to kernel warning message in auth_gss code
  * NFSv4: nfs4_open_confirm must not set the open_owner as confirmed on
    error
  * NFSv4: Fix open_to_lock_owner sequenceid allocation...
  * gameport: don't export functions that are static inline
  * Input: spitzkbd - fix suspend key handling
  * Input: pass EV_PWR events to event handlers
  * [ARM] 4735/1: Unbreak pxa25x suspend/resume
  * IB/srp: Fix list corruption/oops on module reload
  * Console is utf-8 by default
  * [IA64] Update Altix BTE error return status patch
  * [IA64] Update Altix nofault code
  * [X25]: Add missing x25_neigh_put
  * [XFRM]: Do not define km_migrate() if !CONFIG_XFRM_MIGRATE
  * [CASSINI]: Fix endianness bug.
  * [CASSINI]: Revert 'dont touch page_count'.
  * [CASSINI]: Program parent Inte...

[Chris Samuel (chris-csamuel)]

Is there any ETA on this package making out to the repositories please ?

Currently I'm seeing:

$ apt-cache policy linux-image
linux-image:
  Installed: 2.6.24.5.6
  Candidate: 2.6.24.5.6
  Version table:
 *** 2.6.24.5.6 0
        500 http://mirrors.uwa.edu.au hardy/main Packages
        500 http://archive.ubuntu.com hardy/main Packages
        100 /var/lib/dpkg/status

[Chris Samuel (chris-csamuel)]

This is still not fixed in 2.6.24.7.7 :-(

$ grep -c CPUSET /boot/config-2.6.24-7-generic
0

$ apt-cache policy linux-image-2.6.24-7-generic
linux-image-2.6.24-7-generic:
  Installed: 2.6.24-7.12
  Candidate: 2.6.24-7.12
  Version table:
 *** 2.6.24-7.12 0
        500 http://mirrors.uwa.edu.au hardy/main Packages
        500 http://archive.ubuntu.com hardy/main Packages
        100 /var/lib/dpkg/status

$ apt-cache policy linux-image
linux-image:
  Installed: 2.6.24.7.7
  Candidate: 2.6.24.7.7
  Version table:
 *** 2.6.24.7.7 0
        500 http://archive.ubuntu.com hardy/main Packages
        100 /var/lib/dpkg/status
     2.6.24.5.6 0
        500 http://mirrors.uwa.edu.au hardy/main Packages

I'd much rather run the Ubuntu kernels rather than my own builds of 2.6.24!

[Tim Gardner (timg-tpi)]

Looks like some Apparmor patches broke CGROUP, upon which CPUSETS is dependent, so it got disabled. I fixed the Apparmor patch and enabled CGROUPS and CPUSETS for the x86/x86_64 server version only. Look for it in the 2.6.24-9 kernel.

[Chris Samuel (chris-csamuel)]

Thanks for the explanation Tim, the work on the server kernel is much appreciated!

I'm a bit disappointed that the standard kernel will lack cpuset support though, looks like I'll need to stick to rolling my own mainline kernels on my desktop systems which I work on cpuset dependent code on to avoid this regression from previous Ubuntu releases.

With more and more cores being availble on modern desktop systems (mine has 4) I believe I can see other use cases such as audio processing where people might appreciate being able to migrate all the system processes onto a single core and have the rest untouched for their user processes.

[Launchpad Janitor (janitor)]

This bug was fixed in the package linux - 2.6.24-9.15

---------------
linux (2.6.24-9.15) hardy; urgency=low

  [Alessio Igor Bogani]

  * rt: Fix FTBS
  * rt: Updated configuration files

  [Tim Gardner]

  * SAUCE: make /dev/kmem a config option
  * SAUCE: x86: introduce /dev/mem restrictions with a config option
  * Fixed CGROUP FTBS caused by AppArmor patch.
  * Enabled CGROUP and CPUSETS for server flavor.
    - LP: #182434

  [Colin King]

  * Turn on /proc/acpi/alarm for x86_64 (amd64)
    - LP: #186297

  [Upstream Kernel Changes]

  * Ubuntu: LatencyTOP infrastructure patch

 -- Tim Gardner <email address hidden> Thu, 14 Feb 2008 13:34:55 -0700