zfs-dkms fails to build on arm64 with PREEMPTION and BLK_CGROUP

[Juerg Haefliger]

[Impact]

On arm64 with PREEMPTION and BLK_CGROUP enabled, preempt_schedule_notrace is being pulled in which is a GPL-only function so the DKMS build fails:

DKMS make.log for zfs-0.8.3 for kernel 5.4.0-1015-raspi (aarch64)
Tue Aug 18 06:30:17 UTC 2020
./scripts/make_gitrev.sh
/bin/bash: ./scripts/make_gitrev.sh: No such file or directory
make: [Makefile:1512: gitrev] Error 127 (ignored)
make all-recursive
make[1]: Entering directory '/var/lib/dkms/zfs/0.8.3/build'
Making all in module
make[2]: Entering directory '/var/lib/dkms/zfs/0.8.3/build/module'
list='icp lua'; for targetdir in $list; do \
        make -C $targetdir; \
done
make[3]: Entering directory '/var/lib/dkms/zfs/0.8.3/build/module/icp'
mkdir -p api core spi io os algs algs/aes algs/edonr algs/modes algs/sha1 algs/sha2 algs/skein asm-x86_64 asm-x86_64/aes asm-x86_64/modes asm-x86_64/sha1 asm-x86_64/sha2 asm-i386 asm-generic
make[3]: Leaving directory '/var/lib/dkms/zfs/0.8.3/build/module/icp'
make[3]: Entering directory '/var/lib/dkms/zfs/0.8.3/build/module/lua'
mkdir -p setjmp
make[3]: Leaving directory '/var/lib/dkms/zfs/0.8.3/build/module/lua'
make -C /lib/modules/5.4.0-1015-raspi/build M=`pwd` CONFIG_ZFS=m modules
make[3]: Entering directory '/usr/src/linux-headers-5.4.0-1015-raspi'
  CC [M] /var/lib/dkms/zfs/0.8.3/build/module/avl/avl.o
  CC [M] /var/lib/dkms/zfs/0.8.3/build/module/nvpair/nvpair.o

<SNIP>

  CC [M] /var/lib/dkms/zfs/0.8.3/build/module/zfs/qat_compress.o
  CC [M] /var/lib/dkms/zfs/0.8.3/build/module/zfs/qat_crypt.o
  CC [M] /var/lib/dkms/zfs/0.8.3/build/module/zfs/vdev_raidz_math_aarch64_neon.o
  CC [M] /var/lib/dkms/zfs/0.8.3/build/module/zfs/vdev_raidz_math_aarch64_neonx2.o
  LD [M] /var/lib/dkms/zfs/0.8.3/build/module/zfs/zfs.o
  Building modules, stage 2.
  MODPOST 8 modules
FATAL: modpost: GPL-incompatible module zfs.ko uses GPL-only symbol 'preempt_schedule_notrace'
make[4]: *** [scripts/Makefile.modpost:94: __modpost] Error 1
make[3]: *** [Makefile:1647: modules] Error 2
make[3]: Leaving directory '/usr/src/linux-headers-5.4.0-1015-raspi'
make[2]: *** [Makefile:30: modules] Error 2
make[2]: Leaving directory '/var/lib/dkms/zfs/0.8.3/build/module'
make[1]: *** [Makefile:807: all-recursive] Error 1
make[1]: Leaving directory '/var/lib/dkms/zfs/0.8.3/build'
make: *** [Makefile:676: all] Error 2

Note that Ubuntu currently doesn't support/provide a preemptible arm64 kernel so this hasn't been a problem so far.

[Test Case]

Install zfs-dkms on an arm64 system that has a PREEMPT kernel installed.

[Fix]

Redefine preempt_schedule_notrace() to preempt_schedule() in the ZFS module (only for arm64 and if PREEMPTION and BLK_CGROUP are both enabled). preempt_schedule() is not a GPL symbol.

[Regression Potential]

Per comment in the kernel source, preempt_schedule_notrace() is required when tracing is enabled. As long as tracing is not enabled, this 'fix' should not cause any issues. Problems due to this change could show up as kernel splats like:

     ===============================
     [ INFO: suspicious RCU usage. ]
     3.10.0-rc2+ #1 Not tainted
     -------------------------------
     include/linux/rcupdate.h:771 rcu_read_lock() used illegally while idle!
     other info that might help us debug this:
     RCU used illegally from idle CPU! rcu_scheduler_active = 1, debug_locks = 0
     RCU used illegally from extended quiescent state!
     2 locks held by cc1/63645:
      #0: (&rq->lock){-.-.-.}, at: [<ffffffff816b39fd>] __schedule+0xed/0x9b0
      #1: (rcu_read_lock){.+.+..}, at: [<ffffffff8109d645>] cpuacct_charge+0x5/0x1f0

     CPU: 1 PID: 63645 Comm: cc1 Not tainted 3.10.0-rc2+ #1 [loadavg: 40.57 27.55 13.39 25/277 64369]
     Hardware name: Gigabyte Technology Co., Ltd. GA-MA78GM-S2H/GA-MA78GM-S2H, BIOS F12a 04/23/2010
      0000000000000000 ffff88010f78fcf8 ffffffff816ae383 ffff88010f78fd28
      ffffffff810b698d ffff88011c092548 000000000023d073 ffff88011c092500
      0000000000000001 ffff88010f78fd60 ffffffff8109d7c5 ffffffff8109d645
     Call Trace:
      [<ffffffff816ae383>] dump_stack+0x19/0x1b
      [<ffffffff810b698d>] lockdep_rcu_suspicious+0xfd/0x130
      [<ffffffff8109d7c5>] cpuacct_charge+0x185/0x1f0
      [<ffffffff8109d645>] ? cpuacct_charge+0x5/0x1f0
      [<ffffffff8108dffc>] update_curr+0xec/0x240
      [<ffffffff8108f528>] put_prev_task_fair+0x228/0x480
      [<ffffffff816b3a71>] __schedule+0x161/0x9b0
      [<ffffffff816b4721>] preempt_schedule+0x51/0x80
      [<ffffffff816b4800>] ? __cond_resched_softirq+0x60/0x60
      [<ffffffff816b6824>] ? retint_careful+0x12/0x2e
      [<ffffffff810ff3cc>] ftrace_ops_control_func+0x1dc/0x210
      [<ffffffff816be280>] ftrace_call+0x5/0x2f
      [<ffffffff816b681d>] ? retint_careful+0xb/0x2e
      [<ffffffff816b4805>] ? schedule_user+0x5/0x70
      [<ffffffff816b4805>] ? schedule_user+0x5/0x70
      [<ffffffff816b6824>] ? retint_careful+0x12/0x2e
     ------------[ cut here ]------------

[Juerg Haefliger (juergh)]

Test packages: https://launchpad.net/~juergh/+archive/ubuntu/raspi

[Launchpad Janitor (janitor)]

This bug was fixed in the package zfs-linux - 0.8.4-1ubuntu11

---------------
zfs-linux (0.8.4-1ubuntu11) groovy; urgency=medium

  * Fix zfs-dkms build on arm64 with PREEMPTION and BLK_CGROUP (LP: #1892001)
    - 4700-Fix-DKMS-build-on-arm64-with-PREEMPTION-and-BLK_CGRO.patch
      preempt_schedule_notrace is GPL-only so redfine it to preempt_schedule
      on arm64 with PREEMPTION and BLK_CGROUP enabled to 'fix' the DKMS
      build failure.

 -- Juerg Haefliger <email address hidden> Tue, 18 Aug 2020 11:10:41 +0200

[Timo Aaltonen (tjaalton)]

Hello Juerg, or anyone else affected,

Accepted zfs-linux into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/zfs-linux/0.8.3-1ubuntu12.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Timo Aaltonen (tjaalton)]

Hello Juerg, or anyone else affected,

Accepted zfs-linux into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/zfs-linux/0.7.5-1ubuntu16.10 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Ubuntu SRU Bot (ubuntu-sru-bot)]

All autopkgtests for the newly accepted zfs-linux (0.7.5-1ubuntu16.10) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

lxd/3.0.3-0ubuntu1~18.04.1 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#zfs-linux

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

[Colin Ian King (colin-king)]

Tested for focal on arm64/amd64 generic and with PREEMPT with ubuntu zfs tests - all OK.

[Colin Ian King (colin-king)]

Tested for bionic on arm64/am64 generic and amd64 with PREEMPT with ubuntu zfs tests - all OK.

[Launchpad Janitor (janitor)]

This bug was fixed in the package zfs-linux - 0.8.3-1ubuntu12.4

---------------
zfs-linux (0.8.3-1ubuntu12.4) focal; urgency=medium

  * Fix zfs-dkms build on arm64 with PREEMPTION and BLK_CGROUP (LP: #1892001)
    - 4700-Fix-DKMS-build-on-arm64-with-PREEMPTION-and-BLK_CGRO.patch
      preempt_schedule_notrace is GPL-only so redfine it to preempt_schedule
      on arm64 with PREEMPTION and BLK_CGROUP enabled to 'fix' the DKMS
      build failure.

 -- Juerg Haefliger <email address hidden> Tue, 18 Aug 2020 11:10:41 +0200

[Łukasz Zemczak (sil2100)]

The verification of the Stable Release Update for zfs-linux has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

[Launchpad Janitor (janitor)]

This bug was fixed in the package zfs-linux - 0.7.5-1ubuntu16.10

---------------
zfs-linux (0.7.5-1ubuntu16.10) bionic; urgency=medium

  * Fix zfs-dkms build on arm64 with PREEMPTION and BLK_CGROUP (LP: #1892001)
    - 4700-Fix-DKMS-build-on-arm64-with-PREEMPTION-and-BLK_CGRO.patch
      preempt_schedule_notrace is GPL-only so redfine it to preempt_schedule
      on arm64 with PREEMPTION and BLK_CGROUP enabled to 'fix' the DKMS
      build failure.

 -- Juerg Haefliger <email address hidden> Tue, 18 Aug 2020 11:10:41 +0200