Results on Intel machine

Hardware configs:
Dell PowerEdge R730xd
Intel(R) Xeon(R) CPU E5-2650 v3 @ 2.30GHz
40 CPUs
188G RAM, numa nodes: 2

Software configs:
OS: ubuntu 20.04
Official kernel: 5.15 hwe (5.15.0-86.96~20.04.1)
Test kernel: 5.15 hwe (5.15.0-86.96~20.04.1+test20231013b0)
https://launchpad.net/~gerald-yang-tw/+archive/ubuntu/focal-no-hz-full

Test case 1, without NO_HZ_FULL built-in (default ubuntu kernel config):
Run test program 4 times without taskset
tail -n 2 log/notaskset.*
==> log/notaskset.1 <==
total 49116169085 nsec
avg 491 nsec

==> log/notaskset.2 <==
total 47852147979 nsec
avg 478 nsec

==> log/notaskset.3 <==
total 49077846508 nsec
avg 490 nsec

==> log/notaskset.4 <==
total 49037126328 nsec
avg 490 nsec

Run test program 4 times with taskset to CPU 4
tail -n 2 log/taskset.*  
==> log/taskset.1 <==
total 48534105655 nsec
avg 485 nsec

==> log/taskset.2 <==
total 48220818730 nsec
avg 482 nsec

==> log/taskset.3 <==
total 48496349690 nsec
avg 484 nsec

==> log/taskset.4 <==
total 48224935123 nsec
avg 482 nsec

Test case 2, with NO_HZ_FULL built-in but not activate in kernel cmdline:
Run test program 4 times without taskset
tail -n 2 nohz-log/notaskset.*
==> nohz-log/notaskset.1 <==
total 48533643569 nsec
avg 485 nsec

==> nohz-log/notaskset.2 <==
total 47933581377 nsec
avg 479 nsec

==> nohz-log/notaskset.3 <==
total 49396311930 nsec
avg 493 nsec

==> nohz-log/notaskset.4 <==
total 48812288206 nsec
avg 488 nsec

Run test program 4 times with taskset to CPU 4
tail -n 2 nohz-log/taskset.*
==> nohz-log/taskset.1 <==
total 48929140711 nsec
avg 489 nsec

==> nohz-log/taskset.2 <==
total 48231661796 nsec
avg 482 nsec

==> nohz-log/taskset.3 <==
total 48482539803 nsec
avg 484 nsec

==> nohz-log/taskset.4 <==
total 48272541984 nsec
avg 482 nsec

Test case 3, with NO_HZ_FULL built-in, activate nohz_full in kernel cmdline:
cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-5.15.0-86-generic root=UUID=69036292-bdc0-4904-8724-974723f1095a ro isolcpus=2-19,22-39 nohz_full=2-19,22-39 rcu_nocbs=2-19,22-39

Run test program 4 times without taskset
tail -n 2 nohz-activate-log/notaskset.*
==> nohz-activate-log/notaskset.1 <==
total 52088354594 nsec
avg 520 nsec

==> nohz-activate-log/notaskset.2 <==
total 49226221648 nsec
avg 492 nsec

==> nohz-activate-log/notaskset.3 <==
total 51462517639 nsec
avg 514 nsec

==> nohz-activate-log/notaskset.4 <==
total 51516303613 nsec
avg 515 nsec

Run test program 4 times with taskset to CPU 4
tail -n 2 nohz-activate-log/taskset.*  
==> nohz-activate-log/taskset.1 <==
total 56753345940 nsec
avg 567 nsec

==> nohz-activate-log/taskset.2 <==
total 55720022538 nsec
avg 557 nsec

==> nohz-activate-log/taskset.3 <==
total 55701214354 nsec
avg 557 nsec

==> nohz-activate-log/taskset.4 <==
total 55740784595 nsec
avg 557 nsec

Test case 4, with NO_HZ_FULL built-in, activate nohz_full in kernel cmdline, but run on non-activate CPU:
Run test program on non-activate nohz_full CPU 20
tail -n 2 nohz-activate-off-log/*
==> nohz-activate-off-log/taskset.1 <==
total 49686932587 nsec
avg 496 nsec

==> nohz-activate-off-log/taskset.2 <==
total 49141560622 nsec
avg 491 nsec

==> nohz-activate-off-log/taskset.3 <==
total 49072045557 nsec
avg 490 nsec

==> nohz-activate-off-log/taskset.4 <==
total 49071430182 nsec
avg 490 nsec