Comment 85 for bug 1768976

Each CPU can have a different request into the PLL (phase locked loop), but the highest one wins, and their vote does not count if they are in an idle state deeper than C1. You can observe this manually by reading the pstate request and granted MSRs directly (requires msr-tools, and the msr module must already be loaded). Example (requested, granted, 100% load on CPU 3):

root@s15:/home/doug/temp-git-git# rdmsr --bitfield 15:8 -d -a 0x199
16
16
16
38
16
16
16
16
root@s15:/home/doug/temp-git-git# rdmsr --bitfield 15:8 -d -a 0x198
36
35
36
36
36
36
36
36

Where 38 is the max and 16 in the min for my i7-2600K.
Why only 36 granted? because actually I guess three cores are active at the time of the sample. From turbostat:

cpu0: MSR_TURBO_RATIO_LIMIT: 0x23242526
35 * 100.0 = 3500.0 MHz max turbo 4 active cores
36 * 100.0 = 3600.0 MHz max turbo 3 active cores
37 * 100.0 = 3700.0 MHz max turbo 2 active cores
38 * 100.0 = 3800.0 MHz max turbo 1 active cores

Note: I ran the above as root because there is a lot over overhead running it under "sudo", which tends to result in another CPU requesting a high pstate by the time the sample is actually taken.

Now, the different reported CPU frequencies, for your example where there is no dominant load, is another story. While there is only one PLL, your mostly idle CPUs would be "awake" at different times, with the PLL ramping up and down based on which incoming votes and requests are being used.

Hope this helps, and apologies for this digression on this bug report.