Timedrift problems with Win7: hpet missing time drift fixups

[Lucas Meneghel Rodrigues]

We've been finding timedrift issues witth Win7 under qemu-kvm on our daily testing

kvm.qemu-kvm-git.smp2.Win7.64.timedrift.with_load FAIL 1 Time drift too large after rest period: 38.63%
kvm.qemu-kvm-git.smp2.Win7.64.timedrift.with_reboot FAIL 1 Time drift too large at iteration 1: 17.77 seconds
kvm.qemu-kvm-git.smp2.Win7.64.timedrift.with_migration FAIL 1 Time drift too large at iteration 2: 3.08 seconds

Steps to reproduce:

timedrift.with_load

1) Log into a guest.
2) Take a time reading from the guest and host.
3) Run load on the guest and host.
4) Take a second time reading.
5) Stop the load and rest for a while.
6) Take a third time reading.
7) If the drift immediately after load is higher than a user-
    specified value (in %), fail.
    If the drift after the rest period is higher than a user-specified value,
    fail.

timedrift.with_migration

1) Log into a guest.
2) Take a time reading from the guest and host.
3) Migrate the guest.
4) Take a second time reading.
5) If the drift (in seconds) is higher than a user specified value, fail.

timedrift.with_reboot

1) Log into a guest.
2) Take a time reading from the guest and host.
3) Reboot the guest.
4) Take a second time reading.
5) If the drift (in seconds) is higher than a user specified value, fail.

This bug is to register those issues and keep an eye on them.

Attached, some logs from the autotest tests executed on the guest

[Anthony Liguori (anthony-codemonkey)]

Does adding -no-hpet make a difference? I assume that Win7 is using hpet by default and we currently don't do any time drift mitigation with hpet.

[Lucas Meneghel Rodrigues (lmr)]

I will try that on a separate test job and will let you know about the results.

[Lucas Meneghel Rodrigues (lmr)]

Indeed -no-hpet made the tests pass. It's still uncertain to me whether this flag is supported across several branches of qemu-kvm, if it's supported in all branches I'm going to update the upstream kvm autotest config file.

[Anthony Liguori (anthony-codemonkey)]

-no-hpet works in every version of qemu/qemu-kvm that has included HPET support. RHEL disables HPET support by default unlike qemu and qemu-kvm.

I've updated the bug priority and title to reflect what the issue is.

We only support edge triggered interrupts with HPET which seems to be what most OSes use anyway. We could potentially use the reset_irq_delivered/get_irq_delivered APIC functions to implement interrupt catch-up but I think it would be better to try to merge Jan's generic IRQ delivered API first.

[Lucas Meneghel Rodrigues (lmr)]

Sent patch http://patchwork.test.kernel.org/patch/2384/ to autotest and will update the autotest server to reflect that option.

[roothorick (8-roothorick-gmail-com)]

Apparently this bug's still alive and kicking.

There's an obvious clock skew problem on Windows 7; in the Date & Time dialog, the clock jumps through seconds visibly too fast.

I also found a case where HPET bugs are causing a real problem: Terraria (dedicated server) seems to be relying on (something that relies on) HPET, and QEMU doesn't get it right. The result is a goofy and aggravating behavior I've nicknamed "Turbo Monsters of Doom" and it makes killing anything tougher than a normal zombie basically impossible.

[roothorick (8-roothorick-gmail-com)]

Forgot to add: Reproduced the above behavior in both 1.5.1 and 1.6.0. Adding -no-hpet to commandline removed both problems (full disclosure: this fix wasn't tested in 1.5.1 but I have no reason to believe behavior would be different.)

[roothorick (8-roothorick-gmail-com)]

Fair enough in itself, but if HPET is known to have problems with
arguably the most popular OS family to use as a guest, why is it
enabled by default?

On Tue, Oct 1, 2013 at 10:56 AM, Gleb Natapov <email address hidden> wrote:
> On Tue, Oct 01, 2013 at 09:34:06AM -0000, Ben A wrote:
>> Apparently this bug's still alive and kicking.
>>
> And no plans to fix it. Do not use hpet with windows guests this buys
> you nothing.
>
>> There's an obvious clock skew problem on Windows 7; in the Date & Time
>> dialog, the clock jumps through seconds visibly too fast.
>>
>> I also found a case where HPET bugs are causing a real problem: Terraria
>> (dedicated server) seems to be relying on (something that relies on)
>> HPET, and QEMU doesn't get it right. The result is a goofy and
>> aggravating behavior I've nicknamed "Turbo Monsters of Doom" and it
>> makes killing anything tougher than a normal zombie basically
>> impossible.
>>
>> --
>> You received this bug notification because you are a member of qemu-
>> devel-ml, which is subscribed to QEMU.
>> https://bugs.launchpad.net/bugs/599958
>>
>> Title:
>> Timedrift problems with Win7: hpet missing time drift fixups
>>
>> Status in QEMU:
>> Confirmed
>>
>> Bug description:
>> We've been finding timedrift issues witth Win7 under qemu-kvm on our
>> daily testing
>>
>> kvm.qemu-kvm-git.smp2.Win7.64.timedrift.with_load FAIL 1 Time drift too large after rest period: 38.63%
>> kvm.qemu-kvm-git.smp2.Win7.64.timedrift.with_reboot FAIL 1 Time drift too large at iteration 1: 17.77 seconds
>> kvm.qemu-kvm-git.smp2.Win7.64.timedrift.with_migration FAIL 1 Time drift too large at iteration 2: 3.08 seconds
>>
>> Steps to reproduce:
>>
>> timedrift.with_load
>>
>> 1) Log into a guest.
>> 2) Take a time reading from the guest and host.
>> 3) Run load on the guest and host.
>> 4) Take a second time reading.
>> 5) Stop the load and rest for a while.
>> 6) Take a third time reading.
>> 7) If the drift immediately after load is higher than a user-
>> specified value (in %), fail.
>> If the drift after the rest period is higher than a user-specified value,
>> fail.
>>
>> timedrift.with_migration
>>
>> 1) Log into a guest.
>> 2) Take a time reading from the guest and host.
>> 3) Migrate the guest.
>> 4) Take a second time reading.
>> 5) If the drift (in seconds) is higher than a user specified value, fail.
>>
>> timedrift.with_reboot
>>
>> 1) Log into a guest.
>> 2) Take a time reading from the guest and host.
>> 3) Reboot the guest.
>> 4) Take a second time reading.
>> 5) If the drift (in seconds) is higher than a user specified value, fail.
>>
>> This bug is to register those issues and keep an eye on them.
>>
>> Attached, some logs from the autotest tests executed on the guest
>>
>> To manage notifications about this bug go to:
>> https://bugs.launchpad.net/qemu/+bug/599958/+subscriptions
>
> --
> Gleb.

[roothorick (8-roothorick-gmail-com)]

Agh, I forgot reply all.

Seems like something that should be changed, no? It would've saved me
a lot of headache if there was a switch e.g.
-optimize-for=[linux,winxp,
win7,etc] that changed the defaults to be
most accomodating to the specified OS as a guest.

On Tue, Oct 1, 2013 at 11:33 AM, Gleb Natapov <email address hidden> wrote:
> On Tue, Oct 01, 2013 at 11:23:07AM -0500, Ben "Root" Anderson wrote:
>> Fair enough in itself, but if HPET is known to have problems with
>> arguably the most popular OS family to use as a guest, why is it
>> enabled by default?
>>
> Arguably :) But QEMU defaults are arguably far from been optimal for any
> guest.
>
>> On Tue, Oct 1, 2013 at 10:56 AM, Gleb Natapov <email address hidden> wrote:
>> > On Tue, Oct 01, 2013 at 09:34:06AM -0000, Ben A wrote:
>> >> Apparently this bug's still alive and kicking.
>> >>
>> > And no plans to fix it. Do not use hpet with windows guests this buys
>> > you nothing.
>> >
>> >> There's an obvious clock skew problem on Windows 7; in the Date & Time
>> >> dialog, the clock jumps through seconds visibly too fast.
>> >>
>> >> I also found a case where HPET bugs are causing a real problem: Terraria
>> >> (dedicated server) seems to be relying on (something that relies on)
>> >> HPET, and QEMU doesn't get it right. The result is a goofy and
>> >> aggravating behavior I've nicknamed "Turbo Monsters of Doom" and it
>> >> makes killing anything tougher than a normal zombie basically
>> >> impossible.
>> >>
>> >> --
>> >> You received this bug notification because you are a member of qemu-
>> >> devel-ml, which is subscribed to QEMU.
>> >> https://bugs.launchpad.net/bugs/599958
>> >>
>> >> Title:
>> >> Timedrift problems with Win7: hpet missing time drift fixups
>> >>
>> >> Status in QEMU:
>> >> Confirmed
>> >>
>> >> Bug description:
>> >> We've been finding timedrift issues witth Win7 under qemu-kvm on our
>> >> daily testing
>> >>
>> >> kvm.qemu-kvm-git.smp2.Win7.64.timedrift.with_load FAIL 1 Time drift too large after rest period: 38.63%
>> >> kvm.qemu-kvm-git.smp2.Win7.64.timedrift.with_reboot FAIL 1 Time drift too large at iteration 1: 17.77 seconds
>> >> kvm.qemu-kvm-git.smp2.Win7.64.timedrift.with_migration FAIL 1 Time drift too large at iteration 2: 3.08 seconds
>> >>
>> >> Steps to reproduce:
>> >>
>> >> timedrift.with_load
>> >>
>> >> 1) Log into a guest.
>> >> 2) Take a time reading from the guest and host.
>> >> 3) Run load on the guest and host.
>> >> 4) Take a second time reading.
>> >> 5) Stop the load and rest for a while.
>> >> 6) Take a third time reading.
>> >> 7) If the drift immediately after load is higher than a user-
>> >> specified value (in %), fail.
>> >> If the drift after the rest period is higher than a user-specified value,
>> >> fail.
>> >>
>> >> timedrift.with_migration
>> >>
>> >> 1) Log into a guest.
>> >> 2) Take a time reading from the guest and host.
>> >> 3) Migrate the guest.
>> >> 4) Take a second time reading.
>> >> 5) If the drift (in seconds) is higher than a user specified value, fail.
>> >>
>> >> timedrift.with_reboot
>> >>
>> >> 1) Log into a guest.
...

[AndCycle (andcycle-launchpad-net)]

I google about an old link talk about this issue can be fixed by not using virtio

http://forum.proxmox.com/archive/index.php/t-5783.html

[Thomas Huth (th-huth)]

Looking through old bug tickets... can this issue still be reproduced with the latest version of QEMU? Or could we close this ticket nowadays?

[Lucas Meneghel Rodrigues (lmr)]

Absolutely, please close it.

[Launchpad Janitor (janitor)]

[Expired for QEMU because there has been no activity for 60 days.]