time never catches up to reality after VM sleep

[Barry Warsaw]

When I run Ubuntu in a VM, then sleep the VM for a little while, I expect time to eventually catch up to reality, because I have "Set the time ... Automatically from the Internet". However, time never does catch up.

I can trigger a catch up by toggling the datetime indicator to "Manually", wait a little bit, then toggle back to "Automatically from the Internet". I may have to do this a couple of times to fix the time.

This tells me that it's not a problem with the VM environment specifically, but a problem with coordinating time over the internet.

ProblemType: Bug
DistroRelease: Ubuntu 13.04
Package: indicator-datetime 12.10.3daily13.03.07-0ubuntu1
ProcVersionSignature: Ubuntu 3.8.0-12.21-generic 3.8.2
Uname: Linux 3.8.0-12-generic x86_64
ApportVersion: 2.9.1-0ubuntu1
Architecture: amd64
Date: Wed Mar 20 11:47:01 2013
EcryptfsInUse: Yes
ExecutablePath: /usr/lib/x86_64-linux-gnu/indicator-datetime-service
InstallationDate: Installed on 2012-12-26 (84 days ago)
InstallationMedia: Ubuntu 13.04 "Raring Ringtail" - Alpha amd64 (20121225)
MarkForUpload: True
SourcePackage: indicator-datetime
UpgradeStatus: No upgrade log present (probably fresh install)

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Charles Kerr (charlesk)]

Hi Barry,

Is this still an issue in Trusty? I rewrote the sleep/skew detection code post-13.10 so if you have a convenient setup I'd love to hear how this goes.

[Barry Warsaw (barry)]

On Mar 06, 2014, at 02:06 AM, Charles Kerr wrote:

>Is this still an issue in Trusty? I rewrote the sleep/skew detection
>code post-13.10 so if you have a convenient setup I'd love to hear how
>this goes.

Hi. It's still a problem in Trusty. I have a fairly fresh and up-to-date
Trusty VM, which I suspended for 30 minutes. Came back this morning (many
hours later) to find that it's still lagging by the sleep amount.

I have ntpdate installed but not ntp. System settings is supposedly set to
set the time "Automatically from the Internet", although it doesn't seem to be
working.

Happy to test anything that would help debug the issue.

[Charles Kerr (charlesk)]

It sounds like you're saying that the entire system's time is off, not just the indicator's display of it.

What happens when you run "date" from the command line? Does that time match the actual time, or the lagging time being displayed by the indicator?

[Barry Warsaw (barry)]

On Mar 06, 2014, at 09:37 PM, Charles Kerr wrote:

>It sounds like you're saying that the entire system's time is off, not
>just the indicator's display of it.

Correct.

>What happens when you run "date" from the command line? Does that time
>match the actual time, or the lagging time being displayed by the
>indicator?

`date` lags too, matching the lagging time in the indicator.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Charles Kerr (charlesk)]

So if the entire system's time is falling out of sync after sleep, this isn't an indicator-datetime bug.

Reassigning to systemd... though I'm not positive this is the right package....

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Martin Pitt (pitti)]

This is first and foremost a QEMU or linux bug (not sure which), it should really update its internal time after suspend. But I suppose ntp could also listen to resume events (perhaps through pm-utils' /usr/lib/pm-utils/sleep.d/ scripts); although this should already be covered by its existing if-up.d script, i. e. as soon as the VM gets back online after resuming /etc/network/if-up.d/ntpdate ought to run. It doesn't in your case?

[Barry Warsaw (barry)]

On Mar 08, 2014, at 04:59 PM, Martin Pitt wrote:

>This is first and foremost a QEMU or linux bug (not sure which), it
>should really update its internal time after suspend. But I suppose ntp
>could also listen to resume events (perhaps through pm-utils' /usr/lib
>/pm-utils/sleep.d/ scripts); although this should already be covered by
>its existing if-up.d script, i. e. as soon as the VM gets back online
>after resuming /etc/network/if-up.d/ntpdate ought to run. It doesn't in
>your case?

Please note that this is a VMware Fusion VM. ntp is not installed, but if you
go to System Settings -> Time & Date, you will see that "Set the time" is set
to "Automatically from the Internet". Shouldn't this be enough to have time
catch up after resume from VM suspend? If not, and ntp is actually required
(as was the case way back when), then I rather think that the System Setting
is misleading. I can handle installing and running ntp, as I used to do, but
I think this is a regression (hard for me to remember exactly).

So, if ntp is required not installed, what effect does the Time & Date dialog
actually have, other than dimming the Time and Date widgets?

[Serge Hallyn (serge-hallyn)]

Hi,

Thanks for filing this bug. I'd like to try to reproduce it but need a few more details. Can you tell us exactly how you set up the vm, how you start it, and how you initiate suspend? Is this all done through virt-manager? You say it is a vmware fusion vm - does that mean you started with a .vmdk of an appliance, converted it to qcow2 or raw, and are using that? If not, what does it mean?

[Barry Warsaw (barry)]

On Mar 11, 2014, at 04:54 AM, Serge Hallyn wrote:

>Thanks for filing this bug. I'd like to try to reproduce it but need a
>few more details. Can you tell us exactly how you set up the vm, how
>you start it, and how you initiate suspend? Is this all done through
>virt-manager? You say it is a vmware fusion vm - does that mean you
>started with a .vmdk of an appliance, converted it to qcow2 or raw, and
>are using that? If not, what does it mean?

This is a Trusty guest running in VMware Fusion 6.0.2 on an OS X 10.9.2 host.
Trusty was installed fresh some time ago and has been rolling updated ever
since.

When you quit Fusion, it suspends the VM. You can of course suspend the VM
explicitly any other time. You can also take a disk snapshot and restore that
snapshot at a later date. All of these exhibit the same symptom - the VM's
time gets behind, sometimes by a long while, and it never catches up.

If you go to System Settings (in Ubuntu of course) -> Time & Date, and look at
"Set time:" then "Automatically from the Internet" is set. What this implies
to me at least, is that the system will keep its time in sync with the network
servers. I would thus expect that when the VM is resumed, it would eventually
catch up either with the "real time".

What I think is happening is that this settings panel actually only has an
effect at system boot time. Can you confirm whether this setting should
periodically sync system time to the internet servers, or whether ntp must be
installed in order to keep time in sync. ntp is *not* currently installed.

I'm perfectly willing to accept that ntp is required in order for a suspended
VM's time to resync with internet server time. If that's the case, then I
think we have a design bug here - the "Automatically from the Internet"
setting is either misleading, or should prompt to install ntp (maybe with
"panic 0" set?). At one point I think this did happen.

However, if this setting is supposed to keep time in sync without ntp, then
there's a functional bug here.

I'm mostly trying to get verification on what expected behavior is.

[Serge Hallyn (serge-hallyn)]

Thanks, Barry. So IIUC this has nothing to do with qemu, so I'm switching it to linux.

Is it safe to assume that other VMs - other Ubuntu releases, or other distros, or windows, do not have this behavior?

[Brad Figg (brad-figg)]

This bug is missing log files that will aid in diagnosing the problem. From a terminal window please run:

apport-collect 1157914

and then change the status of the bug to 'Confirmed'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

[Barry Warsaw (barry)]

On Mar 11, 2014, at 07:56 PM, Serge Hallyn wrote:

>Is it safe to assume that other VMs - other Ubuntu releases, or other
>distros, or windows, do not have this behavior?

I haven't tried other guest OSes. I'll give Debian and Windows 7 a try.

[Neil Wilson (neil-aldur)]

I don't see the issue on Windows XP. Only when running Ubuntu.

[Neil Wilson (neil-aldur)]

Barry,

Can you check you VMWare settings for the virtual machine and check that 'synchronise time' is checked in the Advanced Section.

Also do you have the 'open-vm-tools' installed (i.e. is vmtoolsd running which is what does the time sync to the Host every 60 seconds).

[Barry Warsaw (barry)]

On Mar 11, 2014, at 08:31 PM, Neil Wilson wrote:

>Can you check you VMWare settings for the virtual machine and check that
>'synchronise time' is checked in the Advanced Section.

Yes, that is checked.

>Also do you have the 'open-vm-tools' installed (i.e. is vmtoolsd running
>which is what does the time sync to the Host every 60 seconds).

Ah, I do not have open-vm-tools installed. I recall way back in the past that
installing this caused several display problems, so I removed it and never
noticed a problem. I should re-install it and see if it fixes the issue
without causing other problems.

(I've verified that installing ntp also "fixes" the problem, but you need to
add "tinker panic 0" to /etc/ntp.conf otherwise if your VM is suspended for a
long time, ntp will refuse to synchronize.)

[Neil Wilson (neil-aldur)]

It would be as well to try it out - particularly as the package has been updated recently.

I tried a fresh install of Trusty and when 'vmtoolsd' is running from the 'open-vm-tools' package, the output from 'vmware-toolbox-cmd stat hosttime' and 'date' are the same after a restore from vmware suspend.

The graphic indicator lags, but caught up within about a minute.

Check 'vmware-toolbox-cmd timesync status' and run 'vmware-toolbox-cmd timesync enable' if necessary.

[Barry Warsaw (barry)]

On Mar 12, 2014, at 07:31 AM, Neil Wilson wrote:

>It would be as well to try it out - particularly as the package has been
>updated recently.

Indeed. I installed it, timesync is enabled, removed ntp, and indeed after
about a 20m suspend, once resumed time syncs up again. Since I've not noticed
any adverse effects so far, I'll stick with this solution. Thanks!

(My original question about expected behavior of the system settings panel is
still not answered, but I'll leave that alone for now ;).

[Launchpad Janitor (janitor)]

[Expired for linux (Ubuntu) because there has been no activity for 60 days.]