ntpdate run on boot whether or not network configured yet

[Travis Newman]

Dell Latitude D410.

At bootup, the time syncing fails. The rest of the network works fine, but this
just won't work. I do have wireless and wired internet on the laptop, and I know
the wired network is set up as the default, so I don't think that's the issue.

[Ian Jackson (ijackson)]

This works for me with ntpdate 1:4.2.0a+stable-8ubuntu1, which is the current
version in breezy. Which version of ntpdate do you have installed ?

Is there a firewall between your machine and the internet ? If so then
obviously you will have to make sure that it is not preventing the relevant packets.

If the problem persists, please provide the output of the following commands,
run as root:

sh -x /etc/init.d/ntpdate start

strace -tt /usr/sbin/ntpdate -b -s -u ntp.ubuntulinux.org

[Travis Newman (tnewman)]

OK, doing it manually works, it's just at bootup that it won't work. It says
"temporary failure in name resolution"

When I do it manually it works fine.
Is it a problem with the network not starting prior to ntpdate?

[Scott James Remnant (Canonical) (canonical-scott)]

This happens when the network takes a little while to come up, it's especially
common on 802.11g wireless cards which need firmware uploaded to them (so takes
a few extra seconds). As this all happens in the background, it's possible for
the network card to not yet be ready by the time the ntpdate init script runs.

For dapper I'd like to move the ntpdate script into if-up.d so it happens when a
network interface comes up, rather than on boot.

[Travis Newman (tnewman)]

Funny thing is, I've totally disabled the wireless card and I still have this
problem.

I should also note, I didn't have this problem on Hoary.

[Hidde Brugmans (hcbrugmans-deactivatedaccount)]

I have this on my onboard wired ethernet controller too.
network comes up just fine, but ntp usually fails.
Not always tho.

[Ian Jackson (ijackson)]

Please provide the information I asked for !

Which version of ntpdate do you have installed ?

Is there a firewall between your machine and the internet ? If so then
obviously you will have to make sure that it is not preventing the relevant packets.

If the problem persists, please provide the output of the following commands,
run as root:

sh -x /etc/init.d/ntpdate start

strace -tt /usr/sbin/ntpdate -b -s -u ntp.ubuntulinux.org

[Hidde Brugmans (hcbrugmans-deactivatedaccount)]

for me:

hidde@system:~$ apt-cache show ntpdate |grep Version:
Version: 1:4.2.0a+stable-8ubuntu1

root@system:~# sh -x /etc/init.d/ntpdate start
+ PATH=/sbin:/bin
+ test -f /usr/sbin/ntpdate
+ test -f /etc/default/ntpdate
+ . /etc/default/ntpdate
++ NTPSERVERS=ntp.ubuntulinux.org
++ NTPOPTIONS=-u
+ test -n ntp.ubuntulinux.org
+ . /lib/lsb/init-functions
+ case "$1" in
+ log_begin_msg 'Synchronizing clock to ntp.ubuntulinux.org...'
+ '[' -z 'Synchronizing clock to ntp.ubuntulinux.org...' ']'
+ type usplash_write
+ usplash_write 'TEXT Synchronizing clock to ntp.ubuntulinux.org...'
+ echo ' * Synchronizing clock to ntp.ubuntulinux.org...'
 * Synchronizing clock to ntp.ubuntulinux.org...
+ /usr/sbin/ntpdate -b -s -u ntp.ubuntulinux.org
+ log_end_msg 0
+ '[' -z 0 ']'
+ type usplash_write
+ '[' 0 -eq 0 ']'
+ usplash_write 'SUCCESS ok'
+ TPUT=/usr/bin/tput
+ EXPR=/usr/bin/expr
+ '[' -x /usr/bin/tput ']'
+ '[' -x /usr/bin/expr ']'
+ /usr/bin/tput hpa 60
++ /usr/bin/tput cols
+ COLS=129
+ '[' -n 129 ']'
++ /usr/bin/expr 129 - 7
+ COL=122
++ /us'/bin/tput cuu1
++ /usr/bin/tput hpa 122
+ END='
                                          '
++ /usr/bin/tput hpa 0
' START='
++ /usr/bin/tput setaf 1
+ RED=''
++ /usr/bin/tput op
+ NORMAL=''
+ '[' 0 -eq 0 ']'
                                          [ ok ]'+ return 0
+ exit 0
root@system:~#

root@system:~# strace -o strace.txt -tt /usr/sbin/ntpdate -b -s -u
ntp.ubuntulinux.org
root@system:~#

uploading strace

[Hidde Brugmans (hcbrugmans-deactivatedaccount)]

Created an attachment (id=3628)
strace of ntpdate, as requested

[Scott James Remnant (Canonical) (canonical-scott)]

Like I said, it's not _just_ wireless cards -- that's just one set it happens to
be common on.

There's a race between the network card being brought up, and dhcp completing,
and ntpdate running.

S:S40hotplug
  network card detected
  device renamed (if necessary)
  ------------------------------------------> wait for networking to come up
S:S40ifrename |
  other devices renamed |
S:S40networking |
  networking comes up |
S:S41hotplug-net \_/
  waiting network plug events released -----> begin configuration of network cards
S:S45mountnfs.sh |
  NFS filesystems mounted (if any) \_/
S:S48console-screen.sh Bring device up, perform any necessary
  Console initialised configuration such as wireless essid.
S:S50alsa-utils |
  Sound cards initialised \_/
S:S50hwclock.sh Begin DHCP configuration
  Hardware clock set |
**********************************************************|*************************
S:S51ntpdate \_/
  Time set via NTP DHCP complete, network card ready
for use

If the time to configure the card and obtain configuration from the DHCP server
is longer than the time the parallel boot sequence takes to reach S:S51ntpdate,
ntpdate will fail.

This is why it's intermittent; sometimes one side wins, sometimes the other side
does.

The reason this worked in warty/hoary is that the network cards weren't
configured in parallel like this; and instead the boot sequence waited until
each was ready until carrying on.

[mon (javiermon-deactivatedaccount)]

(In reply to comment #9)
> Like I said, it's not _just_ wireless cards -- that's just one set it happens to
> be common on.
>
> There's a race between the network card being brought up, and dhcp completing,
> and ntpdate running.
>
> S:S40hotplug
> network card detected
> device renamed (if necessary)
> ------------------------------------------> wait for networking to come up
> S:S40ifrename |
> other devices renamed |
> S:S40networking |
> networking comes up |
> S:S41hotplug-net \_/
> waiting network plug events released -----> begin configuration of network cards
> S:S45mountnfs.sh |
> NFS filesystems mounted (if any) \_/
> S:S48console-screen.sh Bring device up, perform any
necessary
> Console initialised configuration such as wireless
essid.
> S:S50alsa-utils |
> Sound cards initialised \_/
> S:S50hwclock.sh Begin DHCP configuration
> Hardware clock set |
>
**********************************************************|*************************
> S:S51ntpdate \_/
> Time set via NTP DHCP complete, network card ready
> for use
>
>
> If the time to configure the card and obtain configuration from the DHCP server
> is longer than the time the parallel boot sequence takes to reach S:S51ntpdate,
> ntpdate will fail.
>
> This is why it's intermittent; sometimes one side wins, sometimes the other side
> does.
>
> The reason this worked in warty/hoary is that the network cards weren't
> configured in parallel like this; and instead the boot sequence waited until
> each was ready until carrying on.

I have the same problem here. If someone needs info,
I'll be glad to help

[Scott James Remnant (Canonical) (canonical-scott)]

I'll steal this off you Ian, as it's part of the startup changed I'll be
planning at UBZ.

[Scott James Remnant (Canonical) (canonical-scott)]

*** Bug 22526 has been marked as a duplicate of this bug. ***

[Matt Zimmerman (mdz)]

Please keep this open; duplicates are being reported.

Is there anything safe we can do about this for Breezy, apart from having it
fail silently?

[Scott James Remnant (Canonical) (canonical-scott)]

Nothing for breezy -- it's a genuine race condition; about the most we could do,
as you said, is hide the error.

I'm attacking startup with lots of weaponry for dapper, so we can actually have
it all 100% reliable and efficient.

[Scott James Remnant (Canonical) (canonical-scott)]

*** Bug 22750 has been marked as a duplicate of this bug. ***

[Scott James Remnant (Canonical) (canonical-scott)]

Will be addressed by moving things like this out of the main boot sequence and
instead to a "network up" event.

[Scott James Remnant (Canonical) (canonical-scott)]

*** Bug 22940 has been marked as a duplicate of this bug. ***

[Ricardo Pérez López (ricardo)]

I have a solution (works for me):

In Hoary, all ok. But in Breezy, sometimes the sync clock fails.

The solution:

To add the following line into /etc/network/interfaces:

  auto eth1

That's all! eth1 is the I use to connect to Internet.

By now, the problem is solved. I've tried several times, and now the clock
synchronizes correctly.

Hope this helps.

[Ricardo Pérez López (ricardo)]

Confirmed. This solution works perfectly for me (I tried to boot many many times :))

I think the "auto ..." line must be in the /etc/network/interfaces file by
default, maybe introduced during installation by the installer.

For the Hoary -> Breezy update, there could be a package which does this change
in e/n/i.

Ricardo.

(In reply to comment #18)
> I have a solution (works for me):
>
> In Hoary, all ok. But in Breezy, sometimes the sync clock fails.
>
> The solution:
>
> To add the following line into /etc/network/interfaces:
>
> auto eth1
>
> That's all! eth1 is the I use to connect to Internet.
>
> By now, the problem is solved. I've tried several times, and now the clock
> synchronizes correctly.
>
> Hope this helps.

[Scott James Remnant (Canonical) (canonical-scott)]

In theory it should not be there by default, adding the auto line (and also
removing the hotplug map line) means that your boot sequence will wait for the
device to be configured -- for laptops with a wired and wireless card this can
add a minute or two to the boot sequence,

[Ricardo Pérez López (ricardo)]

(In reply to comment #20)
> In theory it should not be there by default, adding the auto line (and also
> removing the hotplug map line) means that your boot sequence will wait for the
> device to be configured -- for laptops with a wired and wireless card this can
> add a minute or two to the boot sequence,

I have the auto line, but I didn't remove the hotplug map line.

Here's my /etc/network/interfaces:

------- BEGINS HERE
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

# The loopback network interface
auto lo
iface lo inet loopback

# This is a list of hotpluggable network interfaces.
# They will be activated automatically by the hotplug subsystem.
mapping hotplug
        script grep
        map eth1

# The primary network interface
iface eth1 inet dhcp

auto eth1
------- ENDS HERE

[Scott James Remnant (Canonical) (canonical-scott)]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Format: 1.7
Date: Tue, 4 Oct 2005 09:10:48 +0100
Source: hotplug
Binary: hotplug hotplug-udeb
Architecture: source
Version: 0.0.20040329-22ubuntu13
Distribution: breezy
Urgency: low
Maintainer: Fumitoshi UKAI <email address hidden>
Changed-By: Scott James Remnant <email address hidden>
Description:
 hotplug - Linux Hotplug Scripts
 hotplug-udeb - Linux Hotplug Scripts
Changes:
 hotplug (0.0.20040329-22ubuntu13) breezy; urgency=low
 .
   * Wait for at least one hotplugged network interface to complete
     before carrying on the boot sequence; this provides a temporary solution
     to the problem of following services (ntpdate, nfs mounting, pppoeconf,
     etc.) requiring a network card that may not have finished DHCP in the
     completely parallel system. (Ubuntu: #14729, #16720, #16774)
Files:
 860a974767acca773974a9f593a02d69 695 admin standard
hotplug_0.0.20040329-22ubuntu13.dsc
 2b973b08290c0480e89fb4567fba441c 57486 admin standard
hotplug_0.0.20040329-22ubuntu13.diff.gz
Package-Type: udeb

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFDQj3kSnQiFMl4yK4RAlqhAJ0UuHMzWC4a6lZF/8Z687thX34+ewCaAmST
pdhJwFVrgn1znwkFITN6GWI=
=SKQY
-----END PGP SIGNATURE-----

[Matt Zimmerman (mdz)]

*** Bug 23234 has been marked as a duplicate of this bug. ***

[Andreas Schildbach (schildbach)]

I just upgraded to the latest hotplug (0.0.20040329-22ubuntu13), but the issue
is still present.

Could please someone re-open the bug?

[Scott James Remnant (Canonical) (canonical-scott)]

(In reply to comment #24)
> I just upgraded to the latest hotplug (0.0.20040329-22ubuntu13), but the issue
> is still present.
>
> Could please someone re-open the bug?
>

Do you see "Waiting for network interface to come up" on boot?

[Andreas Schildbach (schildbach)]

Yes, I do see this this message. And guess what, on this boot it worked as
expected. Then I tried once more, and it also worked.

I am 100% sure that I already had the latest hotplug version installed on the
last time synch failure. So it looks like there is still some sort of race
condition.

I'll closely watch and will let you know if it happens again.

[yannick (yannick-torres)]

Please, let this bug open.
My bug report has been marked as duplicate of this bug but my problem is always present.

See : http://bugzilla.ubuntu.com/show_bug.cgi?id=16551

[Andreas Schildbach (schildbach)]

I had about 5 successful boots, and on the last boot it failed again.

Of course, it could also be just what it says: a temporary internet problem (out
of control for Ubuntu). But then again, I can't remember getting this error in
my Hoary installation.

Is there some way to determine why it fails?

[Ricardo Pérez López (ricardo)]

(In reply to comment #28)
> I had about 5 successful boots, and on the last boot it failed again.
>
> Of course, it could also be just what it says: a temporary internet problem (out
> of control for Ubuntu). But then again, I can't remember getting this error in
> my Hoary installation.
>
> Is there some way to determine why it fails?
>

Did you tried putting the "auto ..." line into /etc/network/interfaces?

That solves my problem.

[Scott James Remnant (Canonical) (canonical-scott)]

(In reply to comment #29)
> Did you tried putting the "auto ..." line into /etc/network/interfaces?
>
> That solves my problem.
>

Please stop asking people to do this, this isn't a solution to the problem, it
just hides it. If everyone did this, we wouldn't be able to fix the real bug!

[Scott James Remnant (Canonical) (canonical-scott)]

(In reply to comment #27)
> Please, let this bug open.
> My bug report has been marked as duplicate of this bug but my problem is
always present.
>
> See : http://bugzilla.ubuntu.com/show_bug.cgi?id=16551
>
As noted in that bug, it wasn't fixed by this as it's an init order problem and
not the race condition.

[Scott James Remnant (Canonical) (canonical-scott)]

(In reply to comment #28)
> I had about 5 successful boots, and on the last boot it failed again.
>
Could you attach your /etc/network/interfaces file.

[Ricardo Pérez López (ricardo)]

(In reply to comment #30)
> (In reply to comment #29)
> > Did you tried putting the "auto ..." line into /etc/network/interfaces?
> >
> > That solves my problem.
> >
>
> Please stop asking people to do this, this isn't a solution to the problem, it
> just hides it. If everyone did this, we wouldn't be able to fix the real bug!

Sorry, I only want to help :) But I'm with you.

I want to note that the "auto ..." line is inserted automatically by
network-admin if you use that tool to configure your network. This is why I have
the "auto ..." line in my /etc/network/interfaces. I don't know if this is a
network-admin issue.

[Andreas Schildbach (schildbach)]

(In reply to comment #32)

> Could you attach your /etc/network/interfaces file.

Certainly. Is it important that the error occured on the same boot?

[Ricardo Pérez López (ricardo)]

If I remove the "auto ..." line from /etc/network/interfaces, the problem
appears again.

I can see the "Configuring network interfaces" and "Waiting for network
interface to come up" messages, all OK, but the "Synchronizing clock..." failed
(sometimes).

[Scott James Remnant (Canonical) (canonical-scott)]

(In reply to comment #35)
> If I remove the "auto ..." line from /etc/network/interfaces, the problem
> appears again.
>
> I can see the "Configuring network interfaces" and "Waiting for network
> interface to come up" messages, all OK, but the "Synchronizing clock..." failed
> (sometimes).
>

Could you add the following lines to the top (below any comments) of
/etc/init.d/hotplug-net, /etc/hotplug/net.agent and /etc/hotplug/net.ifup

----8<--------8<--------8<--------8<----
exec 2>>/tmp/network.log
date 1>&2
echo "$0" "$@" 1>&2
set -x
---->8-------->8-------->8-------->8----

Then do a reboot, if it fails attach /tmp/network.log

Thanks

[Ricardo Pérez López (ricardo)]

(In reply to comment #36)
> (In reply to comment #35)
> > If I remove the "auto ..." line from /etc/network/interfaces, the problem
> > appears again.
> >
> > I can see the "Configuring network interfaces" and "Waiting for network
> > interface to come up" messages, all OK, but the "Synchronizing clock..." failed
> > (sometimes).
> >
>
> Could you add the following lines to the top (below any comments) of
> /etc/init.d/hotplug-net, /etc/hotplug/net.agent and /etc/hotplug/net.ifup
>
> ----8<--------8<--------8<--------8<----
> exec 2>>/tmp/network.log
> date 1>&2
> echo "$0" "$@" 1>&2
> set -x
> ---->8-------->8-------->8-------->8----
>
> Then do a reboot, if it fails attach /tmp/network.log
>
> Thanks

Mmmm... Rather strange... I added the above lines in the three files you tell
me, I removed the "auto ..." line from e/n/i, I booted several (seven or eight)
times, and I'm unable to reproduce the bug :/

It could be casualness, or maybe the problem is now solved (but I don't know
why... perhaps because of an update...)

Can you give me one more day to study this in detail, and to try to reproduce
the bug?

Thanks a lot.

[Andreas Schildbach (schildbach)]

It just happened again (on a fresh install of 5.10RC + apt-get upgrade), and my
/etc/network/interfaces reads as follows:

# The loopback network interface
auto lo
iface lo inet loopback

# This is a list of hotpluggable network interfaces.
# They will be activated automatically by the hotplug subsystem.
mapping hotplug
        script grep
        map eth1

# The primary network interface
iface eth1 inet dhcp
        # wireless-* options are implemented by the wireless-tools package
        wireless-mode managed
        wireless-essid xxx
        wireless-key1 xxx

[Scott James Remnant (Canonical) (canonical-scott)]

(In reply to comment #38)
> It just happened again (on a fresh install of 5.10RC + apt-get upgrade), and my
> /etc/network/interfaces reads as follows:
>
Could you add the same lines (exec/set/etc.) as above and try again ...

[Scott James Remnant (Canonical) (canonical-scott)]

(In reply to comment #37)
> Mmmm... Rather strange... I added the above lines in the three files you tell
> me, I removed the "auto ..." line from e/n/i, I booted several (seven or eight)
> times, and I'm unable to reproduce the bug :/
>
The effect of the "auto" lines is that your network cards will be configured in
both S:S40networking and in parallel with that -- this could mean that your
network is configured, then deconfigured, which would explain what you saw with
those auto lines in place. It would also be temperamental.

[Ricardo Pérez López (ricardo)]

I'd got a "failed" again. It seems that the ntp sync failed 1% times (one time
over one hundred, or even more).

[Ricardo Pérez López (ricardo)]

I still have this bug in Breezy. Is there a work in Dapper to try to fix it?