tftpd-hpa fails to start when the NIC is not available at startup time

Bug #1342580 reported by Oleksij Rempel on 2014-07-16
186
This bug affects 41 people
Affects Status Importance Assigned to Milestone
tftp-hpa (Debian)
New
Unknown
tftp-hpa (Ubuntu)
Medium
Unassigned
Trusty
Medium
Unassigned
Xenial
Medium
Unassigned
Yakkety
Medium
Unassigned

Bug Description

[SRU justification]
This fix is required in order to make sure that the tftpd daemon will be started even if the Network Interface is not yet available.

[Impact]
The daemon fails to start if the NIC is not enable at startup time

[Fix]
Remove IPv6's [::] portion of the address. The blank value is adequate as an accept-all address for both v4 & v6

[Test Case]
With the previous version installed, TFTP_ADDRESS reads :

# grep -i address /etc/default/tftpd-hpa
TFTP_ADDRESS="[::]:69"

With the new package it reads :

TFTP_ADDRESS=":69"

Running the reproducer as identified in comment #18 leads to a working tftpd daemon even when the NIC is not yet started.

[Regression]
None expected. This is the solution suggested by the upstream developer (hpa) in the debian bug of the same topic :

https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771441#176

>Now, what I want to know is why you are specifying the accept-all
>address explicitly as 0.0.0.0 instead of an empty string.
>
> -hpa

Tests were done also with a user modified TFTP_ADDRESS which lead to the same content before and after the upgrade. A version barrier was included in the postinst script to avoid rewriting the value once it has been changed once

[Original description of the problem]

It is new version of Bug 522509 https://bugs.launchpad.net/ubuntu/+source/tftp-hpa/+bug/522509
Since old bug was fixed for previous system. I report a new one against 14.04

ProblemType: Bug
DistroRelease: Ubuntu 14.04
Package: tftpd-hpa 5.2-7ubuntu3
ProcVersionSignature: Ubuntu 3.13.0-30.55-generic 3.13.11.2
Uname: Linux 3.13.0-30-generic x86_64
ApportVersion: 2.14.1-0ubuntu3.2
Architecture: amd64
CurrentDesktop: Unity
Date: Wed Jul 16 10:12:42 2014
InstallationDate: Installed on 2014-06-04 (41 days ago)
InstallationMedia: Ubuntu 14.04 LTS "Trusty Tahr" - Release amd64 (20140417)
SourcePackage: tftp-hpa
UpgradeStatus: No upgrade log present (probably fresh install)
mtime.conffile..etc.init.tftpd.hpa.conf: 2014-07-16T10:11:56.364626

Oleksij Rempel (olerem) wrote :
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in tftp-hpa (Ubuntu):
status: New → Confirmed
Changed in tftp-hpa (Ubuntu):
importance: Undecided → High
JamesH (jnahughes) wrote :

Just to confirm that I am seeing this on 14.10.

sgofferj (sgofferj) wrote :

Confirmed for 14.04 LTS server flavor as described also here:
http://ubuntuforums.org/showthread.php?t=2254382

Nicolas Ferre (nicolasferre) wrote :

Confirmed for 14.04 LTS.

Solved by modifying /etc/init/tftpd-hpa.conf

with:
start on (filesystem and net-device-up IFACE!=lo)

Cf: http://upstart.ubuntu.com/cookbook/#start-on

Beware, there is a bug in report #1 attachement: modified.conffile..etc.init.tftpd.hpa.conf.txt
There is a syntax error in "start on runlevel (filesystem and net-device-up IFACE!=lo)"

falstaff (falstaff) wrote :

The proposed fix (from #1 as well as the corrected version from #5) does not solve the problem for me. Actually, when I look at what time the system tries to start the tftp-hpa process, it makes it even worse: The process gets started earlier (around ~11s vs ~13s)! Also, the relevant errors look somewhat different (IPv6 resolve error). Fwiw, I use /etc/network/interface to configure eth1 statically.

Default start script
...
Jan 5 14:38:30 testseat2 in.tftpd[1052]: cannot resolve local IPv6 bind address: ::(Name or service not known)
Jan 5 14:38:30 testseat2 kernel: [ 13.111093] init: tftpd-hpa main process (1052) terminated with status 66
Jan 5 14:38:30 testseat2 kernel: [ 13.111102] init: tftpd-hpa main process ended, respawning
...

Altered start script:
....
Jan 5 14:30:59 testseat2 kernel: [ 11.365352] init: tftpd-hpa main process (708) terminated with status 66
Jan 5 14:30:59 testseat2 kernel: [ 11.365360] init: tftpd-hpa main process ended, respawning
...

falstaff (falstaff) wrote :
falstaff (falstaff) wrote :

A workaround for my case was to bind to the IPv4 address explicitly:

In /etc/default/tftpd-hpa:

TFTP_ADDRESS="0.0.0.0:69"

Jan Buchenberger (o-jan-3) wrote :

@falstaff

Your workaround works also for my configuration, just changed TFTP_ADDRESS to 0.0.0.0:69 and rebooted

FriscoSoxFan (jay-o) wrote :

TFTP_ADDRESS="0.0.0.0:69"

worked for me on 14.04. The other workaround of: "start on (filesystem and net-device-up IFACE!=lo)" did not help at all.

Kai Mast (kai-mast) wrote :

Same experience as jay-o. Please fix :/

Cpt (cptspacetoaster) wrote :

I did NOT need to edit TFTP_ADDRESS in /etc/default/tftpd-hpa. I only needed to change the 'start on' condition in /etc/init/tftpd-hpa.conf to:

start on (filesystem and net-device-up IFACE!=lo)

The service started successfully after I made this change, and rebooted.

Gfish (gareth-goldswain) wrote :

As with comment #10:

TFTP_ADDRESS="0.0.0.0:69" worked for me on 14.04 (server edition). The other workaround of: "start on (filesystem and net-device-up IFACE!=lo)" did not help at all. Even tried: "start on (filesystem and started network-services)"

Maybe something to do with fact that I have 2 network interfaces?

Why is the status still reported as "start/running" when the process clearly is not running? This is, I suppose, why the respawning of the job by upstart does not get done even though the respawn stanza is present in the conf file?

Mark Hamzy (mark-hamzy) wrote :

The IPv6 definition of localhost (::1) is clearly wrong in the configuration file.

root@pkvmci808:~# cat /etc/default/tftpd-hpa
...
TFTP_ADDRESS="[::]:69"
...
root@pkvmci808:~# systemctl start tftpd-hpa.service
Job for tftpd-hpa.service failed because the control process exited with error code. See "systemctl status tftpd-hpa.service" and "journalctl -xe" for details.
ubuntu@pkvmci808:~$ cat /var/log/syslog
...
Jun 27 17:56:51 pkvmci808 in.tftpd[21463]: cannot resolve local IPv6 bind address: ::(Name or service not known)
...

root@pkvmci808:~# vi /etc/hosts
...
TFTP_ADDRESS="[::1]:69"
...
root@pkvmci808:~# systemctl start tftpd-hpa.service

Stéphane Graber (stgraber) wrote :

Just tested here and confirmed that tftpd-hpa installs and runs perfectly fine on both a clean Ubuntu 14.04 and Ubuntu 16.04 system with its default configuration.

Changing from "[::]" to "0.0.0.0" is completely wrong as that turns off IPv6 support for tftpd-hpa which people do need.

My best guess as to why some folks need to do that change is because they somehow turned off IPv6 on their system, which then causes any daemon that's configured to bind both IPv4 and IPv6 to fail.

While it'd be nice if tftpd-hpa dealt with such setups a bit nicer, turning off IPv6 support on Ubuntu isn't something that we actively support (understand, that it's at your own risk) and such regressions are absolutely to be expected in such cases.

I've confirmed the following:
 - An Ubuntu 14.04 system with IPv6 enabled and no IPv6 address works fine
 - An Ubuntu 16.04 system with IPv6 enabled and no IPv6 address works fine
 - An Ubuntu 14.04 system with IPv6 disabled fails as described in this bug report
 - An Ubuntu 16.04 system with IPv6 disabled fails as described in this bug report

And for good measure:
 - An Ubuntu 14.04 system with IPv6 enabled and an IPv6 address works fine too
 - An Ubuntu 16.04 system with IPv6 enabled and an IPv6 address works fine too

There may still be some kind of race in tftpd-hpa which I'm not hitting here and that would cause it to only work after a little while (similar to the other bug which was referenced here).
But if it's not working at all and only works after you change it to bind 0.0.0.0, that's most likely because you reconfigured your system's networking in a way we don't support.

Robie Basak (racb) wrote :

Thanks Stéphane.

Let's mark this Incomplete until someone provides exact steps to reproduce. If the steps are to disable IPv6, then this bug will be Invalid or Won't Fix. If there are some other steps that can be followed to reproduce this, then we can look again.

To those affected: please carefully read Stéphane's comment above. If you have steps to reproduce this problem that do not involve disabling IPv6 and you believe that those steps demonstrate a bug in vsftpd or vsftpd packaging that leads to this failure, then please post your steps here and change the bug status back to New.

Changed in tftp-hpa (Ubuntu):
status: Confirmed → Incomplete
Launchpad Janitor (janitor) wrote :

[Expired for tftp-hpa (Ubuntu) because there has been no activity for 60 days.]

Changed in tftp-hpa (Ubuntu):
status: Incomplete → Expired
Louis Bouchard (louis) wrote :

Hello,

I may have found a reproducer and the explanation for this bug.

Reproducer
==========
1) Set a trusty VM with one NIC
2) Install tftp-hpa
3) From the VM console :
$ sudo -s
# service tftpd-hpa status
tftpd-hpa start/running, process 867
# service tftpd-hpa stop
tftpd-hpa stop/waiting
# ps aux | grep tftp | grep -v grep
#
# ifdown eth0
# service tftpd-hpa start
# ps aux | grep tftp | grep -v grep
#

If the NIC is down, the tftpd daemon will not start and the following will appear in /var/log/syslog :
Jan 9 16:12:26 TrustyS in.tftpd[1114]: cannot resolve local IPv6 bind address: ::(Name or service not known)

Starting tftpd while the network interface is still down, while rare, may happen is situation where the NIC is slow to come up or, for instance, for people using tftpd on laptop with Wifi only connectivity.

This is caused by the fact that tftpd will do a call to getaddrinfo() if an IPv6 host address is set, which is the case with [::]:69. This call fails if the NIC is not configured.

If no IPv6 address is set and :69 is used, the codepath reverts to INADDR_ANY which does not do the getaddrinfo(). The tftpd daemon binds correctly to the interface even if it is down as we can see :

# service tftpd-hpa stop
# ifdown eth0
# /usr/sbin/in.tftpd --listen --user tftp --address :69 --secure /var/lib/tftpboot
# ps aux | grep tftp | grep -v grep
root 1712 0.0 0.0 15120 152 ? Ss 16:25 0:00 /usr/sbin/in.tftpd --listen --user tftp --address :69 --secure /var/lib/tftpboot
# lsof -i | grep tftp
in.tftpd 1712 root 4u IPv4 10675 0t0 UDP *:tftp
in.tftpd 1712 root 5u IPv6 10676 0t0 UDP *:tftp

Debian is currently debating the same issue in https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=771441 and the current option is to change the default to :69.

I suggest to fix this bug by using the :69 value that will eventually also be used in debian.

Changed in tftp-hpa (Ubuntu):
status: Expired → New
assignee: nobody → Louis Bouchard (louis-bouchard)
Louis Bouchard (louis) on 2017-01-09
Changed in tftp-hpa (Ubuntu Xenial):
assignee: nobody → Louis Bouchard (louis-bouchard)
Changed in tftp-hpa (Ubuntu Yakkety):
assignee: nobody → Louis Bouchard (louis-bouchard)
Louis Bouchard (louis) on 2017-03-07
Changed in tftp-hpa (Ubuntu):
status: New → In Progress
Changed in tftp-hpa (Ubuntu Xenial):
status: New → Confirmed
Changed in tftp-hpa (Ubuntu Yakkety):
status: New → Confirmed
Changed in tftp-hpa (Ubuntu Xenial):
importance: Undecided → Medium
Changed in tftp-hpa (Ubuntu Yakkety):
importance: Undecided → Medium
Changed in tftp-hpa (Ubuntu):
importance: High → Medium
Louis Bouchard (louis) on 2017-03-07
Changed in tftp-hpa (Ubuntu Xenial):
status: Confirmed → Invalid
status: Invalid → Confirmed
Louis Bouchard (louis) on 2017-03-07
Changed in tftp-hpa (Ubuntu Trusty):
status: New → Confirmed
importance: Undecided → Medium
assignee: nobody → Louis Bouchard (louis-bouchard)
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package tftp-hpa - 5.2+20150808-1ubuntu2

---------------
tftp-hpa (5.2+20150808-1ubuntu2) zesty; urgency=medium

  * Replace the default value of TFTP_ADDRESS to :69 instead of [::]:69.
    The previous default caused a failure to start when the NIC is not
    available at startup time (LP: #1342580)

 -- Louis Bouchard <email address hidden> Thu, 09 Feb 2017 18:13:54 +0100

Changed in tftp-hpa (Ubuntu):
status: In Progress → Fix Released
Steve Langasek (vorlon) wrote :

+ # Replace previous dual-stack syntax
+ # that fails to work if NIC is not available
+ # (LP: #1342580)
+ if [ "${TFTP_ADDRESS}" = "[::]:69" ]; then
+ TFTP_ADDRESS=":69"
        fi

Please include a version guard in this, so that if the user insists on setting the address to [::]:69, we aren't constantly rewriting it. Compare the block of code immediately above:

        # Move from IPv4-only to dual-stack
        if [ "${TFTP_ADDRESS}" = "0.0.0.0:69" ] && \
         dpkg --compare-versions $2 lt 5.2-7ubuntu3; then
[...]

The change is ok other than that, but a version check should be included as part of this SRU.

Rejecting the current upload; please reupload with the version check.

An upload of tftp-hpa to xenial-proposed has been rejected from the upload queue for the following reason: "needs a version check for the rewrite".

Changed in tftp-hpa (Ubuntu Yakkety):
status: Confirmed → Fix Committed

Accepted the yakkety package to -proposed - unfortunately the review script crashed mid-process and didn't manage to write the comment about it.

Please test the yakkety-proposed 5.2+20150808-1ubuntu1.16.10.1 version of the package and mark it as verification-done if the bug is fixed and no regressions found. Thank you!

Changed in tftp-hpa (Ubuntu Xenial):
status: Confirmed → Fix Committed
tags: added: verification-needed

Hello Oleksij, or anyone else affected,

Accepted tftp-hpa into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/tftp-hpa/5.2+20150808-1ubuntu1.16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in tftp-hpa (Ubuntu Trusty):
status: Confirmed → Fix Committed
Łukasz Zemczak (sil2100) wrote :

Hello Oleksij, or anyone else affected,

Accepted tftp-hpa into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/tftp-hpa/5.2-7ubuntu3.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

summary: - tftpd-hpa doesn't start on boot
+ tftpd-hpa fails to start when the NIC is not available at startup time
Changed in tftp-hpa (Debian):
status: Unknown → New
Louis Bouchard (louis) on 2017-03-28
description: updated
tags: added: sts-sru-done verification-done
removed: verification-needed
Louis Bouchard (louis) wrote :

FYI, test was performed for T, X & Y, upgrading using proposed with and without a pre-set value in /etc/default/tftpd-hpa.

Value was set to :69 unless previously modified.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package tftp-hpa - 5.2-7ubuntu3.1

---------------
tftp-hpa (5.2-7ubuntu3.1) trusty; urgency=medium

  * Replace the default value of TFTP_ADDRESS to :69 instead of [::]:69.
    The previous default caused a failure to start when the NIC is not
    available at startup time (LP: #1342580)

 -- Louis Bouchard <email address hidden> Thu, 09 Feb 2017 18:13:54 +0100

Changed in tftp-hpa (Ubuntu Trusty):
status: Fix Committed → Fix Released

The verification of the Stable Release Update for tftp-hpa has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package tftp-hpa - 5.2+20150808-1ubuntu1.16.04.1

---------------
tftp-hpa (5.2+20150808-1ubuntu1.16.04.1) xenial; urgency=medium

  * Replace the default value of TFTP_ADDRESS to :69 instead of [::]:69.
    The previous default caused a failure to start when the NIC is not
    available at startup time (LP: #1342580)

 -- Louis Bouchard <email address hidden> Tue, 07 Mar 2017 12:00:08 +0100

Changed in tftp-hpa (Ubuntu Xenial):
status: Fix Committed → Fix Released
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package tftp-hpa - 5.2+20150808-1ubuntu1.16.10.1

---------------
tftp-hpa (5.2+20150808-1ubuntu1.16.10.1) yakkety; urgency=medium

  * Replace the default value of TFTP_ADDRESS to :69 instead of [::]:69.
    The previous default caused a failure to start when the NIC is not
    available at startup time (LP: #1342580)

 -- Louis Bouchard <email address hidden> Tue, 07 Mar 2017 12:01:25 +0100

Changed in tftp-hpa (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Louis Bouchard (louis) on 2017-04-05
Changed in tftp-hpa (Ubuntu):
assignee: Louis Bouchard (louis-bouchard) → nobody
Changed in tftp-hpa (Ubuntu Trusty):
assignee: Louis Bouchard (louis-bouchard) → nobody
Changed in tftp-hpa (Ubuntu Xenial):
assignee: Louis Bouchard (louis-bouchard) → nobody
Changed in tftp-hpa (Ubuntu Yakkety):
assignee: Louis Bouchard (louis-bouchard) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.