[Ubuntu 16.04] nginx won't start on boot while network does not up yet

Bug #1666368 reported by stephon chen
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
nginx (Ubuntu)
Won't Fix
Undecided
Unassigned

Bug Description

Hi Ubuntu bug report,

After installing nginx and setup done, nginx works normally by "systemctl restart nginx.service"

But nginx won't start on boot.

After running "systemctl status nginx.service", I found that network & dns lookup have not done yet on boot.

nginx should be started after network online and dns lookup working normally, instead of after network.target.

In the attachment, I modified /lib/systemd/system/nginx.service to require network-online.target & nss-lookup.target works normally.

Would you like to merge the patch to nginx package?

Thanks a lot.
--
Stephon Chen

Revision history for this message
stephon chen (stephon) wrote :
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote :

The attachment "Patch for /lib/systemd/system/nginx.services, let nginx start requiring network-online and nss-lookup work normally." seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

tags: added: patch
Revision history for this message
Simon Déziel (sdeziel) wrote :

@stephon, the default vhost provided by the package doesn't produce a configuration that requires DNS resolution to start so could you share your nginx/vhost config? Please also share the journalctl entries for nginx.

Changed in nginx (Ubuntu):
status: New → Incomplete
Revision history for this message
Simon Déziel (sdeziel) wrote :

Also, if your local config does indeed require DNS to be resolvable when nginx starts, I see 2 possible fixes/workarounds for you.

1) you could use a systemd drop-in snippet to tune the startup order like you did in your patch.

2) you could put the IP(s) and DNS name(s) under /etc/hosts. This will only work for static IPs though.

Revision history for this message
stephon chen (stephon) wrote :

Hi @Simon,

From above comment, nginx default package in ubuntu just setup for IP-based vhost only.

Since my environment needs domain-based vhosts config, I have adopted systemd drop-in snipped like 1) already.

Thanks for your information, and this ticket can be closed.

Revision history for this message
Thomas Ward (teward) wrote :

The SystemD spec currently says daemons should seek network.target, not network-online.target.

We will not be altering the SystemD specs for the package or its SystemD unit at this time. Providing the SystrmD drop in snippets is the proper approach as the default daemon behavior on a clean install does not need network-online.target.

Changed in nginx (Ubuntu):
status: Incomplete → Won't Fix
Revision history for this message
Thomas Ward (teward) wrote :

The SystemD documentation still suggests to NOT pull in network-online.target.

From https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/ which is referred to in the special targets documentation, specifically the description of `network-online.target`:

> It is strongly recommended not to pull in this target too liberally: for example network server software should generally not pull this in (since server software generally is happy to accept local connections even before any routable network interface is up), its primary purpose is network client software that cannot operate without network.

This basis is what we're following in this case.

Revision history for this message
stephon chen (stephon) wrote : Re: [Bug 1666368] Re: [Ubuntu 16.04] nginx won't start on boot while network does not up yet

Hi Thomas,

Thanks for your helpful update, and we will try to modify our workflow to
mitigate.

Thanks and Regards,
--
Stephon Bo-Tsung Chen

On Tue, Jan 7, 2020 at 4:30 AM Thomas Ward <email address hidden> wrote:

> The SystemD documentation still suggests to NOT pull in network-
> online.target.
>
> >From https://www.freedesktop.org/wiki/Software/systemd/NetworkTarget/
> which is referred to in the special targets documentation, specifically
> the description of `network-online.target`:
>
> > It is strongly recommended not to pull in this target too liberally:
> for example network server software should generally not pull this in
> (since server software generally is happy to accept local connections
> even before any routable network interface is up), its primary purpose
> is network client software that cannot operate without network.
>
> This basis is what we're following in this case.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1666368
>
> Title:
> [Ubuntu 16.04] nginx won't start on boot while network does not up yet
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/nginx/+bug/1666368/+subscriptions
>

Revision history for this message
Sven Kieske (s-kieske) wrote :

Hi,

so this bug, and it clearly is a bug if a service doesn't start, hit us and I'd like to argue why the current decision seems wrong to me.

I would argue that the most common scenario is, to want to start nginx on some configured IP, which of course needs to be configured, before the service can start.

IMHO a systemd service file should cater to the 99% usecase: nginx listening on a non local ip address and serving files to the interwebs.

fwiw this was fixed in fedora and thus in rhel:

https://bugzilla.redhat.com/show_bug.cgi?id=1943779

so imho it would also be good to align between distributions and do the sensible thing.

notice also, that what you cite is:

a) only a recommendation, albeit a "strong" one
b) "its primary purpose is network client software that cannot operate without network."

well nginx is arguably a network client/server (depending on how you look at it), which, in 99% of all use cases, "can't operate without network" in a meaningful way.

so it would be really much appreciated if you would reconsider this and would not put the burden of workarounds on all downstream users and developers.

thank you very much for your time, consideration and work

kind regards

Sven

tags: added: server-triage-discuss
tags: removed: server-triage-discuss
Robie Basak (racb)
tags: added: network-online-ordering server-triage-discuss
Revision history for this message
Thomas Ward (teward) wrote :

The Server Team had a discussion on this today, including taking into account the distro-wide spec under review for defining 'online' - https://discourse.ubuntu.com/t/spec-definition-of-an-online-system/27838/6 - and it was decided that the original decision in 2020 which echoes SystemD documentation, and in accordance with the current revision of the distro-wide spec for Ubuntu, still stands. The SystemD service for nginx will not change from `network.target`, and if you need `network-online.target` this should be a configuration override done at the admin level by the admin configuring the endpoint system.

Also, to quote vorlon in the spec discussion thread:

> network-online.target should still be avoided as a dependency whenever not strictly necessary. A service that can be configured to bind to a specific address, but doesn’t by default, should not depend on this target as it will make the boot slower/less reliable for users in the default case.

Interpreted through me, this should be read as "If you are running a configuration which requires network-online.target, you should configure a systemd override accordingly for your service rather than enforce this globally for an application that by default SHOULD be able to run on just localhost without DNS, etc. online." - which NGINX qualifies for in this interpretation.

We will once again *not* be implementing any changes to the SystemD service for the target it uses for loading.

tags: removed: server-triage-discuss
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.