nginx daemon should be provided in a package that doesn't have dependencies to systemd (or nginx-common)

[Andres Rodriguez]

nginx-core provides the nginx daemon with postinst files that handle the start of the services.\

At the same time, nginx-core depends on nginx-common. The latter installs all the service files that allow the services to be managed by systemd. However, this is a problem in environments where you only want the nginx daemon and not managed by systemd.

I would expect this to be the case:

 - nginx-core to install all systemd related files
 - nginx-core to depend on nginx-daemon
 - nginx-daemon to provide only the daemon with no dependencies, allowing it to work on environments where we dont want systemd to manage it (e.g. docker).

[Andreas Hasenack (ahasenack)]

I'll let Thomas Ward (@teward) chime in, but to me this looks like a wishlist item at the moment.

Initially I even thought it would be a big departure from the debian base package, but looks like ubuntu already builds a different set of binary packages.

[Robie Basak (racb)]

MySQL uses the following pattern to achieve this use case:

mysql-server-core-5.7 ships binaries only with no configuration
mysql-server-5.7 depends on mysql-server-core-5.7 and provides management of /var/lib/mysql, supplies configuration in /etc/mysql (along with mysql-common) and provides system service definitions.

Perhaps nginx packaging could be adjusted to use a similar pattern (Andres' suggestion is equivalent; I'm just pointing out the pattern that already exists).

[Thomas Ward (teward)]

To follow this suggestion would be problematic.

And I say problematic because this would add a significant delta.

The core flavor of NGINX, back in the 14.04 cycle, was a flavor of NGINX with the same module set as the full flavor but without third party modules in it, to satisfy the Security team's concerns for the MIR.

To meet the specs that this feature request needs, this flavor name needs to change.

This also would need an additional core package for each individual flavor of NGINX (light, full, extras) with a core package that doesnt depend on -common because each flavor differs in module set and the binaries differ because of those module sets (compiled in and not dynamically includeable). This isnt just one package with one binary that's equivalent across all flavors of nginx package we install, this is multiple flavors of the binary with different module sets resulting in different nginx binaries for each flavor.

Either method I see for doing this would make us permanently divergent from Debian to the point we can no longer merge from them and would make maintaining this more difficult because of the additional complexity added into the builds.

Robie, I can discuss more about this on IRC after I get to work, the discussion on this further will take up pages of text - not just a bug response.

[Andres Rodriguez (andreserl)]

Hi Thomas,

Maybe I’m missing some context, but could you explain why we couldn’t do
the same in Debian?

I don’t think it is generally crazy to just want to install what nginx-core
provides today without having to install what nginx-common provides today.

nginx-core could just be a meta package that installs nginx-daemon (or
similar as Robbie pointed out) which would prevent the need from modifying
the other packages?

Thanks.

On Tue, Jul 17, 2018 at 8:25 AM Thomas Ward <email address hidden> wrote:

> To follow this suggestion would be problematic.
>
> And I say problematic because this would add a significant delta.
>
> The core flavor of NGINX, back in the 14.04 cycle, was a flavor of NGINX
> with the same module set as the full flavor but without third party
> modules in it, to satisfy the Security team's concerns for the MIR.
>
> To meet the specs that this feature request needs, this flavor name
> needs to change.
>
> This also would need an additional core package for each individual
> flavor of NGINX (light, full, extras) with a core package that doesnt
> depend on -common because each flavor differs in module set and the
> binaries differ because of those module sets (compiled in and not
> dynamically includeable). This isnt just one package with one binary
> that's equivalent across all flavors of nginx package we install, this
> is multiple flavors of the binary with different module sets resulting
> in different nginx binaries for each flavor.
>
> Either method I see for doing this would make us permanently divergent
> from Debian to the point we can no longer merge from them and would make
> maintaining this more difficult because of the additional complexity
> added into the builds.
>
> Robie, I can discuss more about this on IRC after I get to work, the
> discussion on this further will take up pages of text - not just a bug
> response.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1781971
>
> Title:
> nginx daemon should be provided in a package that doesn't have
> dependencies to systemd (or nginx-common)
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/nginx/+bug/1781971/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: distribution=ubuntu; sourcepackage=nginx; component=main;
> status=Triaged; importance=Wishlist; assignee=None;
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: ahasenack andreserl racb teward
> Launchpad-Bug-Reporter: Andres Rodriguez (andreserl)
> Launchpad-Bug-Modifier: Thomas Ward (teward)
> Launchpad-Message-Rationale: Subscriber
> Launchpad-Message-For: andreserl
>
--
Andres Rodriguez (RoAkSoAx)
Ubuntu Server Developer
MSc. Telecom & Networking
Systems Engineer

[Thomas Ward (teward)]

Andres,

I think you and I need to discuss off-bug the extent of the changes that your proposal would entail. Preferably with Robie looped in because I had a bit of a more in-depth discussion with Robie earlier today via IRC of my concerns with this request, as well as three 'options' to solve this depending on the information made available. It's not as simple as making "one metapackage" as there is not "one nginx binary" in use here.

Essentially, as a summary of my concerns in the most basic form (in which an actual discussion would be more useful than back and forth emails or bug comments) is that to do what you're saying would necessitate the creation of **four new packages**, the renaming of two packages from that set of resultant packages completely, **breaking** upgradeability between Ubuntu releases, and including not three but four or five packages in Main.

The core problem is becasue we have **four separate NGINX binaries** of differing flavors:
 - nginx-full contains a more 'full' featureset including some third-party modules (this flavor was picked upstream in Debian)
 - nginx-light contains a lesser set of features, but enough to do the minimum requisite things for an average NGINX deployment (it has a few third party modules though).
 - nginx-extras has all the core modules and a bunch of extra modules from third parties.
 - nginx-core (the "Core NGINX flavor" really, which was decided upon with coordination between myself, the Server Team, and the TB) contains the nginx-full featureset minus third-party modules (needed to exclude those for the MIR done during the Trusty cycle).

To make a 'core' version of any of those requires first renaming nginx-core to some other name (which will break the ability to upgrade between releases safely with NGINX on board), and then split *four* flavors into nginx-{flavor} and nginx-{flavor}-core because the binaries differ across all four flavors. This is because the NGINX binaries do not have completely plug-and-play modules, and instead have a hybrid of dynamically-includable modules and **hard-coded compiled-into-the-binary** modules, making each flavor's nginx binary different from each other, so it's not easy to just change the -core package to just provide 'nginx' without any configuration.

This also will make any merges from Debian impossible because we've fundamentally altered the packaging at that point so much so that we will not be able to cleanly merge *anything* from Debian in terms of packaging changes, patches to package-specific things, etc.

This is just the tip of the iceberg in terms of how much your proposal will increase package management and maintainability complexities into the far future. As I said I brought all this up with Robie earlier today via IRC, it may be more prudent for us to have this discussion over a phone call or over a hangout rather than try and do back-and-forth here over bug responses/comments or email.

[Thomas Ward (teward)]

And as an aside, I don't have any influence in how Debian changes the packages - if this proposal is made in Debian for their nginx-{light,full,extras} flavors it can be extended to include our flavors, but we'll have to do some decision making, *and* we need to allow a method to migrate existing nginx-core users to nginx-newname (where newname is the 'new' name for the Core Flavor we established back in 14.04). Otherwise, upgrades in-place between OS versions are very likely to explode/break.

[Thomas Ward (teward)]

In discussion with Andres, this would only impact the 'core' flavor of NGINX. It would not be scoped to include any other packages other than the Main-included ones.

The original request came up because we don't have any way to not fail currently during installation or upgrading if something else is bound to port 80. Rather than changing the fundamentals of the packaging radically such as proposed, it may be easier to simply institute checking if port 80 is bound to or not. (see Bug #1782226 where this is proposed by myself)

[Thomas Ward (teward)]

I'm working on this, though the initial attempts on splitting this have failed to build properly locally. Looking into that, then going to put this up to a PPA so we can do some testing on this (without having to dpkg -i all the packages manually)

[Thomas Ward (teward)]

Solved the local build problem, now just kicking around the nginx-daemon package (this is a **temporary** naming, it probably will become nginx-core-daemon in the future) to get rid of its dependency on nginx-common, and then making sure installations work as-expected.

https://launchpad.net/~teward/+archive/ubuntu/nginx-1781971 is the PPA I'm working with for now on this.

[Thomas Ward (teward)]

Note there's some changes in progress; I'll let you know when packages are stabilized and ready for full testing.

[Thomas Ward (teward)]

Digging deeper, we'll have to strip the dynamic modules out, there's no way to provide them without including nginx-common because of the way they install as the package, so this will remove the dynamic modules which include the following feature sets:

 * HTTP: XSLT Filter
 * HTTP: GeoIP
 * HTTP: Image Filter
 * Mail
 * Stream

Otherwise, there is no way to include those without depending on nginx-common.

I'll see how feasible this is after I cool off - my coworkers are horribly inconsiderate today and are causing me too many distractions to focus on this at the moment.

[Thomas Ward (teward)]

Test-worthy packages now exist at https://bugs.launchpad.net/ubuntu/+source/nginx/+bug/1781971

The package versions that you can utilize to test this are version 1.14.0-0ubuntu3~lp1781971.6

The 'daemon only' package is nginx-core-daemon. Please feel free to test these packages heavily and let me know what you think. If there's no additional issues, and no complaints from anyone else anywhere, I will look into getting this applied to the existing packages and uploaded to the repositories.

[Thomas Ward (teward)]

Added Bionic series to the task at the request of Andres Rodriguez (via IRC, and asking if this could eventually be an SRU'd item)

[Thomas Ward (teward)]

As there has been no testing or suggestions of this or push to further pursue this, I am marking this as Incomplete for expiration in 60 days; if there is more of a need for this, please update the bug, as this is at this point a stale bug that isn't really moving anywhere.

[Launchpad Janitor (janitor)]

[Expired for nginx (Ubuntu Bionic) because there has been no activity for 60 days.]