[MIR] nodejs as dependency of mailman3

[Christian Ehrhardt ]

[Availability]
The package is already universe for quite a while and build/works fine so far.
It is for example already used for https://lists.canonical.com/mailman3/postorius/lists/
OTOH it is the nodejs core element that can/could be used for much more than just the mailman3 stack.

For the mailman stack we'd pull in most binaries, being libnode64, nodejs and nodejs-doc

[Rationale]
This is part of the MIR activity for all dependencies of mailman3
The "main" MIR of it is at bug 1775427:

Mailman (2) has only python2 support, but we strive for python3,
therefore Mailman3 which has python3 support should be promoted to main.

I know this is dragging in a lot of components, but mailman3 was re-implmented
using common frameworks and that meands django, node, ...

[Security]

This is one of the components of overall mailman3 stack that is not looking so good.
We know of 7 CVEs of which only one is known fixed.
The rest would need analysis and triage to be sure
=> https://people.canonical.com/~ubuntu-security/cve/pkg/nodejs.html

When checking on Mitre the list is longer, but mapping to actual src packages can be hard:
=> http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=nodejs
I found no extra CVEs other than those tracked by the security Team that are really on src:nodejs

For CVE-2019-1559 and CVE-2019-1543 we are safe due to a new openssl.

CVE-2019-5737 was fixed in a recent upload that is synced to Debian.

Therefore no open known issues left right now.

But in general that package clearly has a higher exposure and therefore effort for the security Team once promoted.
Therefore their ack is needed even more than usual.

[Quality assurance]

As part of the mailman3 stacks as of now (Disco) this installs fine and works fine.
On itself it is useful to (many) other dependencies and does not need a post install configuration on its own.

The package does not ask debconf questions.

28 bugs in Ubuntu for this - the majority is the usual crap, but there are quite some SIGABRTs which reflects the effort we will have to maintain it once promoted to main.
Another issue becoming clear on these reports is that the users often mix&match NPM with Archive versions and that is unsupported, but hard to debug and quite some effort.
Also the number is that high because currently no team owns them, it seems many could be cleared (I did some now).
Debian has 9 bugs open, mostly odd architecture FTBFS and a segfault that waits for more info - nothing critical we have to wait on.
Upstream OTOH seems very active, there are 596 open and 8989 closed bugs.

The package seems to get regular updates by upstream and Debian.

No exotic HW involved.

The package utilizes a lot build time self tests.
In addition it has some (trivial) autopkgtests that would e.g. detect breakage by a main node.js upgrade.

No Lintian warning except a few newer Standards/Compat versions and watch GPG checks and a few false positives about too long lines - nothing severe.
It also has
W: nodejs source: patch-file-present-but-not-mentioned-in-series 2012_kfreebsd.patch but that isn't important for non BSD builds and just kept around for their comfort.

The package does not rely on demoted or obsolete packages.
No new gt2k dependencies

[UI standards]

It uses internationalization via ICU, it even has an own "intl" issue category upstream.
No End-user applications that needs a standard conformant desktop file.

[Dependencies]

Some dependencies are not in main, but we drive MIR for all related packages
that are not in main at the same time.
Please check the list of bugs from the main Mailman3 MIR in bug 1775427 to get an overview.

[Standards compliance]

The package mostly meets the FHS and Debian Policy standards.

Not really a policy violation but a usual MIR red flag is that bundles a lot of things, see deps/* and I think this would usually at first sight be a reason to nack it, but not only would it be a huge effort to split it.

I checked and it seems it uses almost all from shared libs, the only one it does use the embedded is the v8 source.
Fortunately the config seems to disable most of these - see in d/rules setting configure to use --shared-*

As further confirmation libnode-dev conflicts with libv8-dev to be a full alternative (even better see below), still the one thing it duplicates is the v8 engine which otherwise is in src:libv8-3.14.

While not perfect it seems that is done for a reason, and that would mean especially the security Team would have two v8 implementations to check.

My lack of experience let me summarize the above about the bundling of v8, but fortunately the Debian maintainer chimed in - see comment #2 and put a lot of sense behind the reasoning to bundle v8, quoting:

Kapoer: "it uses embedded v8 because upstream backports v8 security
    fixes to the branch they cut for nodejs, and because v8 versions are
    hard to follow, ABI-wise, and upstream nodejs pay attention to
    keeping ABI compatibility, so downstream distributors benefit
    directly from their work.

    It's so much easier that way, that since libnode64 exports v8
    symbols, the libv8-dev package provides symlinks to libnode-dev so
    others can build against libnode to get a working, stable v8.

    A package willing to do that just has to -lv8, -I/usr/include/v8 and
    build-depends on libv8-dev, as one would expect.

    All other directories in deps could be removed in theory, but I kept
    most of them because it makes debugging our builds much easier - one
    has to check rebuilding without shared libs still shows the bug
    before reporting upstream.

    Also nodejs-dev distributes openssl.cnf file that is useful for
    running testsuites of node modules expecting specific openssl
    config.

    The build could be using shared libhttp-parser too, but i spotted
    this too late for next debian release (we're in deep freeze)"

The packaging itself (e.g. d/rules) is complex as well, with many special cases. Nothing insane at all fortunately, just complex - but more potential bits to understand when providing service later on.

[Maintenance]

The Server team will subscribe for the package for maintenance

[Background]
The package description explains the general purpose and context of the package well.

[Christian Ehrhardt  (paelzer)]

Not yet subscribing the MIR team until we have clarified if we could drop the dependency to the nodejs bits completely (https://<email address hidden>/thread/HFNP4DPZ3CLXV3OOEJVHCVOLZ7OMAI7G/).

[kapouer (kapouer)]

There are several things about nodejs package that are not really right,
let me fix this below.

Le ven. 15 mars 2019 à 10:45, Christian Ehrhardt  <
<email address hidden>> a écrit :

> Public bug reported:
>
> [Availability]
> The package is already universe for quite a while and build/works fine so
> far.
> It is for example already used for
> https://lists.canonical.com/mailman3/postorius/lists/
> OTOH it is the nodejs core element that can/could be used for much more
> than just the mailman3 stack.
>
> For the mailman stack we'd pull in most binaries, being libnode64,
> nodejs and nodejs-doc
>
> [Rationale]
> This is part of the MIR activity for all dependencies of mailman3
> The "main" MIR of it is at bug 1775427:
>
> Mailman (2) has only python2 support, but we strive for python3,
> therefore Mailman3 which has python3 support should be promoted to main.
>
> I know this is dragging in a lot of components, but mailman3 was
> re-implmented
> using common frameworks and that meands django, node, ...
>
> [Security]
>
> This is one of the components of overall mailman3 stack that is not
> looking so good.
> We know of 7 CVEs of which only one is known fixed.
> The rest would need analysis and triage to be sure
> => https://people.canonical.com/~ubuntu-security/cve/pkg/nodejs.html
>
> When checking on Mitre the list is longer, but mapping to actual src
> packages can be hard:
> => http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=nodejs
> I found no extra CVEs other than those tracked by the security Team that
> are really on src:nodejs
>
> For CVE-2019-1559 and CVE-2019-1543 we are safe due to a new openssl.
>
> CVE-2019-5737 was fixed in a recent upload that is synced to Debian.
>
> Therefore no open known issues left right now.
>
> But in general that package clearly has a higher exposure and therefore
> effort for the security Team once promoted.
> Therefore their ack is needed even more than usual.
>
> [Quality assurance]
>
> As part of the mailman3 stacks as of now (Disco) this installs fine and
> works fine.
> On itself it is useful to (many) other dependencies and does not need a
> post install configuration on its own.
>
> The package does not ask debconf questions.
>
> 28 bugs in Ubuntu for this - the majority is the usual crap, but there are
> quite some SIGABRTs which reflects the effort we will have to maintain it
> once promoted to main.
> Another issue becoming clear on these reports is that the users often
> mix&match NPM with Archive versions and that is unsupported, but hard to
> debug and quite some effort.
> Also the number is that high because currently no team owns them, it seems
> many could be cleared (I did some now).
> Debian has 9 bugs open, mostly odd architecture FTBFS and a segfault that
> waits for more info - nothing critical we have to wait on.
> Upstream OTOH seems very active, there are 596 open and 8989 closed bugs.
>
> The package seems to get regular updates by upstream and Debian.
>
> No exotic HW involved.
>
> The package utilizes a lot build time self tests.
> In addition it has some (trivial) autopkgtests that would e.g. detect
> breakage by a main node.js upgrade.
>

There is a huge test suite tha...

[Christian Ehrhardt  (paelzer)]

Thanks Kapouer,
yeah the test/fixtures/throws_error5.js files are not really javascript - good hint.

And the insights you have provided on why v8 is embedded are really great - thank you a lot for chiming in here - I'll copy and paste a bit of that into the description!

While I still would want to make mailman3 not depending on nodejs, that helped a lot if we end up continuing to MIR nodejs.

[Christian Ehrhardt  (paelzer)]

After evaluating dependencies, required further changes and mostly maintainability for security and packaging it was decided there are too many concerns - not about any single package in particular, but the overall Mailman3 stack - about the ability to maintain and monitor it as well as we need it for support in main.

We have closed the primary LP bug already, the MIRs that are already approved will stay that way, but we will make no seed change to pull things in for now. Yet if other needs come up for those they have a prepared MIR already.
Other bugs which are not yet completed in terms of review will be closed as Won't Fix.
Others are special cases like this one - here I worked with Debian to drop the node&ruby dependencies completely which is done and would be in Ubuntu on the next merge/sync - therefore we would not need this MIR anyway.

Even thou it ended being aborted, I think that is a valid outcome of the MIR evaluations. Never the less I want to thank everybody involved for all the work spent in what was nearly a year working through these MIRs.