[MIR] libmail-dmarc-perl

Bug #2023971 reported by Bryce Harrington
14
This bug affects 2 people
Affects Status Importance Assigned to Milestone
libmail-dmarc-perl (Ubuntu)
Fix Released
High
Unassigned

Bug Description

[MIR] libmail-dmarc-perl

Package: libmail-dmarc-perl
-----
Dependency status update (Feb 01, 2024):

On Feb 1, the six needed packages to be promoted to main and their bug status are:

* libmail-dmarc-perl (this bug): bug 2023971, New, Ioanna assigned: pending review, package to be tested at [1] with all the changes required in the other MIR bugs, and the dependency splitting.

* libemail-simple-perl: bug 2031491, New, assigned security team : conditional ACK from security team by George-Andrei Iosif (iosifache)
           -> maintenance by Server Team if we lack upstream support
           -> ask for demotion (in the future) if another package if a more
              suitable can be used as an alternative

* libfile-sharedir-perl: bug 2039566 , In progress, unassigned -> ACK by slyon, all TODOS done.

  * libclass-inspector-perl: bug 2039569 , Fix commited, unassigned -> ACK by Didier, All todos done

* libnet-ip-perl: bug 2039456 , In progress, unassigned -> ACK by Ioanna, All todos done

* libregexp-common-perl: bug 2039563 , Fix commited, unassigned-> ACK by Didier, All todos done

[1] https://launchpad.net/~mirespace/+archive/ubuntu/libmail-dmarc-perl-suggested/+sourcepub/15684761/+listing-archive-extra

-----
Please promote libmail-dmarc-perl and its universe dependencies (recursively) to main. According to `check-mir` (+ recursive searching) this would need ~44 packages to be promoted (as runtime binary dependencies, see [Dependencies] section for detailed dependencies tree) .

However, libmail-dmarc-perl is only used by the spamassassin package, and spamassassin only uses the validation feature of DMARC, so we can narrow the necessary binaries dependencies to be promoted too. Those packages are the following ones (11), which I propose for promotion instead of the complete dependencies tree:

* libemail-mime-perl: universe
   MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-perl/+bug/2030880
   - libemail-messageid-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-messageid-perl/+bug/2030956
   - libemail-mime-contenttype-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-contenttype-perl/+bug/2030962
     + libtext-unidecode-perl: universe
        MIR bug: https://bugs.launchpad.net/ubuntu/+source/libtext-unidecode-perl/+bug/2031109
   - libemail-mime-encodings-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-encodings-perl/+bug/2031487
   - libemail-simple-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-simple-perl/+bug/2031491
 * libfile-sharedir-perl: universe
   MIR bug https://bugs.launchpad.net/ubuntu/+source/libfile-sharedir-perl/+bug/2039566
   + libclass-inspector-perl: universe
     MIR bug https://bugs.launchpad.net/ubuntu/+source/libclass-inspector-perl/+bug/2039569
 * libnet-idn-encode-perl: universe
   MIR bug https://bugs.launchpad.net/ubuntu/+source/libnet-idn-encode-perl/+bug/2038929
 * libnet-ip-perl: universe
   https://bugs.launchpad.net/ubuntu/+source/libnet-ip-perl/+bug/2039456
 * libregexp-common-perl: universe
   MIR bug https://bugs.launchpad.net/ubuntu/+source/libregexp-common-perl/+bug/2039563

The rest would be moved to Suggested dependencies.

[Availability]
The package libmail-dmarc-perl is already in Ubuntu universe.
The package libmail-dmarc-perl builds for the architectures it is designed to work on.
It currently builds and works for architectures: amd64 (all)
Link to package https://launchpad.net/ubuntu/+source/libmail-dmarc-perl

[Rationale]

tldr; DMARC support in SpamAssassin is important for stronger spam filtering.

Spam email is an ever-present and ever-evolving presence in our online lives, and SpamAssassin is a key tool for end users and service providers to identify likely spam for filtering. SpamAssassin 4.0, introduced in Ubuntu "lunar" 22.10, introduced a number of major new features including three new plugins, the most significant of which is the DMARC policy checker.

DMARC (or "Domain-based Message Authentication, Reporting & Conformance") [0] is a new convention for email service providers to communicate to email recipient programs about how to handle authentication failures. It builds on prior protocols (namely, SPF and DKIM) to address their limitations. Essentially, DMARC protects against direct domain spoofing, such that when an email purports to be from a given domain (say, @gmail.com or @irs.gov) but fails proper authentication using the authentication methods published by that domain, it tells the email receiver whether to reject the email as spam, quarantine it for evaluation, or something else. DMARC also establishes a way for the email receiver to give feedback back to the sender about emails that failed to pass this check.

libmail-dmarc-perl contains the official Perl implementation of DMARC support. SpamAssassin is the primary user of this package

❯ apt-cache rdepends libmail-dmarc-perl
libmail-dmarc-perl
Reverse Depends:
  spamassassin

and requires it in order to perform this DMARC evaluation: Spamassassin dmarc plugin [1] only uses the MAIL::DMARC::PurePerl module [2] and, inside it, the validate function in particular [3]. It means that Spamassasin only uses by default the DMARC's validation feature, and optionally the DMARC's Reporting feature.

libmail-dmarc-perl is a relatively new source package that was proposed for lunar [4] but deleted on 2023-05-01 due to build/autopkgtest issues and later re-introduced to mantic once those issues were resolved, on 2023-06-13.

[0] (https://dmarc.org/)
[1] (https://git.launchpad.net/ubuntu/+source/spamassassin/tree/lib/Mail/SpamAssassin/Plugin/DMARC.pm)
[2] (https://git.launchpad.net/ubuntu/+source/spamassassin/tree/lib/Mail/SpamAssassin/Plugin/DMARC.pm#n242)
[3] (https://git.launchpad.net/ubuntu/+source/spamassassin/tree/lib/Mail/SpamAssassin/Plugin/DMARC.pm#n322)
[4] (https://bugs.launchpad.net/ubuntu/+source/libmail-dmarc-perl/+bug/2023606)

[Security]
No CVEs/security issues in this software in the past:
  - (0) https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=libmail-dmarc-perl
  - (0) https://ubuntu.com/security/cves?q=&package=libmail-dmarc-perl
  - (0) https://security-tracker.debian.org/tracker/source-package/libmail-dmarc-perl
Broadening the search to "spamassassin dmarc" shows only a single issue within the past few years, CVE-2022-3620, which appears to be more of an issue in exim4 specifically, and has been deemed not relevant to the exim4 package shipped in Ubuntu (See https://ubuntu.com/security/CVE-2022-3620).

No `suid` or `sgid` binaries.
No executables in `/sbin` and `/usr/sbin`.
Package does not install services, timers or recurring jobs, but contains an example of a cron job for updating the public_suffix_list (https://publicsuffix.org/).
Package does not open privileged ports (ports < 1024).
Package does not expose any external endpoints.
Package contains extensions to security-sensitive software (filters, scanners, plugins, UI skins, ...): It is the basis for the DMARC plugin of spamassassin, implementing the DMARC policy validation (and reporting).

Upstream's source code includes the build of a web server (dmarc_httpd) and an HTTP client (dmarc_http_client), but these binaries are not installed in the debian package (they are suppressed on build time for autoinstallation).

[Quality assurance - function/usage]
The package works well right after installed with its required installation dependencies.

Integration with spamassassin has been tested by co-installing it, updating the configuration, running sa-compile, and checking for errors. Behavioral performance was checked by running:

  $ spamassassin --debug < /home/bryce/pkg/Spamassassin/spam.mbox

Processing of emails by spamassassin using its DMARC plugin that uses libmail-dmarc-perl validation function was tested also grabbing an email (i.e, from gmail) and doing:

  $ spamc -R < message-mattermost-hey.eml | grep DMARC
  -0.0 DMARC_PASS DMARC pass policy

also, non pass results with an already classified as spam mail (by gmail, cleaning the header):
   0.9 DMARC_NONE DMARC none policy

or introducing typos to malform domains/header:
   0.0 DMARC_MISSING Missing DMARC policy

For rejection examples, we use the data from spamassassin's test, and we can see:

root@Mspamassasin-suggested:~# l spam_mails/
nodmarc.eml noneko.eml quarko.eml rejectko.eml strictrejectko.eml
root@Mspamassasin-suggested:~# spamc -R < spam_mails/rejectko.eml | grep DMARC
 1.8 DMARC_REJECT DMARC reject policy
root@Mspamassasin-suggested:~# spamc -R < spam_mails/strictrejectko.eml | grep DMARC
 1.8 DMARC_REJECT DMARC reject policy
root@Mspamassasin-suggested:~# spamc -R < spam_mails/quarko.eml | grep DMARC
 1.2 DMARC_QUAR DMARC quarantine policy
root@Mspamassasin-suggested:~# spamc -R < spam_mails/nodmarc.eml | grep DMARC
 0.0 DMARC_MISSING Missing DMARC policy
root@Mspamassasin-suggested:~# spamc -R < spam_mails/noneko.eml | grep DMARC
 0.9 DMARC_NONE DMARC none policy

I'm working on an autopkgtest that covers this: The code is uploaded but not working yet. For manual testing, you can get the eml messages from this commit:

https://git.launchpad.net/~mirespace/ubuntu/+source/libmail-dmarc-perl/commit/?id=feb85cd6f97508c98bdeb732fe25f876f33790d4

where d/t/data/nice contains messages that qualifies as PASS due to the DMARC rules (DMARC_PASS) and d/t/data/spam contains messages that are market as REJECT, QUARENTINE, MISSING or NONE.

The specific test on spamassassin for DMARC (t/dmarc.t) was also run successfully (without the implicit net tests on this file):

  $ make test TEST_FILES="t/dmarc.t"
  "/usr/bin/perl" build/mkrules --exit_on_no_src --src rulesrc --out rules --manifest MANIFEST --manifestskip MANIFEST.SKIP
  mkrules: no rules updated
  "/usr/bin/perl" build/preprocessor -Mvars -DVERSION="4.000000" -DPREFIX="/usr/local" -DDEF_RULES_DIR="/usr/local/share/spamassassin" -DLOCAL_RULES_DIR="/etc/mail/spamassassin" -DLOCAL_STATE_DIR="/var/lib/spamassassin" -DINSTALLSITELIB="/usr/local/share/perl/5.36.0" -DCONTACT_ADDRESS="the administrator of that system" -DRE2C_BIN="re2c" -Msharpbang -Mconditional -DPERL_BIN=""/usr/bin/perl"" -DPERL_WARN="" -DPERL_TAINT="" -m755 -isa-update.raw -osa-update
  cp sa-update blib/script/sa-update
  "/usr/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/sa-update
  PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/dmarc.t
  t/dmarc.t .. Nov 3 12:36:16.977 [8195] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
  t/dmarc.t .. 1/18 Nov 3 12:36:18.906 [8197] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
  t/dmarc.t .. 3/18 Nov 3 12:36:21.247 [8199] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
  t/dmarc.t .. 5/18 Nov 3 12:36:23.737 [8201] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
  t/dmarc.t .. 7/18 Nov 3 12:36:26.153 [8203] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
  t/dmarc.t .. 9/18 Nov 3 12:36:29.117 [8205] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
  t/dmarc.t .. 11/18 Nov 3 12:36:32.796 [8207] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
  t/dmarc.t .. 13/18 Nov 3 12:36:35.338 [8209] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
  t/dmarc.t .. 15/18 Nov 3 12:36:37.374 [8211] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
  t/dmarc.t .. ok
  All tests successful.
  Files=1, Tests=18, 31 wallclock secs ( 0.02 usr 0.00 sys + 12.18 cusr 0.88 csys = 13.08 CPU)
  Result: PASS

[Quality assurance - maintenance]
The package is maintained well in Debian/Ubuntu and does
not have too many, long-term & critical, open bugs:
   - Ubuntu (0) https://bugs.launchpad.net/ubuntu/+source/libmail-dmarc-perl/+bug
   - Debian (0) https://bugs.debian.org/cgi-bin/pkgreport.cgi?src=libmail-dmarc-perl
   - Upstream's bug tracker (4) https://github.com/msimerson/mail-dmarc/issues
     + Upstream's repo last activity: https://github.com/msimerson/mail-dmarc/issues
       - last commit: in master, Oct 25, 2023
       - Issues without answer: 2
       - Updated issue/PR: Oct 25, 2023
       - last fixed/closed/merged issue: Oct 25, 2023
       - last merged PR: Oct 25, 2023
The package has one important/old open bugs on upstream: Invalid XML in generated reports, https://github.com/msimerson/mail-dmarc/issues/190
The package does not deal with exotic hardware we cannot support

[Quality assurance - testing]
The package runs a test suite on build time, if it fails
it makes the build fail: https://launchpadlibrarian.net/677780063/buildlog_ubuntu-mantic-amd64.libmail-dmarc-perl_1.20230215-1_BUILDING.txt.gz :

Creating new 'Build' script for 'Mail-DMARC' version '1.20230215'
   dh_auto_build
 /usr/bin/perl Build
Building Mail-DMARC
   dh_auto_test
 /usr/bin/perl Build test --verbose 1

The package doesn't run an autopkgtest.

[Quality assurance - packaging]
debian/watch is present and works.

debian/control defines a correct Maintainer field : Noah Meyerhans <email address hidden> ( https://git.launchpad.net/ubuntu/+source/libmail-dmarc-perl/tree/debian/control#n2)

This package does not yield massive lintian Warnings, Errors
  - recent build log of the package https://launchpadlibrarian.net/677780063/buildlog_ubuntu-mantic-amd64.libmail-dmarc-perl_1.20230215-1_BUILDING.txt.gz
  - full output from `lintian --pedantic` :
    #source
    ❯ lintian -EvIL +pedantic --show-overrides
      E: libmail-dmarc-perl changes: bad-distribution-in-changes-file noble
      W: libmail-dmarc-perl: changelog-distribution-does-not-match-changes-file unstable != noble [usr/share/doc/libmail-dmarc-perl/changelog.Debian.gz:1]
      W: libmail-dmarc-perl changes: distribution-and-changes-mismatch noble unstable
      I: libmail-dmarc-perl: typo-in-manual-page messsage message [usr/share/man/man3/Mail::DMARC::Report::Receive.3pm.gz:162]
      I: libmail-dmarc-perl: typo-in-manual-page messsage message [usr/share/man/man3/Mail::DMARC::Result.3pm.gz:184]
      P: libmail-dmarc-perl: repeated-path-segment share [usr/share/perl5/auto/share/]
      X: libmail-dmarc-perl: application-in-library-section perl [usr/bin/dmarc_lookup]
      X: libmail-dmarc-perl: application-in-library-section perl [usr/bin/dmarc_receive]
      X: libmail-dmarc-perl: application-in-library-section perl [usr/bin/dmarc_send_reports]
      X: libmail-dmarc-perl: application-in-library-section perl [usr/bin/dmarc_update_public_suffix_list]
      X: libmail-dmarc-perl: application-in-library-section perl [usr/bin/dmarc_view_reports]
      X: libmail-dmarc-perl: library-package-name-for-application [usr/bin/dmarc_lookup]
      X: libmail-dmarc-perl: library-package-name-for-application [usr/bin/dmarc_receive]
      X: libmail-dmarc-perl: library-package-name-for-application [usr/bin/dmarc_send_reports]
      X: libmail-dmarc-perl: library-package-name-for-application [usr/bin/dmarc_update_public_suffix_list]
      X: libmail-dmarc-perl: library-package-name-for-application [usr/bin/dmarc_view_reports]
    #binary
    ❯ lintian -EvIL +pedantic --show-overrides ../libmail-dmarc-perl_1.20230215-1.dsc
      I: libmail-dmarc-perl source: out-of-date-standards-version 4.6.0 (released 2021-08-18) (current is 4.6.2)
      X: libmail-dmarc-perl source: debian-watch-does-not-check-openpgp-signature [debian/watch]
      X: libmail-dmarc-perl source: libmodule-build-perl-needs-to-be-in-build-depends
      X: libmail-dmarc-perl source: update-debian-copyright 2022 vs 2023 [debian/copyright:11]

This package does not rely on obsolete or about to be demoted packages.
This package has no python2 or GTK2 dependencies.

The package will not be installed by default.

Packaging and build is easy, link to debian/rules: https://git.launchpad.net/ubuntu/+source/libmail-dmarc-perl/tree/debian/rules

[UI standards]
Application is not end-user facing (does not need translation).

[Dependencies]

As exposed at the beginning, this package needed initially +-44 MIR bugs (depending on the processing of the Recommends ones).
This is the original dependency tree, only listing on first occurrence:

 * libdbd-sqlite3-perl: universe
    MIR bug: https://bugs.launchpad.net/ubuntu/+source/libdbd-sqlite3-perl/+bug/2029379
 * libdbix-simple-perl: universe
   MIR bug: https://bugs.launchpad.net/ubuntu/+source/libdbix-simple-perl/+bug/2030731
 * libemail-mime-perl: universe
   MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-perl/+bug/2030880
   - libemail-messageid-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-messageid-perl/+bug/2030956
   - libemail-mime-contenttype-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-contenttype-perl/+bug/2030962
     + libtext-unidecode-perl: universe
        MIR bug: https://bugs.launchpad.net/ubuntu/+source/libtext-unidecode-perl/+bug/2031109
   - libemail-mime-encodings-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-encodings-perl/+bug/2031487
   - libemail-simple-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-simple-perl/+bug/2031491
 * libemail-sender-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-sender-perl/+bug/2037389
   - libemail-abstract-perl: universe
     MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-abstract-perl/+bug/2037405
     + libmodule-pluggable-perl: universe
     + libmro-compat-perl: universe
       • libclass-c3-perl: universe
         ◦ libalgorithm-c3-perl: universe
       • libclass-c3-xs-perl: universe (Recommends)
   - libmoox-types-mooselike-perl: universe
   - libscalar-list-utils-perl: universe
   - libthrowable-perl: universe
     MIR bug https://bugs.launchpad.net/ubuntu/+source/libthrowable-perl/+bug/2037392
     + libmoose-perl: universe (Recommends)
       • libclass-load-perl: universe
       • libclass-load-xs-perl: universe
       • libdevel-globaldestruction-perl: universe
       • libdevel-overloadinfo-perl: universe
       • libeval-closure-perl: universe
         ◦ libdevel-lexalias-perl: universe (Recommends)
           · libdevel-caller-perl: universe
              ·· libpadwalker-perl: universe
       • libmodule-runtime-conflicts-perl: universe
         ◦ libdist-checkconflicts-perl: universe
       • libpackage-deprecationmanager-perl: universe
         ◦ libscalar-list-utils-perl: universe
       • libdevel-partialdump-perl: universe (Recommends)
         ◦ libclass-tiny-perl: universe
 * libfile-sharedir-perl: universe
   MIR bug https://bugs.launchpad.net/ubuntu/+source/libfile-sharedir-perl/+bug/2039566
   + libclass-inspector-perl: universe
     MIR bug https://bugs.launchpad.net/ubuntu/+source/libclass-inspector-perl/+bug/2039569
 * libnet-idn-encode-perl: universe
   MIR bug https://bugs.launchpad.net/ubuntu/+source/libnet-idn-encode-perl/+bug/2038929
 * libnet-imap-simple-perl: universe (Recommends)
   - libparse-recdescent-perl: universe (Recommends)
 * libnet-ip-perl: universe
   https://bugs.launchpad.net/ubuntu/+source/libnet-ip-perl/+bug/2039456
 * libnet-smtps-perl: universe
 * libregexp-common-perl: universe
   MIR bug https://bugs.launchpad.net/ubuntu/+source/libregexp-common-perl/+bug/2039563
 * libtest-file-sharedir-perl: universe
   - libfile-copy-recursive-perl: universe
   - libfile-sharedir-perl: universe
     + libclass-inspector-perl: universe
   - libscope-guard-perl: universe
 * libtest-output-perl: universe

But, attending on how spamassassin works -explained above-, I believe we can split these dependencies in dependencies used by the reporting feature and dependencies used by validation feature. Walking this way,
the dependencies used in validation remains as needed runtime binary dependencies, and the dependencies used in reporting goes to suggested runtime dependencies. Therefore, only 11 packages would need to go
through the MIR process to satisfy libmail-dmarc-perl dependencies:

  * libemail-mime-perl: universe
    MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-perl/+bug/2030880
    - libemail-messageid-perl: universe
      MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-messageid-perl/+bug/2030956
    - libemail-mime-contenttype-perl: universe
      MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-contenttype-perl/+bug/2030962
      + libtext-unidecode-perl: universe
         MIR bug: https://bugs.launchpad.net/ubuntu/+source/libtext-unidecode-perl/+bug/2031109
    - libemail-mime-encodings-perl: universe
      MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-encodings-perl/+bug/2031487
    - libemail-simple-perl: universe
      MIR bug: https://bugs.launchpad.net/ubuntu/+source/libemail-simple-perl/+bug/2031491
  * libfile-sharedir-perl: universe
    MIR bug https://bugs.launchpad.net/ubuntu/+source/libfile-sharedir-perl/+bug/2039566
    + libclass-inspector-perl: universe
      MIR bug https://bugs.launchpad.net/ubuntu/+source/libclass-inspector-perl/+bug/2039569
  * libnet-idn-encode-perl: universe
    MIR bug https://bugs.launchpad.net/ubuntu/+source/libnet-idn-encode-perl/+bug/2038929
  * libnet-ip-perl: universe
    https://bugs.launchpad.net/ubuntu/+source/libnet-ip-perl/+bug/2039456
  * libregexp-common-perl: universe
    MIR bug https://bugs.launchpad.net/ubuntu/+source/libregexp-common-perl/+bug/2039563

a package following this scheme was built and it is available for testing on mantic here:

ppa:mirespace/libmail-dmarc-perl-suggested
https://launchpad.net/~mirespace/+archive/ubuntu/libmail-dmarc-perl-suggested

and the debdiff (for noble now) is here: https://pastebin.ubuntu.com/p/2kW5HwSsCq/ .
Note that, in this package, there is a suppression of a perl library (on main) that I suspect is not needed, as the package builds well and tests passed also. I opened this question to upstream (https://github.com/msimerson/mail-dmarc/issues/215).

Also, there is this MP (https://code.launchpad.net/~mirespace/ubuntu/+source/libmail-dmarc-perl/+git/libmail-dmarc-perl/+merge/456713) opened for tracking changes to the package. For the moment, apart from the split, we have:
- refactor for not using libemail-mime-perl (done)
- adding autopkgtest for validating the splitting (wip)

This package passes all the tests explained in the [Quality assurance - function/usage] section.

[Standards compliance]
This package correctly follows FHS and Debian Policy

[Maintenance/Owner]
Owning Team will be the Ubuntu Server Team, if they're not covered already by other teams (the Foundations team has covered maintenance at points in the past).
Team is not yet, but will subscribe to the package before promotion.
This does not use static builds.
This does not use vendored code.
This package is not rust based.

The package successfully built during the most recent test rebuild : https://launchpad.net/ubuntu/+archive/test-rebuild-20230830-mantic/+build/26600016/+files/buildlog_ubuntu-mantic-amd64.libmail-dmarc-perl_1.20230215-1_BUILDING.txt.gz

[Background information]
The Package description explains the package well.
Upstream Name is Mail-DMARC .
Link to upstream project https://metacpan.org/release/Mail-DMARC

About SSLeay (https://github.com/msimerson/mail-dmarc/issues/215) and for the moment I keep the library on the dependencies,
but libhttp-tiny-perl is not in use by the debian package because that is provided by perl package itself, so I'm wondering about it yet.

This has been in the archive since this year (Lunar, 1.20211209-1).

Bryce Harrington (bryce)
description: updated
Changed in libmail-dmarc-perl (Ubuntu):
status: Incomplete → New
Revision history for this message
Christian Ehrhardt  (paelzer) wrote (last edit ):

Actually, this shouldn't be tasks nowadays. Individual bugs would be better.
Since they are so similar all the meta-discussion can happen here.
And a link farm to reach the others is useful, see how Lucas has done pcs.

So the others can - for now - be shallow forks for tooling to work well.

Changed in libmail-dmarc-perl (Ubuntu):
assignee: nobody → Miriam España Acebal (mirespace)
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Incomplete as Miriam will prep a bit more for all the dependencies

Changed in libmail-dmarc-perl (Ubuntu):
status: New → Incomplete
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Miriam will continue spreading out the prep work by Bryce to create more individual bugs to process.
But for now the initial 6 have been opened and are ready for review (to not stall getting this started further).

We can see on those if our hope for "most should be trivial and quick perl cases" is true.

description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

AFAICS libtest-file-sharedir-perl is only used in tests:

root@m:~/libmail-dmarc-perl-1.20230215# grep -Hrn use * | grep Test::File::ShareDir
t/09.HTTP.t:7:use Test::File::ShareDir
t/03.Base.t:7:use Test::File::ShareDir
t/00.Dmarc.t:7:use Test::File::ShareDir
t/06.Result.t:7:use Test::File::ShareDir
t/26.Report.Sender.t:10:use Test::File::ShareDir
t/21.Report.Send.t:7:use Test::File::ShareDir
t/04.PurePerl.t:8:use Test::File::ShareDir
t/17.Report.Aggregate.Schema.t:7:use Test::File::ShareDir -share => { -dist => { 'Mail-DMARC' => 'share' } };
t/11.Report.Store.t:7:use Test::File::ShareDir

It is only there as it is listed manually in d/control.

That would eliminate this whole block:
 * libtest-file-sharedir-perl: universe
   - libclass-tiny-perl: universe
   - libfile-copy-recursive-perl: universe
   - libfile-sharedir-perl: universe
     + libclass-inspector-perl: universe
   - libscope-guard-perl: universe

Looking at d/control:

Package: libmail-dmarc-perl
Architecture: all
Depends: ${misc:Depends},
         ${perl:Depends},
         libconfig-tiny-perl,
         libio-socket-ssl-perl,
         libfile-sharedir-perl,
         libnet-dns-perl,
         libnet-idn-encode-perl,
         libnet-ip-perl,
         libnet-ssleay-perl,
         libemail-mime-perl,
         libtest-file-sharedir-perl,
         libemail-sender-perl,
         libdbix-simple-perl,
         libdbd-sqlite3-perl,
         libtest-output-perl,
         libregexp-common-perl,
         libsocket6-perl,
         liburi-perl,
         libxml-libxml-perl,
         publicsuffix,

That is a long list of manual depends which might be wrong.

I think there are two tasks here, now seeming like work but potentially eliminating a lot more work.
0. check code for candidates e.g.
   # Install libs listed
   $ apt install lib...
   $ dpkg -L lib...
   # find its include path like
   # /usr/share/perl5/Config/Tiny.pm
   # Check for usage of that

Example of a potentially real dependency:
# grep -Hrn use * | grep Config::Tiny
lib/Mail/DMARC/Base.pm:8:use Config::Tiny;

1. Remove all the manual dependencies, see what just ${perl:Depends} detects
2. Use libmail-dmarc-perl without them installed, does it still work

Once confirmed, update the dependency tree in the bug and in the package.

As you @Miriam have found yourself furthermore there is this in Install.md

NOTE: Most of the dependencies are optionally required for the DMARC reporting features. Mail::DMARC will perform validation with only these modules:

    Regexp::Common
    Config::Tiny
    File::ShareDir
    Net::DNS::Resolver
    Net::IP
    Socket6

So an approach could be a much more limited libmail-dmarc-perl in main and a separated only suggested package libmail-dmarc-perl-reporting which adds the rest. Doing this does not prevent considering to add the reporting as well in the future and is independent to the check which dependencies might be entirely non-needed.

Of these only the two here might be needed then:
 * libfile-sharedir-perl: universe
   + libclass-inspector-perl: universe

Revision history for this message
Miriam España Acebal (mirespace) wrote :
Download full text (4.0 KiB)

TL;DR We could split the dmarc package into validation (in main) and reporting (suggested to validation, in universe). Still working in checking it through spamassassin. Packages to be MIR:

     libfile-sharedir-perl
       + libclass-inspector-perl
     libnet-idn-encode-perl
     libnet-ip-perl
     libregexp-common-perl

I went throught the list of dependencies declared in the control file as exposed in the comment above, plus doing a matching of what perl modules were used in the test build suite provided in the libmail-dmarc-perl package, in the following sense:

- the module MAIL::DMARC::* does a "use" or "require" of perl modules provided by these X perl packages.
- The t/*.test test in libmail-dmarc-perl's source tree that test the above module use or require these X perl packages.

After that, I splitted the libmail-dmarc-perl package into validation functionality and reporting feature according to the above [1], wich coincides with the ones that are mentioned in the INSTALL.md for validation:

- Creating two d/*.install files for deploying MAIL::DMARC::* modules according with validation/reporting: all the DMARC::Report::* modules go to the reporting package, and associated script located in the source tree on /bin folder.
- Splitting the binary dependencies between validation functionality and reporting in the d/control file, as ${perl:Depends} is not calculating anything on its end (maybe due to [8])

     libconfig-tiny-perl,
     libfile-sharedir-perl,
  libnet-dns-perl,
     libnet-idn-encode-perl,
     libnet-ip-perl,
     libregexp-common-perl,
     libsocket6-perl,
  liburi-perl,

I used the above tests as a usability check of the split packages, doing a prove -v t/*.test with the packages installed (not at build time). First, only with the validation package, later installing also the reporting package for the t/*.Report.*t tests.

Furthermore, the doubt about all of this could be ... is enough the validation functionality of dmarc for the spamassassin plugin? Yes, with an exception that it's already optional: saving reports, that due to the comment, seems that the Store module of DMARC wasn't a submodule of DMARC::Report at the beginning of this perl implementation of DMARC.

Spamassasin dmarc plugin [2] only uses the MAIL::DMARC::PurePerl module [3] and, inside it, the validate function in particular [4]. However, is true that it also could use the save_aggregate function if the dmarc_save_reports variable is set throught mail-dmarc.ini to 1 (defaults is 0 [5], and also in the ini file [6]). That function will need the Mail::DMARC::Report::Store module and the Mail::DMARC::Report::URI module.

It would need a third verification for the splitting: to test it from spamassasin. It has also a t/dmarc.t test [7], that is skipped in building time nowadays because tests involving net are disabled (logical in our infra). I'm working on how to run this test properly or how it can be re-used for this needed check.

So, the packages to be MIR processed in the case of splitting (validation package in main, reporting package in universe and suggested to validation package) are:

     libfile-sharedir-perl
       + libclass-in...

Read more...

Revision history for this message
Miriam España Acebal (mirespace) wrote :

The first unpolished approach of splitting the package (used mainly to test the needed-to-be dependencies).
Debdiff between archive version and ppa package

Revision history for this message
Miriam España Acebal (mirespace) wrote :

Bad news... for making spamassassin's dmarc.t test working, we need the following files to allow the requires of the dmarc PurePerl module (perl -e "requires MAIL::DMARC::PurePerl;"):

lib/Mail/DMARC/Report/URI.pm
lib/Mail/DMARC/Report.pm
lib/Mail/DMARC/Report/Aggregate.pm
lib/Mail/DMARC/Report/Aggregate/Metadata.pm
lib/Mail/DMARC/Report/Send.pm
lib/Mail/DMARC/Report/Send/SMTP.pm
lib/Mail/DMARC/Report/Send/HTTP.pm
lib/Mail/DMARC/Report/Store.pm
lib/Mail/DMARC/Report/Receive.pm
lib/Mail/DMARC/Report/Aggregate/Record.pm
lib/Mail/DMARC/Report/Aggregate/Record/Identifiers.pm
lib/Mail/DMARC/Report/Aggregate/Record/Auth_Results/
lib/Mail/DMARC/Report/Aggregate/Record/Auth_Results.pm
lib/Mail/DMARC/Report/Aggregate/Record/Row.pm
lib/Mail/DMARC/Report/Aggregate/Record/Row/Policy_Evaluated.pm

plus

apt install libxml-libxml-perl
apt install libemail-mime-perl

which will introduce this needed MIR bug tree, apart from reconsidering this initial splitting by pure Report modules:

* libemail-mime-perl
   - libemail-messageid-perl
   - libemail-mime-contenttype-perl
     + libtext-unidecode-perl
   - libemail-mime-encodings-perl
   - libemail-simple-perl

 Still is 11 versus 44 to be MIR processed.

Revision history for this message
Miriam España Acebal (mirespace) wrote :

The above is still for the use of the save_aggregate function on MAIL:DMARC that is called on MAIL::DMARC:PurePerl at line 48, inside the validate function called from Mail::SpamAssassin::Plugin::DMARC
at line 322.

Thinking about what from the dmarc tests on spamassasin is totally necessary or not if we could consider the autosave report disabled by default (that's it).

Revision history for this message
Miriam España Acebal (mirespace) wrote :

Another possibility is not to split the package but to move all the dependencies related to Reporting features from binary dependencies to Suggested. With this, we could avoid touching dmarc or spamassassin code (importing partially subs with autoload) and supporting only validation feature, in theory.

Focussing on:

- the modules that need to be imported in all the cases (still wip on filling this bugs).
- Check the not-split-but-reporting-dependencies-suggested way.

Next step: checking with upstream this separation on reporting/validation

description: updated
description: updated
description: updated
description: updated
description: updated
description: updated
Revision history for this message
Miriam España Acebal (mirespace) wrote :

Dear MIR approval team:

The bugs for the essential 11 perl modules to be promoted are filed. At the time I'm writing this, pending your review are the following:

 https://bugs.launchpad.net/ubuntu/+source/libfile-sharedir-perl/+bug/2039566
 https://bugs.launchpad.net/ubuntu/+source/libclass-inspector-perl/+bug/2039569
 https://bugs.launchpad.net/ubuntu/+source/libregexp-common-perl/+bug/2039563
 https://bugs.launchpad.net/ubuntu/+source/libnet-idn-encode-perl/+bug/2038929
 https://bugs.launchpad.net/ubuntu/+source/libnet-ip-perl/+bug/2039456

Three more bugs were filed in advance to the above (for libemail-sender-perl, libemail-abstract-perl and libthrowable-perl). Please don't prioritize these three over the others above.

Checking in the meantime the dependency separation on validation/reporting without splitting the code in two packages.

Thanks in advance and for your patient!

Revision history for this message
Miriam España Acebal (mirespace) wrote :

The t/dmarc.t on spamassassin passes using the libmail-dmarc-perl package that puts all the non-necessary modules for validation as suggested packages (it's on ppa:mirespace/libmail-dmarc-perl-suggested).

Following spamassassin/t/README, and commenting the line 17 on t/dmarc.t for net tests disabled (while looking for a more proper way):

 root@Mspamassasin-suggested:~/spamassassin# make test TEST_FILES="t/dmarc.t"
"/usr/bin/perl" build/mkrules --exit_on_no_src --src rulesrc --out rules --manifest MANIFEST --manifestskip MANIFEST.SKIP
mkrules: no rules updated
"/usr/bin/perl" build/preprocessor -Mvars -DVERSION="4.000000" -DPREFIX="/usr/local" -DDEF_RULES_DIR="/usr/local/share/spamassassin" -DLOCAL_RULES_DIR="/etc/mail/spamassassin" -DLOCAL_STATE_DIR="/var/lib/spamassassin" -DINSTALLSITELIB="/usr/local/share/perl/5.36.0" -DCONTACT_ADDRESS="the administrator of that system" -DRE2C_BIN="re2c" -Msharpbang -Mconditional -DPERL_BIN=""/usr/bin/perl"" -DPERL_WARN="" -DPERL_TAINT="" -m755 -isa-update.raw -osa-update
cp sa-update blib/script/sa-update
"/usr/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/sa-update
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/dmarc.t
t/dmarc.t .. Nov 3 12:36:16.977 [8195] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
t/dmarc.t .. 1/18 Nov 3 12:36:18.906 [8197] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
t/dmarc.t .. 3/18 Nov 3 12:36:21.247 [8199] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
t/dmarc.t .. 5/18 Nov 3 12:36:23.737 [8201] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
t/dmarc.t .. 7/18 Nov 3 12:36:26.153 [8203] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
t/dmarc.t .. 9/18 Nov 3 12:36:29.117 [8205] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
t/dmarc.t .. 11/18 Nov 3 12:36:32.796 [8207] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
t/dmarc.t .. 13/18 Nov 3 12:36:35.338 [8209] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
t/dmarc.t .. 15/18 Nov 3 12:36:37.374 [8211] warn: deprecated method; size() is an alias of "UDPsize()" at ../blib/lib/Mail/SpamAssassin/DnsResolver.pm line 602.
t/dmarc.t .. ok
All tests successful.
Files=1, Tests=18, 31 wallclock secs ( 0.02 usr 0.00 sys + 12.18 cusr 0.88 csys = 13.08 CPU)
Result: PASS

So I believe we can continue only with the 11 perl modules that are implied on Validation, and moving the rest to suggested.

Revision history for this message
Miriam España Acebal (mirespace) wrote (last edit ):

I'm sure we can go with this dependency splitting into validation (for binary/runtime dependencies ) and reporting (suggested dependencies): after a talk with Bryce (thanks!), passing a mail (downloaded from your inbox) through spamassassin directly and checking if the DMARC line is present in the spamassassin report:

root@Mspamassasin-suggested:~# spamc -R < message-mattermost-hey.eml | grep DMARC
-0.0 DMARC_PASS DMARC pass policy

with

root@Mspamassasin-suggested:~# apt-cache policy libmail-dmarc-perl
libmail-dmarc-perl:
  Installed: 1.20230215-1~mirespace1
  Candidate: 1.20230215-1
  Version table:
     1.20230215-1 500
        500 http://archive.ubuntu.com/ubuntu mantic/universe amd64 Packages
 *** 1.20230215-1~mirespace1 500
        500 https://ppa.launchpadcontent.net/mirespace/libmail-dmarc-perl-suggested/ubuntu mantic/main amd64 Packages
        100 /var/lib/dpkg/status

after

root@Mspamassasin-suggested:~# sudo apt install libmail-dmarc-perl=1.20230215-1~mirespace1
[...]
Suggested packages:
  libdbd-sqlite3-perl libdbix-simple-perl libemail-sender-perl libnet-imap-simple-perl libnet-server-perl libnet-smtps-perl libtest-file-sharedir-perl
  libtest-output-perl libmojolicious-perl libscalar-number-perl libxml-sax-expatxs-perl
[..]
0 upgraded, 26 newly installed, 0 to remove and 0 not upgraded.
[..]
root@Mspamassasin-suggested:~#

In a fresh installation of libmail-dmarc-perl in Mantic (without spamassassin previously installed), we will have:

0 upgraded, 149 newly installed, 0 to remove and 0 not upgraded.

and if we install the splitted-dependencies package, we will have:

0 upgraded, 66 newly installed, 0 to remove and 0 not upgraded.

I'll make the modifications for the Noble package (that would be -ubuntu1 suffixed then) and
I will update the description in this bug also for:
 - mentioning there the packages used in the DMARC validation process only that will need MIR
 - testing the package [updating some of the Quality assurance sections]

description: updated
Revision history for this message
Bryce Harrington (bryce) wrote :

Glad to see you are able to verify the DMARC entry after running spamc. Why don't you also find an email that does *not* pass DMARC to verify a negative result. If you have some spam emails on hand I'll bet at least one of them fails. Or you could create one synthetically by putting typos into the From header to a non-existent domain.

Changed in libmail-dmarc-perl (Ubuntu):
importance: Undecided → High
description: updated
description: updated
Changed in libmail-dmarc-perl (Ubuntu):
status: Incomplete → New
description: updated
Changed in libmail-dmarc-perl (Ubuntu):
assignee: Miriam España Acebal (mirespace) → nobody
Lukas Märdian (slyon)
Changed in libmail-dmarc-perl (Ubuntu):
assignee: nobody → Ioanna Alifieraki (joalif)
Revision history for this message
Miriam España Acebal (mirespace) wrote :

I forwarded the split idea (and also a patch) to Debian Maintainer (Noah Meyerhans):
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1058492

Revision history for this message
Miriam España Acebal (mirespace) wrote (last edit ):

The status of the dependencies is the following (as a recap/resume):

* libemail-mime-perl: bug 2030880 , Won't fix -> Refactored to use MIME::Entity and MIME::Parser form MIME::Tools (fordwarded upstream [1])
   - libemail-messageid-perl: bug 2030956 , In progress-> ACK by Ioanna, All todos done
   - libemail-mime-contenttype-perl: bug 2030962 , In progress -> ACK by Ioanna, All todos done
       + libtext-unidecode-perl: bug 2031109 , In progress -> ACK by Ioanna, All todos done
   - libemail-mime-encodings-perl: bug 2031487 , Fix commited -> ACK by slyion,
   - libemail-simple-perl: bug 2031491 , New -> ACK by slyon, All todos done, (on sc review sec-2674) . We still need this package although we won't use libemail-mime-perl.

* libfile-sharedir-perl: bug 2039566 , In progress -> ACK by slyon, pending 1 recommend TODO adressed, waiting on MIR-team comment (if needed).
   + libclass-inspector-perl: bug 2039569 , Fix commited-> ACK by Didier, All todos done

* libnet-idn-encode-perl: bug 2038929 , New -> Skip to review by slyon. Package replaced by libnet-libidn-perl

* libnet-ip-perl: bug 2039456 , In progress -> ACK by Ioanna, All todos done

* libregexp-common-perl: bug 2039563 , Fix commited-> ACK by Didier, All todos done

The libmail-dmarc-perl package, with all these changes (including the dependency splitting), is in [2].

[1] https://github.com/msimerson/mail-dmarc/pull/217
[2] https://launchpad.net/~mirespace/+archive/ubuntu/libmail-dmarc-perl-suggested/+sourcepub/15684761/+listing-archive-extra

description: updated
Revision history for this message
Miriam España Acebal (mirespace) wrote :

I updated the dependency tree to be promoted as a result of the replacements of:

  libemail-mime-perl -> libmime-tools-perl
  libnet-idn-encode-perl -> libnet-libidn-perl

The replacements are already in main.

We still need libemail-simple-perl (libemail-mime-perl) because it is used directly by libmail-dmarc-perl.

description: updated
Revision history for this message
Ioanna Alifieraki (joalif) wrote (last edit ):
Download full text (4.4 KiB)

Review for Source Package: libmail-dmarc-perl

[Summary]
The review is based on the package provided by Miriam here:
https://launchpad.net/~mirespace/+archive/ubuntu/libmail-dmarc-perl-suggested/

MIR team ACK under the constraint to resolve the below listed
required TODOs and as much as possible having a look at the
recommended TODOs.
This does need a security review, so I'll assign ubuntu-security, after the
required TODOs are addressed.

List of specific binary packages to be promoted to main: libmail-dmarc-perl
Specific binary packages built, but NOT to be promoted to main: <None>

Notes:
Required TODOs:
1. There 5 dependencies waiting to get in main, all of them have an ACK:
  a. libemail-simple-perl: https://bugs.launchpad.net/ubuntu/+source/libemail-simple-perl/+bug/2031491
  b. libfile-sharedir-perl: https://bugs.launchpad.net/ubuntu/+source/libfile-sharedir-perl/+bug/2039566
  c. libclass-inspector-perl: https://bugs.launchpad.net/ubuntu/+source/libclass-inspector-perl/+bug/2039569 - as a dependency of the above
  d. libnet-ip-perl: https://bugs.launchpad.net/ubuntu/+source/libnet-ip-perl/+bug/2039456
  e. libregexp-common-perl: https://bugs.launchpad.net/ubuntu/+source/libmail-dmarc-perl/+bug/2023971

2. Could you please clarify the status of the autopkgtest ?

Recommended TODOs:

3. It would be nice to have https://github.com/msimerson/mail-dmarc/pull/217 merged upstream.
4. It would be nice to have https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1058492 in debian.

[Rationale, Duplication and Ownership]
There is no other package in main providing the same functionality.
A team is committed to own long term maintenance of this package.
The rationale given in the report seems valid and useful for Ubuntu

[Dependencies]
OK:
- no -dev/-debug/-doc packages that need exclusion
- No dependencies in main that are only superficially tested requiring
  more tests now.

Problems:
- other Dependencies to MIR due to this

[Embedded sources and static linking]
OK:
- no embedded source present
- no static linking
- does not have unexpected Built-Using entries
- not a go package, no extra constraints to consider in that regard
- not a rust package, no extra constraints to consider in that regard
- Does not include vendored code

Problems: None

[Security]
OK:
- history of CVEs does not look concerning
- does not run a daemon as root
- does not use webkit1,2
- does not use lib*v8 directly
- does not expose any external endpoint (port/socket/... or similar)
- does not use centralized online accounts
- does not integrate arbitrary javascript into the desktop
- does not deal with system authentication (eg, pam), etc)
- does not deal with security attestation (secure boot, tpm, signatures)
- this makes appropriate (for its exposure) use of established risk
  mitigation features (dropping permissions, using temporary environments,
  restricted users/groups, seccomp, systemd isolation features,
  apparmor, ...)

Problems:
- does parse data formats (files [images, video, audio,
  xml, json, asn.1], network packets, structures, ...) from
  an untrusted source.
- does process arbitrary web content
- does not deal with cryptography (en-/decryption, certi...

Read more...

Changed in libmail-dmarc-perl (Ubuntu):
status: New → Incomplete
assignee: Ioanna Alifieraki (joalif) → nobody
Changed in libmail-dmarc-perl (Ubuntu):
assignee: nobody → Miriam España Acebal (mirespace)
Revision history for this message
Miriam España Acebal (mirespace) wrote (last edit ):

Hi Ioanna,

sorry, I forgot to update the bug: I finished the autopackage tests (they were OK, and I removed the task from my mind), and it can be run:

https://autopkgtest.ubuntu.com/results/autopkgtest-noble-mirespace-libmail-dmarc-perl-suggested/

  - libmail-dmarc-perl/1.20230215-1ubuntu1~mirespace7
    + ✅ libmail-dmarc-perl on noble for amd64 @ 21.02.24 11:08:26 Log️ 🗒️
    + ✅ libmail-dmarc-perl on noble for arm64 @ 21.02.24 11:08:55 Log️ 🗒️
    + ✅ libmail-dmarc-perl on noble for armhf @ 21.02.24 11:13:34 Log️ 🗒️
    + ✅ libmail-dmarc-perl on noble for i386 @ 21.02.24 11:08:16 Log️ 🗒️
    + ✅ libmail-dmarc-perl on noble for ppc64el @ 21.02.24 11:09:25 Log️ 🗒️
    + ✅ libmail-dmarc-perl on noble for s390x @ 21.02.24 11:39:48 Log️ 🗒️

Log for amd64 https://autopkgtest.ubuntu.com/results/autopkgtest-noble-mirespace-libmail-dmarc-perl-suggested/noble/amd64/libm/libmail-dmarc-perl/20240221_110826_ac1bc@/log.gz

Changed in libmail-dmarc-perl (Ubuntu):
assignee: Miriam España Acebal (mirespace) → nobody
Revision history for this message
Ioanna Alifieraki (joalif) wrote :

Hi Miriam,

Thanks for the autopackage tests.
We still need to wait for the dependencies but I'll assign this to security to get it into the pipeline.

Changed in libmail-dmarc-perl (Ubuntu):
assignee: nobody → Ubuntu Security Team (ubuntu-security)
Mark Esler (eslerm)
tags: added: sec-3890
Revision history for this message
Miha Purg (mihap) wrote :

Hi all,

During the security review I noticed that several dependencies for this package were replaced with those that already exist in main:
- Net::IDN::Encode -> Net::LibIDN [1]
- Email::MIME -> MIME::Parser & MIME::Entity [2]
To my understanding, this was done to avoid introducing unnecessary and/or duplicate functionality to main, and to mitigate a known security vulnerability in the Email::MIME library [3].

I have some concerns regarding the patch needed to replace Email::MIME which I wanted to bring up for discussion, especially in light of the fact that the vulnerability has since been confirmed fixed [4] . Although an elegant solution, the patch makes non-trivial changes to the source, and, although it passes all tests, these change have not been battle tested. Moreover, I suspect there will be some unwanted implications on maintenance and support for the modified library on the long run if upstream does not accept the proposed changes (see ongoing discussion in [5]). Should we still consider this the same library as upstream in the end and who will maintain the modified code? I'm wondering if this is still the best approach considering that the vulnerability has been fixed, and that upstream is not leaning in our direction [5].

[1]: https://bugs.launchpad.net/ubuntu/+source/libnet-idn-encode-perl/+bug/2038929
[2]: https://bugs.launchpad.net/ubuntu/+source/libemail-mime-perl/+bug/2030880
[3]: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=960062
[4]: https://github.com/msimerson/mail-dmarc/issues/216#issuecomment-1945033737
[5]: https://github.com/msimerson/mail-dmarc/pull/217

Revision history for this message
Miriam España Acebal (mirespace) wrote (last edit ):

Hi Miha,

First of all, thanks for your work on this Miha.

You've summarized it very well and you're right that moving in a different direction than upstream always involves more delicate work in the future. Upstream's discussion is ongoing [1], and we don't know the decision they could make even if we (I) provide as fast as possible what they need to take it:

- specific tests
- data (that we have from this MIR study already)
- Extending the patch: completely switching even from libemail-simple-perl for general email handling.
- and as extra and beyond: making/checking it is compliant with latest RFC DMARC/DMARCbisfor, making our proposal more robust (although it is out-of-scope of the original PR) [1.1]

since for the moment they don't find a compelling reason to change the dependency [1.2] because:

- the security issue is solved [2] (as you rightly point out too)
- they are perfectly within their rights that our main/universe distinctions are not a priority for them.

Coming back to the Requires TODO for ACK in the MIR review of libemail-mime-perl [3], the #0 is that libemail-mime-perl passes the security review: Would that package be reviewed by security to see if we can go back to it? The security review (SEC-2671) was blocked and later rejected due to the switching to libmime-tools-perl.

I have no objection to reverting back to libemail-mime-perl if:
- The libemail-mime-perl security review (SEC-2671) passes with an ACK/OK.
- TODO #2 (duplicity) can be overcome.

Thank you in advance!

[1] https://github.com/msimerson/mail-dmarc/pull/217
[1.1] https://github.com/msimerson/mail-dmarc/pull/217#issuecomment-2010843067
[1.2] https://github.com/msimerson/mail-dmarc/pull/217#issuecomment-2010862484
[2] https://github.com/msimerson/mail-dmarc/issues/216#issuecomment-1945033737
[3] https://bugs.launchpad.net/ubuntu/+source/libemail-mime-perl/+bug/2030880/comments/1

Revision history for this message
Lukas Märdian (slyon) wrote :

Thanks for the summary, Miriam!

We have to make a call between duplicated work because of two similar packages in "main". Or extra work because of carrying non-mainstream patches...

Both libemail-mime-perl and libmime-tools-perl are owned by ~ubuntu-server, so I'd like to get their opinion.

> sarnold> one of the comments on https://github.com/msimerson/mail-dmarc/pull/217 suggested that the original requirements are also requirements for spamassassin 4.0, so suddenly it feels more plausible to use the original requirements..

https://github.com/msimerson/mail-dmarc/pull/217#issuecomment-2010087005

Should we really need to support libemail-mime-perl in the future for spamassassin 4.0, it could make sense to introduce duplication in the archive.

Revision history for this message
Miha Purg (mihap) wrote :
Download full text (9.6 KiB)

I reviewed libmail-dmarc-perl 1.20230215-1 as checked in noble. This shouldn't be considered a full
audit but rather a quick gauge of maintainability.

libmail-dmarc-perl is a Perl module implementing DMARC. It can be used by:
- MTAs and filtering tools like SpamAssassin to **validate** that incoming messages are aligned with purported sender's policy
- email senders, to **receive DMARC reports** from other mail servers and display them via CLI and web interfaces
- MTA operators to **send DMARC reports** to DMARC author domains

The main use-case and the only reverse dependency of libmail-dmarc-perl in Ubuntu is SpamAssassin.
It requires only the validation functionality by default, thus, as was discussed in the MIR process,
unnecessary dependencies related to reporting functionality were moved to Suggested and will not be
promoted to main at this time, thus limiting the functionality and reducing attack surface in the main archive.

- CVE History
  - None
  - Github issues with potential security implications (and not related to *reporting* functionality):
    - https://github.com/msimerson/mail-dmarc/issues/121 (does not seem exploitable)
  - Dependency issues:
    - https://github.com/rjbs/Email-MIME/issues/66#issuecomment-626260473 (fixed)

- Build-Depends
  - large dependency tree when both validation and reporting functionality is
    required, including DBD::SQLite, Data::Dumper, HTTP::Tiny, IO::Socket::SSL,
    NET::DNS::Resolver, Email::Sender, NET::SSLeay, Sys::Syslog, Net::IDN::Encode...
  - the main use-case (SpamAssassin) only uses the validation functionality,
    which significantly reduced the dependency count (Regexp::Common, Config::Tiny,
    File::ShareDir, Net::DNS::Resolver, Net::IP, Net::IDN::Encode).The original MIR
    proposes splitting dependencies into main (validation) and universe (reporting).
  - Suggested change for including in main is to replace `Net::IDN::Encode` (pure perl
    implementation of IDN, last update 2018, unresponsive maintainer -
    https://github.com/cfaerber/Net-IDN-Encode/pull/11#issuecomment-1919484551)
    with `Net::LibIDN` (perl binding for gnu-libidn, last update 2009, last update for
    gnu-libidn 2024) to avoid adding duplicate functionality to main. This replaces an
    unmaintained single-purpose library for a maintained and common library, but memory
    unsafe, which is an acceptable tradeoff.
  - Suggested change for including in main is to replace `Email::MIME` (last update 2023-1,
    several open issues) with `MIME::Tools` (last update 2024-02, infrequent updates,
    many very old open issues), in an attempt to fix a DoS issue, requiring non-trivial
    modifications to source code (see previous comment in MIR bug report).
- pre/post inst/rm scripts
  - none
- init scripts
  - none
- systemd units
  - none
- dbus services
  - none
- setuid binaries
  - none
- binaries in PATH
  - Several cmdline tools for looking up, receiving, and sending DMARC reports.
    The threat model is the same as for the library itself (reporting functionality)
    and depends on the user context. The tools should ideally not be run by a privileged user.
    - rwxr-xr-x root/root 2212 2023...

Read more...

Changed in libmail-dmarc-perl (Ubuntu):
status: Incomplete → In Progress
assignee: Ubuntu Security Team (ubuntu-security) → nobody
Revision history for this message
Bryce Harrington (bryce) wrote :

> We have to make a call between duplicated work because of two similar packages in "main". Or extra work because of carrying non-mainstream patches...
>
> Both libemail-mime-perl and libmime-tools-perl are owned by ~ubuntu-server, so I'd like to get their opinion.

From a ~ubuntu-server perspective, perl packages are very, very low maintenance work, so having two similar packages is not a problem in the slightest since they sync from debian without needing any attention at all. Maintaining some non-mainstream patches going forward would be a small bit of work but absolutely acceptable if that is desired.

Revision history for this message
Miriam España Acebal (mirespace) wrote :
Download full text (5.0 KiB)

After Miha did the security review, he found that dmarc_receive was failing. It wasn't covered here as dmarc's binary scripts are not used by spamassassin, but I thought it was worth checking.

The steps to reproduce (thanks Miha!) were:

1) Create dmarc xml based on google's example

2) Convert to gzipped base64

$ cat dmarc_example.xml | gzip | base64 > dmarc_example.xml.gz.base64

3) Copy base64 into example email (test_email.eml) from https://datatracker.ietf.org/doc/html/rfc7489#appendix-B.5

4) Run dmarc_receive

$ dmarc_receive --file test_mail.eml &> out.txt

The bad output contains "Can't locate object method "getline" via package "From: <email address hidden> ..."

I caught the bug and amended it:

diff -Nru libmail-dmarc-perl-1.20230215/debian/patches/use-MIME-Entity-and-MIME-Parser-from-libmime-tools-p.patch libmail-dmarc-perl-1.20230215/debian/patches/use-MIME-Entity-and-MIME-Parser-from-libmime-tools-p.patch
--- libmail-dmarc-perl-1.20230215/debian/patches/use-MIME-Entity-and-MIME-Parser-from-libmime-tools-p.patch 2023-12-11 14:43:31.000000000 +0000
+++ libmail-dmarc-perl-1.20230215/debian/patches/use-MIME-Entity-and-MIME-Parser-from-libmime-tools-p.patch 2023-12-11 14:43:31.000000000 +0000
@@ -125,7 +125,7 @@
 - foreach my $part ( Email::MIME->new( $email->as_string )->parts ) {
 - my ($c_type) = split /;/, $part->content_type || '';
 + my $parser = MIME::Parser->new;
-+ foreach my $part ( $parser->parse( $email->as_string )->parts_DFS ) {
++ foreach my $part ( $parser->parse_data( $email->as_string )->parts_DFS ) {
 + next if defined(!$part->bodyhandle); # something to process
 + my ($c_type) = split /;/, $part->effective_type || '';
          next if $c_type eq 'text/plain';

dmarc building tests are passing [1] and also autopkgtests that cover the splitting:

autopkgtest [14:02:18]: test splitting-check: [-----------------------
autopkgtest [14:03:08]: test splitting-check: -----------------------]
autopkgtest [14:03:09]: test splitting-check: - - - - - - - - - - results - - - - - - - - - -
splitting-check PASS
autopkgtest [14:03:10]: @@@@@@@@@@@@@@@@@@@@ summary
splitting-check PASS

Spamassassin test for dmarc also passes:

root@NDmarc-spamassassin-tests:~/spamassassin# make test TEST_FILES="t/dmarc.t"
"/usr/bin/perl" build/mkrules --exit_on_no_src --src rulesrc --out rules --manifest MANIFEST --manifestskip MANIFEST.SKIP
mkrules: no rules updated
"/usr/bin/perl" build/preprocessor -Mvars -DVERSION="4.000000" -DPREFIX="/usr/local" -DDEF_RULES_DIR="/usr/local/share/spamassassin" -DLOCAL_RULES_DIR="/etc/mail/spamassassin" -DLOCAL_STATE_DIR="/var/lib/spamassassin" -DINSTALLSITELIB="/usr/local/share/perl/5.38.2" -DCONTACT_ADDRESS="the administrator of that system" -DRE2C_BIN="re2c" -Msharpbang -Mconditional -DPERL_BIN=""/usr/bin/perl"" -DPERL_WARN="" -DPERL_TAINT="" -m755 -isa-update.raw -osa-update
cp sa-update blib/script/sa-update
"/usr/bin/perl" -MExtUtils::MY -e 'MY->fixin(shift)' -- blib/script/sa-update
PERL_DL_NONLAZY=1 "/usr/bin/perl" "-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef *Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t...

Read more...

Revision history for this message
Miriam España Acebal (mirespace) wrote (last edit ):

Regarding the debate about libmime-tools-perl vs libemail-mime-perl, I checked that libemail-mime-perl is not being used in the last version of spamassassin directly:

https://salsa.debian.org/debian/spamassassin/-/blob/master/debian/control?ref_type=heads

Also, using the example of dmarc_receive in the previous comment, using libmime-tools-perl is enough to make it work but, if we use libemail-mime-perl, we would need to MIRs :
  - libdbd-sqlite3-perl bug 2029379 : ready for promotion
  - libdbix-simple-perl bug 2030731: needs to move some dependencies from Recommends to Suggests for unblock the MIR and check usability

I checked it by using libemail-mime-perl as suggested in dmarc's dependencies - it was one of the last checks I did for validating the change to libmime-tools-perl before libemail-mime-perl passed the security review-. I had to install by hand all that is needed to run the dmarc_receive tool (the libdb* packages above).

I'm not saying it's time to review all the binary Perl scripts since they were considered outside the validation feature of DMARC and are not in use by spamassassin, but it's a minor collateral win that some of them could work, too, with the changes already made to the dmarc Perl package.

Revision history for this message
Lukas Märdian (slyon) wrote :

Setting status back to "Incomplete" as there are still discussions happening and we don't want it to get lost from the MIR report.

Also see update in https://github.com/rjbs/Email-MIME/issues/66#issuecomment-2024085120

Changed in libmail-dmarc-perl (Ubuntu):
status: In Progress → Incomplete
Revision history for this message
Miriam España Acebal (mirespace) wrote (last edit ):

Version of spamassassin using dmarc has been uploaded to [1] . I'm opening a FFe for spamassassin for this.

Using the libmail-dmarc-perl package that has been ack to be MIRed that is at [2], we can confirm the installation of both and the packages to finally be promoted (that coincides with the summary at the beginning of the description):

#1 Checking smapassassin and libmail-dmarc-perl version to be installed if we do am 'apt install spamassassin' in Noble with the spamassassin package in PPA for FFe.

root@Ndmarc-final:~# apt-cache policy libmail-dmarc-perl
libmail-dmarc-perl:
  Installed: (none)
  Candidate: 1.20230215-1ubuntu1~mirespace10
  Version table:
     1.20230215-1ubuntu1~mirespace10 500
        500 https://ppa.launchpadcontent.net/mirespace/libmail-dmarc-perl-suggested/ubuntu noble/main amd64 Packages
     1.20230215-1 500
        500 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages
root@Ndmarc-final:~# apt-cache policy spamassassin
spamassassin:
  Installed: (none)
  Candidate: 4.0.0-8ubuntu5
  Version table:
     4.0.0-8ubuntu5 500
        500 https://ppa.launchpadcontent.net/mirespace/spamassassin-dmarc-mir/ubuntu noble/main amd64 Packages
     4.0.0-8ubuntu4 500
        500 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages

#2 Checking packages in universe for this, to get the list to be promoted finally:

root@Ndmarc-final:~# for p in $(apt install -s spamassassin | grep Inst | cut -d' ' -f2); do echo -n ${p} && echo -n " " && apt-cache policy ${p} | grep archive.ubuntu | cut -d'/' -f5 | cut -d' ' -f1 ; done | grep universe

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

libclass-inspector-perl universe
libemail-simple-perl universe
libfile-sharedir-perl universe
libnet-ip-perl universe
libregexp-common-perl universe
libmail-dmarc-perl universe

[1] https://launchpad.net/~mirespace/+archive/ubuntu/spamassassin-dmarc-mir
[2] https://launchpad.net/~mirespace/+archive/ubuntu/libmail-dmarc-perl-suggested

Revision history for this message
Miriam España Acebal (mirespace) wrote :
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Thanks for also ensuring an FFE.

All MIR requested modifications have been implemented, thereby this is now ready to go and seen in component mismatches already.
Updating the state.

Changed in libmail-dmarc-perl (Ubuntu):
status: Incomplete → Fix Committed
Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

The list of mismatches in proposed matches the expected set, all of those cases are ready from the MIR and security POV. Furthermore there is a FFE approved for the same context.

I've ensured our team subscriptions are complete (they already are) on each of those.

After the next run of component mismatches hopefully confirming all this I will promote them to clear -proposed.

Revision history for this message
Christian Ehrhardt  (paelzer) wrote :

Full stack ready, all dependencies seen in component mismatches, FFE approved, MIR approved, only the top level change pulling this in is in -proposed and ready there other than this mismatch -> promoting.

Override component to main
libmail-dmarc-perl 1.20230215-1ubuntu1 in noble: universe/misc -> main
libmail-dmarc-perl 1.20230215-1ubuntu1 in noble amd64: universe/perl/optional/100% -> main
libmail-dmarc-perl 1.20230215-1ubuntu1 in noble arm64: universe/perl/optional/100% -> main
libmail-dmarc-perl 1.20230215-1ubuntu1 in noble armhf: universe/perl/optional/100% -> main
libmail-dmarc-perl 1.20230215-1ubuntu1 in noble i386: universe/perl/optional/100% -> main
libmail-dmarc-perl 1.20230215-1ubuntu1 in noble ppc64el: universe/perl/optional/100% -> main
libmail-dmarc-perl 1.20230215-1ubuntu1 in noble riscv64: universe/perl/optional/100% -> main
libmail-dmarc-perl 1.20230215-1ubuntu1 in noble s390x: universe/perl/optional/100% -> main
Override [y|N]? y
8 publications overridden.

Changed in libmail-dmarc-perl (Ubuntu):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.