demotion of pcre3 (8.x) a.k.a pcre (without the 3) in favor of pcre2 (10.x)

[Matthias Klose]

https://people.canonical.com/~ubuntu-archive/transitions/html/pcre2-main.html

demotion of pcre3 in favor of pcre2. These packages need analysis what needs to be done for the demotion of pcre3:

Packages which are ready to build with pcre2 should be marked as 'Triaged', packages which are not ready should be marked as 'Incomplete'.

------
For clarification: pcre2 is actually newer than pcre3. pcre3 is just poorly named.

[Timo Aaltonen (tjaalton)]

sssd builds fine with libpcre2-dev

[Jeremy Bicha (jbicha)]

Debian's git has already switched

[Jeremy Bicha (jbicha)]

Debian's php7.3 has already switched

[Jeremy Bicha (jbicha)]

Debian's clamav has already switched.

[Thomas Ward (teward)]

Nginx properly builds with pcre2

[Axel Beckert (xtaran)]

zsh in Debian Unstable (5.6.2-1) seems to build fine if I exchange libpcre3-dev by libpcre2-dev in Build-Depends and Build-Using (via debian/rules).

[Noël Köthe (noel)]

wget builds and works with pcre2 in Debian (uploaded). I cannot change the Status to Triaged. Maybe somebody with higher privileges can change the wget status of this bug to Triaged. thx.

[Andreas Hasenack (ahasenack)]

I've seen the wget debian change, but just switching builddeps from pcre3-dev to pcre2-dev and rebuilding isn't enough. The package ends up not finding pcre and doesn't enable it:
checking for PCRE... no
checking pcre.h usability... no
checking pcre.h presence... no
checking for pcre.h... no
...
  Libs: -luuid -lidn2 -lnettle -lgnutls -lz -lpsl

So while the package builds, it's not using pcre.

I tried to switch apache, and while I could make it find pcre2-config and use it, pcre2 has different libraries than pcre3:
pcre3-config --libs: -lpcre
pcre2-config --libs: no such parameter

In pcre2, we have --libs8, --libs-posix, --libs32 and --libs16, but no --libs. Is this a bug in pcre2-config?

Any quick tips about this before I dig in further?

[Anders Kaseorg (andersk)]

Andreas: No, it’s not a bug. PCRE2 is a new project that’s not intended to be compatible with the older PCRE (i.e. what Debian misnamed “pcre3”). The API is completely different and this is expected. See bug 1636666 for context, and specifically the PCRE2 release announcement linked in the bug description:

https://lists.exim.org/lurker/message/20150105.162835.0666407a.en.html

That’s why getting rid of PCRE “in favor of” PCRE2 is a pointless way to frame the issue. PCRE2 neither replaces nor conflicts with PCRE, as was pointed out many times in bug 1636666.

But since we’re apparently gathering information here: although Apache 2.4.x does not support PCRE2, Apache trunk does support PCRE2 as of r1773454. (As you can see, this support was added by creating separate conditionally compiled code paths for PCRE and PCRE2.)

https://github.com/apache/httpd/commit/b1a3338e011a17ad190fb7f66cf5ca9acf353570

[Anders Kaseorg (andersk)]

apr-util has a false Build-Depends on libpcre3-dev; it should simply be dropped. A corresponding Depends was already dropped in https://bugs.debian.org/757140.

[Anders Kaseorg (andersk)]

Copying from https://bugs.launchpad.net/ubuntu/+source/pcre2/+bug/1636666/comments/20, HAProxy and SELinux upstream support PCRE2.

https://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=f2592b29f13907ddf2bba42d00bc41cb8ee5b69b
https://github.com/SELinuxProject/selinux/commit/50f0910cf05bdc1d10710c7c3fb748a178473387

[Matthias Klose (doko)]

apr-util: filed a Debian bug.

[Anders Kaseorg (andersk)]

• Aide: no PCRE2 support
• Exim: no PCRE2 support, upstream bug: https://bugs.exim.org/show_bug.cgi?id=1878
• FreeRADIUS: no PCRE2 support, mentioned in passing: https://github.com/FreeRADIUS/freeradius-server/issues/1865
• GLib: no PCRE2 support
• grep: no PCRE2 support

[Anders Kaseorg (andersk)]

• pam_mount: no PCRE2 support
• nmap: no PCRE2 support
• postfix: no PCRE2 support, http://postfix.1071664.n5.nabble.com/Plans-for-using-PCRE-v2-in-Postfix-td83200.html
• pyScss: no PCRE2 support

[Anders Kaseorg (andersk)]

• Quagga: no PCRE2 support
• Rasqal: no PCRE2 support
• S-Lang: no PCRE2 support

[Noël Köthe (noel)]

reported upstream: https://savannah.gnu.org/bugs/index.php?54677

[Timo Aaltonen (tjaalton)]

Actually, sssd build would still pull libpcre3-dev because of some dependency, and then use that... and upstream config phase doesn't support pcre2. I've filed a bug upstream.

[Anders Kaseorg (andersk)]

I think that completes the analysis of the current state of things. Although it seems infeasible to demote PCRE at this time, there’s no reason that should block the promotion of PCRE2, especially seeing as a bundled copy of PCRE2 is already in main (18.04 and 18.10) via libqt5core5a.

[Anders Kaseorg (andersk)]

Note to those filing upstream bugs: probably don’t call the old library “pcre3” since that’s not a thing outside Debian/Ubuntu.

[Timo Aaltonen (tjaalton)]

sssd: https://pagure.io/SSSD/sssd/issue/3833

[Axel Beckert (xtaran)]

Hi,

Andreas Hasenack wrote:
> I've seen the wget debian change, but just switching builddeps from pcre3-dev to pcre2-dev and rebuilding isn't enough. The package ends up not finding pcre and doesn't enable it:
> checking for PCRE... no
> checking pcre.h usability... no
> checking pcre.h presence... no
> checking for pcre.h... no
> ...
> Libs: -luuid -lidn2 -lnettle -lgnutls -lz -lpsl
>
> So while the package builds, it's not using pcre.
>
> I tried to switch apache, and while I could make it find pcre2-config and use it, pcre2 has different libraries than pcre3:
> pcre3-config --libs: -lpcre
> pcre2-config --libs: no such parameter
>
> In pcre2, we have --libs8, --libs-posix, --libs32 and --libs16, but no
> --libs. Is this a bug in pcre2-config?

Indeed, same for zsh:

checking for pcre-config... no
checking pcre.h usability... no
checking pcre.h presence... no
checking for pcre.h... no
checking for pcre_compile... no
checking for pcre_study... no
checking for pcre_exec... no
[...]
../../Test/V07pcre.ztst: starting.
../../Test/V07pcre.ztst: skipped (the zsh/pcre module is not available)

→ Back to Incomplete. I'll revert that commit from the recent 5.6.2-2
  upload.

  Regards, Axel
--
 ,''`. | Axel Beckert <email address hidden>, https://people.debian.org/~abe/
: :' : | Debian Developer, ftp.ch.debian.org Admin
`. `' | 4096R: 2517 B724 C5F6 CA99 5329 6E61 2FF9 CD59 6126 16B5
  `- | 1024D: F067 EA27 26B9 C3FC 1486 202E C09E 1D89 9593 0EDE

[Anders Kaseorg (andersk)]

Likewise, nginx does not support PCRE2: https://trac.nginx.org/nginx/ticket/720

[Thomas Ward (teward)]

Nice find, I looked in trac but must have missed that...

Sent from my Sprint Samsung Galaxy S9+.
-------- Original message --------From: Anders Kaseorg <email address hidden> Date: 9/18/18 06:22 (GMT-05:00) To: <email address hidden> Subject: [Bug 1792544] Re: demotion of pcre3 in favor of pcre2
Likewise, nginx does not support PCRE2:
https://trac.nginx.org/nginx/ticket/720

** Bug watch added: trac.nginx.org/nginx/ #720
   http://trac.nginx.org/nginx/ticket/720

** Changed in: nginx (Ubuntu)
       Status: Triaged => Incomplete

--
You received this bug notification because you are subscribed to nginx
in Ubuntu.
Matching subscriptions: nginx
https://bugs.launchpad.net/bugs/1792544

Title:
  demotion of pcre3 in favor of pcre2

Status in aide package in Ubuntu:
  Incomplete
Status in apache2 package in Ubuntu:
  New
Status in apr-util package in Ubuntu:
  New
Status in clamav package in Ubuntu:
  Triaged
Status in exim4 package in Ubuntu:
  Incomplete
Status in freeradius package in Ubuntu:
  Incomplete
Status in git package in Ubuntu:
  Triaged
Status in glib2.0 package in Ubuntu:
  Incomplete
Status in grep package in Ubuntu:
  Incomplete
Status in haproxy package in Ubuntu:
  New
Status in libpam-mount package in Ubuntu:
  Incomplete
Status in libselinux package in Ubuntu:
  New
Status in nginx package in Ubuntu:
  Incomplete
Status in nmap package in Ubuntu:
  Incomplete
Status in pcre3 package in Ubuntu:
  Confirmed
Status in php7.2 package in Ubuntu:
  Triaged
Status in postfix package in Ubuntu:
  Incomplete
Status in python-pyscss package in Ubuntu:
  Incomplete
Status in quagga package in Ubuntu:
  Incomplete
Status in rasqal package in Ubuntu:
  Incomplete
Status in slang2 package in Ubuntu:
  Incomplete
Status in sssd package in Ubuntu:
  Incomplete
Status in wget package in Ubuntu:
  Incomplete
Status in zsh package in Ubuntu:
  Incomplete

Bug description:
  demotion of pcre3 in favor of pcre2. These packages need analysis what
  needs to be done for the demotion of pcre3:

  Packages which are ready to build with pcre2 should be marked as
  'Triaged', packages which are not ready should be marked as
  'Incomplete'.

  aide
  apache2
  apr-util
  clamav
  exim4
  freeradius
  git
  glib2.0
  grep
  haproxy
  libpam-mount
  libselinux
  nginx
  nmap
  php7.2
  postfix
  python-pyscss
  quagga
  rasqal
  slang2
  sssd
  wget
  zsh

  ------

  For clarification: pcre2 is actually newer than pcre3.  pcre3 is just
  poorly named (according to jbicha).

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/aide/+bug/1792544/+subscriptions

[Jeremy Bicha (jbicha)]

glib is filed upstream as https://gitlab.gnome.org/GNOME/glib/issues/1085

[Launchpad Janitor (janitor)]

This bug was fixed in the package apr-util - 1.6.1-2ubuntu1

---------------
apr-util (1.6.1-2ubuntu1) cosmic; urgency=medium

  * Drop build dependency on libpcre3-dev. Closes: #909077. LP: #1792544.

 -- Matthias Klose <email address hidden> Tue, 18 Sep 2018 13:05:27 +0200

[Thomas Ward (teward)]

NGINX has technically said this is won't fix:

From a response to my inquiry: http://mailman.nginx.org/pipermail/nginx-devel/2018-September/011448.html

------

Hello!

On Tue, Sep 18, 2018 at 08:12:20AM -0400, Thomas Ward wrote:

> Downstream in Ubuntu, it has been proposed to demote pcre3 and
> use pcre2 instead as it is newer.
> https://trac.nginx.org/nginx/ticket/720 shows it was marked 4
> years ago that NGINX does not support pcre2. Are there any
> plans to use pcre2 instead of pcre3?

There are no immediate plans.

When we last checked, there were no problems with PCRE, but PCRE2
wasn't available in most distributions we support, making the
switch mostly meaningless.

Also, it looks like PCRE2 is still not supported even by Exim,
which is the parent project of PCRE and PCRE2:

https://bugs.exim.org/show_bug.cgi?id=1878

As such, adding PCRE2 support to nginx looks premature.

--
Maxim Dounin
http://mdounin.ru/

[Christian Ehrhardt  (paelzer)]

Summarizing the Server Team related task status for a better overview.

For postfix a discussion existed, and they seem not to have any better tracker set up.
For the others I reported new issues upstream, so that they are aware and we have something to track.
I listed those new ones after "---".

Ready Packages since Bug/Commit:
haproxy: https://git.haproxy.org/?p=haproxy.git;a=commitdiff;h=f2592b29f13907ddf2bba42d00bc41cb8ee5b69b
php7.3: https://salsa.debian.org/php-team/php/commit/fbe3eb003c4bc3d81414880b923e276c9b7acb56

Open Upstream Bugs:
apache2: https://bz.apache.org/bugzilla/show_bug.cgi?id=57471
exim4: https://bugs.exim.org/show_bug.cgi?id=1878
nginx: https://trac.nginx.org/nginx/ticket/720
sssd: https://pagure.io/SSSD/sssd/issue/3833
---
postfix: http://postfix.1071664.n5.nabble.com/Plans-for-using-PCRE-v2-in-Postfix-td83200.html
nmap: https://github.com/nmap/nmap/issues/1335
freeradius: https://github.com/FreeRADIUS/freeradius-server/issues/2315
quagga: https://bugzilla.quagga.net/show_bug.cgi?id=997

[Christian Ehrhardt  (paelzer)]

FYI
NMAP
Started to consider pcre2 due to the bug, but they have zillions of regrex use cases and it would be a major effort that doesn't seem to happen anytime soon.

Freeradius implemented pcre2 due to our bug in https://github.com/FreeRADIUS/freeradius-server/commit/9a97b9c2fbc84f9f13c127c2a78b5808de2b6ae2
This is so far only valid for 4.x which isn't released - might be backported to 3.0.18 if upstream wants to do so. Therefore a huge step, but not enough yet to call it triaged here.

[Jeremy Bicha (jbicha)]

https://launchpad.net/debian/+source/libpam-mount/2.16-8

[Jeremy Bicha (jbicha)]

libpam-mount wasn't actually switched to pcre2. I filed a Debian bug to let its maintainer know.

[Jeremy Bicha (jbicha)]

libpam-mount 2.16-9 uses pcre2 (for real this time)

[Jeremy Bicha (jbicha)]

systemd uses pcre2 for a useful journal feature that we can't enable until pcre2 is in main LP: #1751006

[Launchpad Janitor (janitor)]

This bug was fixed in the package libpam-mount - 2.16-9ubuntu2

---------------
libpam-mount (2.16-9ubuntu2) disco; urgency=medium

  * Revert remaining Ubuntu changes now that pcre2 is in main (LP: #1792544)

 -- Jeremy Bicha <email address hidden> Wed, 23 Jan 2019 11:19:59 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package clamav - 0.100.2+dfsg-2ubuntu1

---------------
clamav (0.100.2+dfsg-2ubuntu1) disco; urgency=medium

  * Sync with Debian. Remaining change:
    - clamav-daemon may fail to start due to options removed in new version
      and manually edited configuration file. (LP: #1783632)
      + debian/patches/Deprecate-unused-options-instead-of-removing-it.patch:
        add patch from Debian stretch to simply warn about removed options.
  * Dropped change:
    - Build-Depend on pcre3-dev (pcre2 is now in Ubuntu main) (LP: #1792544)

clamav (0.100.2+dfsg-2) unstable; urgency=medium

  * Increase clamd socket command read timeout to 30 seconds (Closes: #915098)

clamav (0.100.2+dfsg-1) unstable; urgency=medium

  * Import new upstream
    - Bump symbol version due to new version.
    - CVE-2018-15378 (Closes: #910430).
  * add NEWS.md and README.md from upstream
  * Fix infinite loop in dpkg-reconfigure, Patch by Santiago Ruano Rincón
    (Closes: #905044).

 -- Jeremy Bicha <email address hidden> Wed, 23 Jan 2019 12:08:48 -0500

[Jeremy Bicha (jbicha)]

quagga was another false Build-Depends so I filed a Debian bug requesting they drop the pcre3 Build-Depends.

[Jeremy Bicha (jbicha)]

libslang2-modules is in universe although it's built from the slang2 source. I guess that means we can mark it invalid here?

[Launchpad Janitor (janitor)]

This bug was fixed in the package wget - 1.20.1-1ubuntu3

---------------
wget (1.20.1-1ubuntu3) disco; urgency=medium

  * Revert switching to pcre3 since pcre2 is in main (LP: #1792544)

 -- Jeremy Bicha <email address hidden> Wed, 23 Jan 2019 11:56:34 -0500

[Launchpad Janitor (janitor)]

This bug was fixed in the package git - 1:2.20.1-2ubuntu1

---------------
git (1:2.20.1-2ubuntu1) disco; urgency=medium

  * Merge with Debian; remaining change:
    - Build diff-highlight in the contrib dir (closes: #868871, LP: #1713690)
  * Dropped change:
    - Build against pcre3 (pcre2 is now in main) (LP: #1792544)

git (1:2.20.1-2) unstable; urgency=low

  * package git-gui: actually Suggests: meld for mergetool support;
    describe what meld is used for in package description (thx Jens
    Reyer; closes: #707790).
  * package gitweb: Depends: libhttp-date-perl | libtime-parsedate-perl
    instead of ... | libtime-modules-perl (thx gregor herrmann; closes:
    #879165).
  * debian/control: use https in Vcs-Browser URL.
  * debian/rules: build and test quietly if DEB_BUILD_OPTIONS=terse.
  * debian/control: Standards-Version: 4.3.0.1.

git (1:2.20.1-1) unstable; urgency=medium

  * new upstream point release (see RelNotes/2.20.1.txt).
  * package git-gui: Suggests: meld for mergetool support (thx Jens
    Reyer; closes: #707790).

git (1:2.20.0-1) unstable; urgency=medium

  * new upstream release (see RelNotes/2.20.0.txt).
  * package git: Recommends: ca-certificates for https support (thx HJ;
    closes: #915644).

git (1:2.20.0~rc2-1) unstable; urgency=low

  * new upstream release candidate.
    * rebase: specify 'rebase -i' in reflog for interactive rebase
      (closes: #914695).

git (1:2.20.0~rc1-1) unstable; urgency=low

  * new upstream release candidate (see RelNotes/2.20.0.txt).
  * debian/rules: target clean: don't remove t/t4256/1/mailinfo.c.orig.

git (1:2.19.2-1) unstable; urgency=high

  * new upstream point release (see RelNotes/2.19.2.txt).
    * run-command: do not fall back to cwd when command is not in $PATH.

 -- Jeremy Bicha <email address hidden> Wed, 23 Jan 2019 11:30:48 -0500

[Egmont Koblinger (egmont-gmail)]

Please do not forget to remove the pcre patch from Tilix, see https://bugs.launchpad.net/ubuntu/+source/tilix/+bug/1818991 - thanks!