Upstream microrelease 5.7

[Athos Ribeiro]

This bug tracks the following MRE updates for the Squid package:

    kinetic (22.10): Squid 5.7
    jammy (22.04): Squid 5.7

This update includes bugfixes following the SRU policy exception defined at https://wiki.ubuntu.com/SquidUpdates.

[Upstream changes]

http://www.squid-cache.org/Versions/v5/ChangeLog.html
(kinetic: 5.6..5.7); (jammy: 5.2..5.7)

Major changes introduced in this release

- Upstream OpenSSL 3.0 support added for features that were already supported by squid. No new OpenSSL 3.0 feature support added at this time.

- Support for the libssl custom Engine feature for builds linked to OpenSSL 3.0 has been dropped. Therefore, the configuration directive ssl_engine is no longer supported for builds using OpenSSL >= 3.

Moreover, the following changes are worth mentioning for jammy, from the updates between 5.2 and 5.6:

- Fixed regression that made the default value for the esi_parser configuration directive behave differently from its documented behavior. It now correctly uses libxml2 if available and falls back to libexpat otherwise.

- Fixed unexpected dispatch of client CA certificates to https_port clients when OpenSSL SSL_MODE_NO_AUTO_CHAIN mode was on.

[Test Plan]

The proposed changes were applied in the following PPA: https://launchpad.net/~athos-ribeiro/+archive/ubuntu/squid-5.7-mre/+packages.

Build logs can be found for the builds in the PPA linked above. Here are the amd64 ones:

- https://launchpadlibrarian.net/658871035/buildlog_ubuntu-kinetic-amd64.squid_5.7-0ubuntu0.22.10.1~ppa1_BUILDING.txt.gz
- https://launchpadlibrarian.net/658870565/buildlog_ubuntu-jammy-amd64.squid_5.7-0ubuntu0.22.04.1~ppa1_BUILDING.txt.gz

All tests are passing during build time, as shown in the build log (builds would fail otherwise, see LP: #2004050).

autopkgtest results:

Results: (from http://autopkgtest.ubuntu.com/results/autopkgtest-jammy-athos-ribeiro-squid-5.7-mre/?format=plain)
  squid @ amd64:
    http://autopkgtest.ubuntu.com/results/autopkgtest-jammy-athos-ribeiro-squid-5.7-mre/jammy/amd64/s/squid/20230331_113822_0439d@/log.gz
    31.03.23 11:38:22 ✅ Triggers: squid/5.7-0ubuntu0.22.04.1~ppa1
  squid @ arm64:
    http://autopkgtest.ubuntu.com/results/autopkgtest-jammy-athos-ribeiro-squid-5.7-mre/jammy/arm64/s/squid/20230331_114446_2ae73@/log.gz
    31.03.23 11:44:46 ✅ Triggers: squid/5.7-0ubuntu0.22.04.1~ppa1
  squid @ armhf:
    http://autopkgtest.ubuntu.com/results/autopkgtest-jammy-athos-ribeiro-squid-5.7-mre/jammy/armhf/s/squid/20230331_114034_12149@/log.gz
    31.03.23 11:40:34 ❌ Triggers: squid/5.7-0ubuntu0.22.04.1~ppa1
      squid FAIL 🟥
  squid @ ppc64el:
    http://autopkgtest.ubuntu.com/results/autopkgtest-jammy-athos-ribeiro-squid-5.7-mre/jammy/ppc64el/s/squid/20230331_114232_a59e1@/log.gz
    31.03.23 11:42:32 ✅ Triggers: squid/5.7-0ubuntu0.22.04.1~ppa1
  squid @ s390x:
    http://autopkgtest.ubuntu.com/results/autopkgtest-jammy-athos-ribeiro-squid-5.7-mre/jammy/s390x/s/squid/20230331_131649_c2ce8@/log.gz
    31.03.23 13:16:49 ✅ Triggers: squid/5.7-0ubuntu0.22.04.1~ppa1

Results: (from http://autopkgtest.ubuntu.com/results/autopkgtest-kinetic-athos-ribeiro-squid-5.7-mre/?format=plain)
  squid @ amd64:
    http://autopkgtest.ubuntu.com/results/autopkgtest-kinetic-athos-ribeiro-squid-5.7-mre/kinetic/amd64/s/squid/20230331_114204_a5d29@/log.gz
    31.03.23 11:42:04 ✅ Triggers: squid/5.7-0ubuntu0.22.10.1~ppa1
  squid @ arm64:
    http://autopkgtest.ubuntu.com/results/autopkgtest-kinetic-athos-ribeiro-squid-5.7-mre/kinetic/arm64/s/squid/20230331_115207_7bca2@/log.gz
    31.03.23 11:52:07 ✅ Triggers: squid/5.7-0ubuntu0.22.10.1~ppa1
  squid @ armhf:
    http://autopkgtest.ubuntu.com/results/autopkgtest-kinetic-athos-ribeiro-squid-5.7-mre/kinetic/armhf/s/squid/20230331_114117_97eb6@/log.gz
    31.03.23 11:41:17 ❌ Triggers: squid/5.7-0ubuntu0.22.10.1~ppa1
      squid FAIL 🟥
  squid @ ppc64el:
    http://autopkgtest.ubuntu.com/results/autopkgtest-kinetic-athos-ribeiro-squid-5.7-mre/kinetic/ppc64el/s/squid/20230331_114152_68893@/log.gz
    31.03.23 11:41:52 ✅ Triggers: squid/5.7-0ubuntu0.22.10.1~ppa1
  squid @ s390x:
    http://autopkgtest.ubuntu.com/results/autopkgtest-kinetic-athos-ribeiro-squid-5.7-mre/kinetic/s390x/s/squid/20230331_123412_0f94f@/log.gz
    31.03.23 12:34:12 ✅ Triggers: squid/5.7-0ubuntu0.22.10.1~ppa1

[Regression Potential]

Upstream tests are always executed during build-time. Failures would prevent builds from succeeding.

Squid does not have many reverse dependencies. However, any upgrade is a risk to introduce breakage to other packages. Whenever a regression occurs in autopkgtests, we will investigate and provide fixes.

The two changes worth mentioning here are the ones related to the configuration directives.

First, the ssl_engine directive is being dropped for builds linked with OpenSSL >= 3 (which is the case for both jammy and kinetic), meaning squid will fail to start for installations using that configuration directive. There is no current workaround for the issue, since squid does not provide support for OpenSSL >= 3 Providers yet.

We consider this __feature__ change to be an acceptable tradeoff for this particular case since shipping this new version with actual upstream OpenSSL 3 support will reduce the risks and uncertainty around the patches being carried to add OpenSSL 3 support to this package. More upstream context on that particular change is available at https://github.com/squid-cache/squid/pull/694.

Second, the default behavior for the esi_parser configuration directive is also changing. While this is a bug fix since documentation always described the behavior being set in this MRE, users may face issues in their workflows when libxml2 starts being used. This change only applies to the jammy MRE.

[Other Info]

No CVEs are being addressed this time. Therefore, this should go through the updates pockets.

[Athos Ribeiro (athos-ribeiro)]

Note that we should wait for an approval at https://lists.ubuntu.com/archives/ubuntu-release/2023-January/005522.html before proceeding with the MRE.

[Steve Langasek (vorlon)]

In the diff for this upload, I see:

--- squid-5.6/debian/NEWS 2023-01-30 23:12:50.000000000 +0000
+++ squid-5.7/debian/NEWS 2023-03-30 10:27:09.000000000 +0000
@@ -1,3 +1,13 @@
+squid (5.7-0ubuntu0.22.10.1) kinetic; urgency=medium
+
+ The ssl_engine directive has been dropped, meaning squid will fail to start
+ for installations using that configuration directive. There is no current
+ workaround for this issue since squid does not provide support for OpenSSL
+ >= 3 Providers yet. You can find more context on that particular change at
+ https://github.com/squid-cache/squid/pull/694.
+
+ -- Athos Ribeiro <email address hidden> Thu, 06 Apr 2023 18:27:15 -0300
+

This indicates that you are breaking compatibility with certain configs that would work today with the 5.6 package, which is not acceptable in SRU.

I would accept either a package which uses a maintainer script to comment out the unsupported ssl_engine option, or a package which patches the code to make references to ssl_engine non-fatal. But we can't regress working user configs in an SRU.

[Steve Langasek (vorlon)]

An upload of squid to kinetic-proposed has been rejected from the upload queue for the following reason: "documented to break certain user configs on upgrade".

[Steve Langasek (vorlon)]

An upload of squid to jammy-proposed has been rejected from the upload queue for the following reason: "documented to break certain user configs on upgrade".

[Athos Ribeiro (athos-ribeiro)]

> This indicates that you are breaking compatibility with certain configs that would work today with the 5.6 package, which is not acceptable in SRU.

> I would accept either a package which uses a maintainer script to comment out the unsupported ssl_engine option, or a package which patches the code to make references to ssl_engine non-fatal. But we can't regress working user configs in an SRU.

Thanks for the review, Steve.

Here is a further analysis of the current squid status regarding OpenSSL 3 support and the ssh_engine directive suport.
Since it is a bit long, I will start with a short version of it:

- jammy and kinetic carry different patches to support OpenSSL3;
- jammy already includes the regression (crashes) on the ssl_engine configuration directive, kinetic does not;
- commenting out the directive on upgrades would be an improvement when a user upgrades to jammy, but a regression within kinetic, but seems to be the right approach (do we need a kinetic SRU bug?);
- the issue in question is with squid-openssl (universe).

Long version (or skip to the last paragraph):

In squid 5.7 (the version in lunar, being MRE'd through this bug), the upstream
code at
https://git.launchpad.net/ubuntu/+source/squid/tree/src/ssl/support.cc#n662 has
a macro to check the OpenSSL version. When it is >= 3 (which is our case since
jammy), it throws an exception and squid halts execution with

  exception location: support.cc(681) Initialize
  FATAL: Bungled (null) line 3: sslproxy_cert_sign signTrusted all

And the following message is logged to syslog:

  FATAL: bad configuration: Cannot use ssl_engine in Squid built with OpenSSL 3.0 or newer

In jammy, we carry a set of patches for OpenSSL 3 support which were submitted
(and later merged upstream). However, this patch set was a very early version
of what ended up being merged upstream.

The relevant bit here is in
https://git.launchpad.net/ubuntu/+source/squid/tree/debian/patches/openssl3-Switch-to-BN_rand.patch?h=ubuntu/jammy-devel#n53,
where the patch drops the support for the ssl_engine configuration directive.

When starting squid with the directive present in squid's configuration file,
it halts with:

  FATAL: Your OpenSSL has no SSL engine support

Therefore, for jammy, there is actually no regressoin regarding the ssl_engine
directive, but a slight change on how squid fails.

However, in kinetic, the patch set we carried in jammy, which was still being
discussed upstream, was dropped in favor of
https://git.launchpad.net/ubuntu/+source/squid/tree/debian/patches/0006-Fix-build-against-OpenSSL-3-0.patch?h=ubuntu/kinetic-devel,
which was proposed by Debian in
https://salsa.debian.org/squid-team/squid/-/merge_requests/20/diffs.

This patch does not change or add any macros or logic in src/ssl/support.cc,
maintaining the possibility of configuration files having the ssl_engine
directive (although with undefined behavior since OpenSSL 3 does not support)
the feature.

Hence, the proposed MREs would not introduce a regression in jammy, but would
do so for kinetic.

From a user perspective, upgrading to jammy __already__ introduces a
behaviour change for squid-openssl users who rely on the ssl_engi...

[Athos Ribeiro (athos-ribeiro)]

So, regarding jammy, and considering we are __not__ introducing a regression in jammy with this MRE because the service already crashes if the ssl_engine configuration option is present, I am wondering if we should still parse the config file and comment the ssl_engine option if that is present to enhance the experience for focal users or if I should just drop that from my MP in https://code.launchpad.net/~athos-ribeiro/ubuntu/+source/squid/+git/squid/+merge/442033 before we proceed.

ATM, I am inclined to drop the postinst and d/NEWS (documenting the ssl_engine option deprecation) changes for jammy.

Thoughts?

[Athos Ribeiro (athos-ribeiro)]

I dropped the changes related to the config file regarding the offending configuration directive, as discussed in https://code.launchpad.net/~athos-ribeiro/ubuntu/+source/squid/+git/squid/+merge/442033.

I am also closing the kinetic task here since kinetic has reached its EOSS. With this change, we no longer need to maintain two different codepaths for squid (kinetic vs jammy) and there is no regression introduced regarding the ssl_engine directive since the support for for such directive is already not present in the current jammy package.

I uploaded a new version of this MRE to jammy, which should be ready for an SRU review.

[Andreas Hasenack (ahasenack)]

> Therefore, for jammy, there is actually no regressoin regarding the ssl_engine directive,
> but a slight change on how squid fails.

(sic)

This is the key statement that I believe addresses the concerns that Steve had in comment #2

[Andreas Hasenack (ahasenack)]

Hello Athos, or anyone else affected,

Accepted squid into jammy-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/squid/5.7-0ubuntu0.22.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-jammy to verification-done-jammy. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-jammy. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

[Simon Déziel (sdeziel)]

As mentioned in LP: #1975399, I'm happy with this MRE and found no regression with it, thanks!

[Athos Ribeiro (athos-ribeiro)]

Thanks, Simon.

- Build logs look sane;
- The upstream test suite and autopkgtest are passing;
- The package installs and run simple cases correctly.

As per the designed test plan, This should be good to land.

[Launchpad Janitor (janitor)]

This bug was fixed in the package squid - 5.7-0ubuntu0.22.04.1

---------------
squid (5.7-0ubuntu0.22.04.1) jammy; urgency=medium

  * New upstream version. (LP: #2013423):
    - Fix FATAL FwdState::noteDestinationsEnd exception. (LP: #1975399)
    - Fix regression that made the default value for the esi_parser
      configuration directive behave differently from its documented behavior.
      It now correctly uses libxml2 if available and falls back to libexpat
      otherwise.
    - Fix unexpected dispatch of client CA certificates to https_port clients
      when OpenSSL SSL_MODE_NO_AUTO_CHAIN mode is on.
    - Add OpenSSL 3.0 support for features that were already supported by
      squid. No new OpenSSL 3.0 feature support added at this time.
    - The configuration directive ssl_engine is no longer recognized. Since
      this option is not implemented for the OpenSSL 3 used in Ubuntu 22.04
      LTS, this is not a functional regression. Now, instead of failing with
      "FATAL: Your OpenSSL has no SSL engine support", it fails with "FATAL:
      bad configuration: Cannot use ssl_engine in Squid built with OpenSSL 3.0
      or newer".
    - For a comprehensive list of changes, please see
      http://www.squid-cache.org/Versions/v5/ChangeLog.html.
  * d/p/close-tunnel-if-to-server-conn-closes-after-client.patch: remove
    upstreamed patch.
    [ Fixed in 5.4 ]
  * d/p/0004-Change-default-Makefiles-for-debian.patch: remove upstreamed
    patch.
    [ Fixed in 5.5 ]
  * d/p/CVE-2021-46784.patch: remove upstreamed patch.
    [ Fixed in 5.6 ]
  * d/p/CVE-2022-41317.patch: drop patch to fix typo in manager ACL.
    [ Fixed in 5.7 ]
  * d/p/CVE-2022-41318.patch: drop patch to fix NTLM decoder truncated strings.
    [ Fixed in 5.7 ]
  * d/p/openssl3-*.patch: drop downstream OpenSSL 3 support patch.
    [ Fixed in 5.7 ]
  * d/p/99-ubuntu-ssl-cert-snakeoil.patch: refresh patch.

squid (5.2-1ubuntu4.4) jammy; urgency=medium

  * Make builds fail when upstream test suite fails (LP: #2004050):
    - d/p/series: do not rely on installed binaries for build time tests.
    - d/rules: halt build upon test failures.
    - d/rules: do not include additional configuration files during
      build time tests. This would lead to test failures due to missing
      paths.
    - d/t/upstream-test-suite: use installed squid binary for
      autopkgtest config file checks.

 -- Athos Ribeiro <email address hidden> Thu, 30 Mar 2023 17:06:59 -0300

[Andreas Hasenack (ahasenack)]

The verification of the Stable Release Update for squid has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.