after upgrade from oneiric to precise, previous squid config unused, cannot be used when relocated

[Jean-Baptiste Lallement]

After upgrade from Oneiric to Precise squid is misconfigured and returns only 403

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: squid3 3.1.18-1ubuntu1
ProcVersionSignature: Ubuntu 3.2.0-12.20-generic 3.2.2
Uname: Linux 3.2.0-12-generic x86_64
NonfreeKernelModules: fglrx
ApportVersion: 1.91-0ubuntu1
Architecture: amd64
Date: Wed Feb 1 10:04:35 2012
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.UTF-8
 SHELL=/bin/bash
SourcePackage: squid3
UpgradeStatus: Upgraded to precise on 2012-01-31 (0 days ago)

[Ubuntu QA Website (ubuntuqa)]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/924739

[Adam Gandelman (gandelman-a)]

Jean-Baptiste, thanks for reporting this. Can you provide the squid.conf that you end up with after upgrade that is causing the problems? Also, was this an upgrade from squid3 (oneiric) to squid3 (precise), or a transition from squid (oneiric) to squid3 (precise)?

[Jean-Baptiste Lallement (jibel)]

This is a transition from squid2.7 to squid3.

Here are the upgrade logs

[Jean-Baptiste Lallement (jibel)]

squid.conf
It was modified to use apt-cacher-ng to cache .deb packages.

[Jean-Baptiste Lallement (jibel)]

and content of /etc/squid3/

[Adam Gandelman (gandelman-a)]

Confirmed that, with the given config, a transition to squid3 results in 403s as described

[Adam Gandelman (gandelman-a)]

@Jean-Baptiste:

The issue is that precise has transitioned to squid3. The new location for squid.conf is /etc/squid3/squid.conf. After upgrade, squid(3) was restarted using the default packaged config that was installed at /etc/squid3/squid.conf after update. Replacing /etc/squid3/squid.conf with yours, I squid3 restarted and read the correct config, however, there were still some issues:

# squid3 -N
2012/02/03 19:26:02| WARNING: (B) '::/0' is a subnetwork of (A) '::/0'
2012/02/03 19:26:02| WARNING: because of this '::/0' is ignored to keep splay tree searching predictable
2012/02/03 19:26:02| WARNING: You should probably remove '::/0' from the ACL named 'all'
2012/02/03 19:26:02| ERROR: Directive 'upgrade_http0.9' is obsolete.
2012/02/03 19:26:02| cache_cf.cc(381) parseOneConfigFile: squid.conf:2993 unrecognized: 'broken_vary_encoding'
2012/02/03 19:26:02| ERROR: Directive 'extension_methods' is obsolete.

IIRC, 'broken_vary_encoding' has been removed from valid config options in squid3. Commenting out line 2993 your config (now located at /etc/squid3/squid.conf) got squid back up and running as a functional proxy.

I'm going to update bug topic and raise to Critical because transition problems like these are going to cause considerable pain for anyone dependent on a squid for normal operation.

[Adam Gandelman (gandelman-a)]

The following is a list of configuration options from 2.7 that have not been ported to squid3, as of 3.19:

http://squidcache.cybermirror.org/squid/squid-3.1.19-RELEASENOTES.html#ss6.1

There are many. Currently, squid3 will fail to parse config + start if any of these yet-to-be-ported options exists in config. So, while we can work around the default config file location in packaging, there is still two issues:

1. squid3 cannot parse the default config that ships with squid(v2). There is a config directive 'broken_vary_encoding' included there that will cause squid3 bailout on startup. If there config file has been modified by user (most likely), any of the options described in the document linked above will cause squid to fail.

2. Even if we patch squid to only warn on these config issues and continue startup, the user expectations will be unmet in terms of functionality. If a user has come to depend on any of the yet-to-be-ported features, their proxy will be broken or, at the least, behave differently post-transition.

[Adam Gandelman (gandelman-a)]

Matters are slightly more complicated because the existing squid packages (both the original squid(v2) and squid3) do not track squid.conf as a conffile, probably because the example configuration is generated at build time.

After discussing the matter with several Ubuntu devs, it appears there are two possible solutions:

1. On upgrade, if the user had modified the original squid(2) configuration, raise a debconf dialog box that very loudly warns that the config may not be compatable after upgrade.

2. On upgrade, if the user has modified the original squid(2) configuration, transfer it to the new squid3 location and attempt to start squid3. If the config is compatable (that is, does not contain unsupported config options), squid3 should start successfully. If the user was using squid(2) features that are unsupported in squid3, the proxy will fail to start and cause upgrade errors.

3. Regardless of whether or not previous squid configuration has been modified, use the default squid3 configuration in all cases and start a unconfigured proxy. Perhaps backup the original squid for reference.

In either case, matters are complicated by the fact that the example squid.conf shipped with either package is not tracked as a conffile by dpkg (it is generated at build time) To do any conffile handling mentioned above, we'll need to hard-code a list of hash sums in maintainer scripts that cover all previous releases of the squid package, and perhaps maintain this list over time if updates are SRU'd.

The real question (as slangasek pointed out) is whether its better to have a broken proxy after dist-upgrade or an uncofigured proxy.

[Drew Scott Daniels (drewdaniels)]

Some options could be translated.
http://www.squid-cache.org/Doc/config/ shows what's compatible and what's not.

The release notes list which tags have been added, changed and removed. E.g.
header_access was replaced by request_header_access and reply_header_access so the one tag becomes two for more fine grained control.

A pointer to the Squid release notes is what Red Hat did for RHEL 6.0's notes:
http://docs.redhat.com/docs/en-US/Red_Hat_Enterprise_Linux/6/html/Migration_Planning_Guide/ch04s12.html
And I think it was referenced elsewhere too.

     Drew Daniels
Blog: http://www.boxheap.net/ddaniels/blog/

[Launchpad Janitor (janitor)]

This bug was fixed in the package squid3 - 3.1.19-1ubuntu2

---------------
squid3 (3.1.19-1ubuntu2) precise; urgency=low

  * debian/NEWS.Debian: Rename NEWS.debian, add note regarding squid3
    transition in 12.04 (LP: #924739)
 -- Adam Gandelman <email address hidden> Thu, 12 Apr 2012 13:46:10 -0700