Comment 0 for bug 1547640

Revision history for this message
Scott Moser (smoser) wrote :

Many people run squid (squid-deb-proxy, or maas-proxy) to provide ubuntu archive mirror caching and proxying. MAAS sets this up by default for users with the 'maas-proxy' package.

On or about Friday February 19, this setup began to fail for many people.
Users would see 'apt-get update' returning 503 errors. For me, I saw 503 on security.ubuntu.com addresses.

The reason for the failure was that the squid proxy began using ipv6 addresses for instead of ipv4. The squid proxy host did not have ipv6 connectivity and thus would fail.

The fix/workaround is to add the following to your squid config:
  # http://www.squid-cache.org/Doc/config/dns_v4_first/
  dns_v4_first on

The appropriate squid config file depends on what is running squid.
  maas-proxy: /usr/share/maas/maas-proxy.conf
  squid-deb-proxy: /etc/init/squid-deb-proxy.conf

I'm not sure how this previously worked, nor what change was made.
One change that was made in this time frame was a glibc update (2.19-0ubuntu6.6 to 2.19-0ubuntu6.7) for security (CVE-2013-7423 CVE-2014-9402 CVE-2015-1472 CVE-2015-1473). But it doesn't seem to make sense that that would change squid3 to start looking for AAAA records when it did not previously.

but if you run squid on a host that has ipv6 , but no ipv6 connectivity
--> Jonathanb (textual@79-67-162-51.dynamic.dsl.as9105.com) has joined #canonical
<-- Grazina has quit (Client exited)
<smoser> it used to work and end up getting ipv4 addresses
<smoser> something changed today or yesterday
<-- bkhan_AWAY has quit (Ping timeout: 121 seconds)
<smoser> and now that is getting ipv6 addresses.
<-- xnox has quit (Quit: ZNC - http://znc.in)
<smoser> meaning if you have no ipv6 route to security.ubuntu.com, you get 503 on 'apt-get update'
<smoser> i can verify that as late as
<smoser> human time: Thu Feb 18 06:36:07 EST 2016
<-- verterok has quit (Quit: Coyote finally caught me)
--- john-mcaleely|away is now known as john-mcaleely
<smoser> i was seeing entries in my squid logs with
<-- alan_g has quit (Quit: Ex-Chat)
<smoser> human time: Thu Feb 18 06:36:07 EST 2016
<-- mup has quit (Connection closed)
<smoser> 1455713142.896 335 10.7.2.103 TCP_REFRESH_UNMODIFIED/200 82620 GET http://security.ubuntu.com/ubuntu/dists/xenial-security/InRelease - HIER_DIRECT/91.189.88.149 -
<smoser> but now i get
<smoser> 1455879482.210 1 10.7.2.103 TCP_REFRESH_FAIL/200 635 GET http://security.ubuntu.com/ubuntu/dists/precise-security/main/i18n/Index - HIER_DIRECT/2001:67c:1562::14 -