BADSIG errors using transparent http proxies

[Stuart Bishop]

Users forced to use substandard proxies, such as many users in Thailand, regularly get signature validation errors with apt-get update and all the tools that rely on apt such as update-manager and synaptic.

W: GPG error: http://security.ubuntu.com breezy-security Release: The following
signatures were invalid: BADSIG 40976EAF437D05B5 Ubuntu Archive Automatic Signing Key <email address hidden>

[Stuart Bishop (stub)]

Stderr from running "sudo apt-get -o Acquire::BrokenProxy=true update -o Debug::Acquire::http=true 2> /tmp/apt-update.log"

[Michael Vogt (mvo)]

Thanks for your bugreport.

One option you can turn on is

apt-get update -o Acquire::http::No-Cache=True

this will set the cache-control headers to turn caching off. Please let mek know if that helps.

Thanks,
 Michael

[Stuart Bishop (stub)]

Michael Vogt wrote:

> One option you can turn on is
>
> apt-get update -o Acquire::http::No-Cache=True
>
> this will set the cache-control headers to turn caching off. Please let
> mek know if that helps.

I just tried a standard apt-get update, and it failed with the BADSIG
errors. I immediately then tried with the Acquire::http::No-Cache=True
option and it worked fine.

Let me know if you need me to run any experiments from my end - I suspect
this issue is fairly common and has the side effect of delaying or blocking
security updates, so it would be good if this situation could be detected
and the requests sent with the correct headers. Perhaps an optional file in
each archive containing an automatically updated timestamp? If the timestamp
is retrieved and it is too far out of date, we know we are behind a
particularly aggresive cache and we need to send the No-Cache header.

--
Stuart Bishop <email address hidden> http://www.canonical.com/
Canonical Ltd. http://www.ubuntu.com/

[Andrew Jorgensen (ajorg)]

I have been getting these errors regularly as well. I will try the options mentioned but I would like to know if the OP can confirm that the NoCache option fixed the problem for him. My experience with the bug has been that it happens the first time I run apt-get update but never the second time. From the OP's most recent post it sounds like it may have only looked like the NoCache option fixed the problem.

[Stuart Bishop (stub)]

To do updates, I currently am running ' sudo apt-get -o Acquire::http::No-Cache=True -o Acquire::BrokenProxy=true update' twice. The first invariably fails. The second always works.

Proxies are mandatory here in Thailand due to Internet censorship laws, and I'm connected via the largest broadband provider, so this affects a few million potential Ubuntu users.

[Michael Vogt (mvo)]

Thanks for this additional feedback and sorry for my late reply.

I looked at the code and rfc2616 again and will attach a patch that may fix the problem. Testing is welcome, I'm happy to build a test-version of apt for you and put it on people.ubuntu.com.

It will always set Cache-Control: max-age=0 for indexfiles. rfc2616 is clear what this means:
"""
The request includes "max-age=0" cache-control directive, which forces each cache along the path to the origin server to revalidate its own entry, if any, with the next cache or server.
"""

I have to admit that I'm puzzled that Acquire::http::No-Cache=True dosn't work (on the first run). The rfc is pretty clear here:
"""
The request includes a "no-cache" cache-control directive or, for compatibility with HTTP/1.0 clients, "Pragma: no-cache". Field names MUST NOT be included with the no-cache directive in a request. The server MUST NOT use a cached copy when responding to such a request.
"""

Could you please make a dump (with -o Debug::Acquire::http=true) of the first and second run? And attach it here?

Thanks,
 Michael

[Michael Vogt (mvo)]

This patch will always set cache control headers (for transparent proxies). It will set max-age=0 for all indexfiles too.

[Stuart Bishop (stub)]

This is the output from the first run of:

sudo apt-get -o Acquire::http::No-Cache=True -o Debug::Acquire::http=true update 2>&1 | tee firstrun.txt

[Stuart Bishop (stub)]

This is the output from the second run of:

sudo apt-get -o Acquire::http::No-Cache=True -o Debug::Acquire::http=true update 2>&1 | tee secondrun.txt

[Michael Vogt (mvo)]

Thanks for the new logs. Could you please give the follow package a try?

Please add this to your sources.list:
"""
deb http://people.ubuntu.com/~mvo/apt/transparent-proxy/ /
"""

and update, upgrade. It should install a new apt (version 0.6.43.3ubuntu2.1)

Please let me know if that package works better. Please try first without the -o Acquire::http::No-Cache=true option and then with it.

Thanks,
 Michael

[Stuart Bishop (stub)]

Run of the April 25th test apt from mvo - output of:

    sudo apt-get -o Debug::Acquire::http=true update 2>&1

[Stuart Bishop (stub)]

Output from the April 25th apt package by mvo of:

sudo apt-get -o Debug::Acquire::http=true update -o Acquire::http::No-Cache=true 2>&1

[Michael Vogt (mvo)]

I updated to version apt 0.6.43.3ubuntu2.2.

Can you please upgrade to this version and run it with:

"""
sudo apt-get -o Debug::Acquire::http=true -o Acquire::http::No-Partial=true update
"""

? This will make not use the If-Range header. That one looked problematic in the logs.

Thanks,
 Michael

[Stuart Bishop (stub)]

Output of sudo apt-get -o Debug::Acquire::http=true -o Acquire::http::No-Partial=true update

[Michael Vogt (mvo)]

Thanks for the new log and sorry for my late reply (linuxtag kept me from working on this). It seems that No-Partial has good effects? Did you had any signature failures with since you installed it? Or did you just run it once/twice for testing?

Thanks,
 MIchael

[Stuart Bishop (stub)]

Michael Vogt wrote:
> Thanks for the new log and sorry for my late reply (linuxtag kept me from working on this). It seems that No-Partial has good effects? Did you had any signature failures with since you installed it? Or did you just run it once/twice for testing?

The latest version does not seem to have improved much as I still see BADSIG
errors. To reliably update, I need to run 'sudo apt-get update' twice.

--
Stuart Bishop <email address hidden> http://www.canonical.com/
Canonical Ltd. http://www.ubuntu.com/

[ginkhao (ginkhao)]

This might be old news, but I am in Thailand (transparent proxy) and have been having bad signature problems since I started using Ubuntu, but the "apt-get update -o Acquire::http::No-Cache=True" command fixed it for me.

[berg (berg-foss)]

The same thing sometimes happen with me. When happen , the solution give by "ginkhao said on 2006-11-28:" work to me too.

I think there is others bugs had same characteriscs like
https://bugs.launchpad.net/ubuntu/+source/update-manager/+bug/24061

Could implement a routine in apt-get to detect badsig errors and try automatically download again with "apt-get update -o Acquire::http::No-Cache=True" characteriscs in a "fail mode" for that mirrors with badsigs ?

[Stuart Bishop (stub)]

Updating status to unblock.

As per ginkhao's message this is still happening at least for a large proportion Thai users. Running "sudo apt-get update -o Acquire::http::No-Cache=True" from the command line seems to be the current work around to use.

[Stuart Bishop (stub)]

Unfortunately there doesn't appear to be anyway to use this work around with update-manager, causing feisty -> grumpy upgrades to fail.

If this failure was indeed caused by this bug, if there is some way of causing update-manager to use those apt flags it would provide a workaround.

[Jean-Baptiste Lallement (jibel)]

I'm closing the update-manager task because this is an issue with the configuration of apt not a bug in update-manager. You can set set these options in /etc/apt/apt.conf or /etc/apt/apt.conf/d/ in order to make it available for every tool using apt

Thanks for your time and don't hesitate to submit any new bug.

[mr_white (marco-nightlabs)]

I recommend that everyone experiencing this problem complains to his national parliament *and* the agency running the proxy. For Thailand that agency is CAT - http://www.cat.net.th - (e-mail <email address hidden>).

Since internet censorship is a growing world-wide problem, I recommend furthermore that you support the OpenNet Initiative - http://opennet.net - as well as local NGOs/parties like FACT - http://facthai.wordpress.com - in Thailand, the Pirate Party - http://en.wikipedia.org/wiki/Pirate_Party - in your country or whatever organisation fights for your rights.

[Wayne Scott (wsc9tt)]

Complaining to Thailand is all well and good, but you can get stuff like this just trying to use apt-cacher-ng or other local cacheing proxies. The problem isn't that the proxy is caching bad data. The problem is that stuff in /var/lib/apt/lists is
corrupted and even when the proxy is removed the bad data won't get detected or flushed.

I had to delete the contents of that directory to proceed.

[Matt C (mvc1095)]

Another use case: I use a DSL modem/router at home which intercepts HTTP requests and returns error pages informing me of the status of my internet connection when my DSL connection goes down. I personally think this is reasonable behaviour, but even if you disagree, the point is that this is a default configuration of many such devices and just telling people to reconfigure their routers is not a very user-friendly solution. Apt should be more robust. Instead, if cron-apt runs when my connection happens to be down, invalid data may get cached in /var/lib/apt/lists, meaning that when the connection is back up I get the BADSIG error when I next try to fetch updates. As Wayne Scott says above, bad data should be detected and flushed from the cache.

[Matt C (mvc1095)]

See also: https://bugs.launchpad.net/ubuntu/+source/update-manager/+bug/24061