BADSIG errors using transparent http proxies

Bug #33505 reported by Stuart Bishop on 2006-03-03
38
This bug affects 5 people
Affects Status Importance Assigned to Milestone
apt (Ubuntu)
High
Michael Vogt
update-manager (Ubuntu)
Undecided
Unassigned

Bug Description

Users forced to use substandard proxies, such as many users in Thailand, regularly get signature validation errors with apt-get update and all the tools that rely on apt such as update-manager and synaptic.

W: GPG error: http://security.ubuntu.com breezy-security Release: The following
signatures were invalid: BADSIG 40976EAF437D05B5 Ubuntu Archive Automatic Signing Key <email address hidden>

Stderr from running "sudo apt-get -o Acquire::BrokenProxy=true update -o Debug::Acquire::http=true 2> /tmp/apt-update.log"

Thanks for your bugreport.

One option you can turn on is

apt-get update -o Acquire::http::No-Cache=True

this will set the cache-control headers to turn caching off. Please let mek know if that helps.

Thanks,
 Michael

Michael Vogt wrote:

> One option you can turn on is
>
> apt-get update -o Acquire::http::No-Cache=True
>
> this will set the cache-control headers to turn caching off. Please let
> mek know if that helps.

I just tried a standard apt-get update, and it failed with the BADSIG
errors. I immediately then tried with the Acquire::http::No-Cache=True
option and it worked fine.

Let me know if you need me to run any experiments from my end - I suspect
this issue is fairly common and has the side effect of delaying or blocking
security updates, so it would be good if this situation could be detected
and the requests sent with the correct headers. Perhaps an optional file in
each archive containing an automatically updated timestamp? If the timestamp
is retrieved and it is too far out of date, we know we are behind a
particularly aggresive cache and we need to send the No-Cache header.

--
Stuart Bishop <email address hidden> http://www.canonical.com/
Canonical Ltd. http://www.ubuntu.com/

I have been getting these errors regularly as well. I will try the options mentioned but I would like to know if the OP can confirm that the NoCache option fixed the problem for him. My experience with the bug has been that it happens the first time I run apt-get update but never the second time. From the OP's most recent post it sounds like it may have only looked like the NoCache option fixed the problem.

Stuart Bishop (stub) wrote :

To do updates, I currently am running ' sudo apt-get -o Acquire::http::No-Cache=True -o Acquire::BrokenProxy=true update' twice. The first invariably fails. The second always works.

Proxies are mandatory here in Thailand due to Internet censorship laws, and I'm connected via the largest broadband provider, so this affects a few million potential Ubuntu users.

Michael Vogt (mvo) wrote :

Thanks for this additional feedback and sorry for my late reply.

I looked at the code and rfc2616 again and will attach a patch that may fix the problem. Testing is welcome, I'm happy to build a test-version of apt for you and put it on people.ubuntu.com.

It will always set Cache-Control: max-age=0 for indexfiles. rfc2616 is clear what this means:
"""
The request includes "max-age=0" cache-control directive, which forces each cache along the path to the origin server to revalidate its own entry, if any, with the next cache or server.
"""

I have to admit that I'm puzzled that Acquire::http::No-Cache=True dosn't work (on the first run). The rfc is pretty clear here:
"""
The request includes a "no-cache" cache-control directive or, for compatibility with HTTP/1.0 clients, "Pragma: no-cache". Field names MUST NOT be included with the no-cache directive in a request. The server MUST NOT use a cached copy when responding to such a request.
"""

Could you please make a dump (with -o Debug::Acquire::http=true) of the first and second run? And attach it here?

Thanks,
 Michael

This patch will always set cache control headers (for transparent proxies). It will set max-age=0 for all indexfiles too.

Changed in apt:
assignee: nobody → mvo
status: Unconfirmed → Needs Info

This is the output from the first run of:

sudo apt-get -o Acquire::http::No-Cache=True -o Debug::Acquire::http=true update 2>&1 | tee firstrun.txt

This is the output from the second run of:

sudo apt-get -o Acquire::http::No-Cache=True -o Debug::Acquire::http=true update 2>&1 | tee secondrun.txt

Thanks for the new logs. Could you please give the follow package a try?

Please add this to your sources.list:
"""
deb http://people.ubuntu.com/~mvo/apt/transparent-proxy/ /
"""

and update, upgrade. It should install a new apt (version 0.6.43.3ubuntu2.1)

Please let me know if that package works better. Please try first without the -o Acquire::http::No-Cache=true option and then with it.

Thanks,
 Michael

Run of the April 25th test apt from mvo - output of:

    sudo apt-get -o Debug::Acquire::http=true update 2>&1

Output from the April 25th apt package by mvo of:

sudo apt-get -o Debug::Acquire::http=true update -o Acquire::http::No-Cache=true 2>&1

I updated to version apt 0.6.43.3ubuntu2.2.

Can you please upgrade to this version and run it with:

"""
sudo apt-get -o Debug::Acquire::http=true -o Acquire::http::No-Partial=true update
"""

? This will make not use the If-Range header. That one looked problematic in the logs.

Thanks,
 Michael

Output of sudo apt-get -o Debug::Acquire::http=true -o Acquire::http::No-Partial=true update

Thanks for the new log and sorry for my late reply (linuxtag kept me from working on this). It seems that No-Partial has good effects? Did you had any signature failures with since you installed it? Or did you just run it once/twice for testing?

Thanks,
 MIchael

Michael Vogt wrote:
> Thanks for the new log and sorry for my late reply (linuxtag kept me from working on this). It seems that No-Partial has good effects? Did you had any signature failures with since you installed it? Or did you just run it once/twice for testing?

The latest version does not seem to have improved much as I still see BADSIG
errors. To reliably update, I need to run 'sudo apt-get update' twice.

--
Stuart Bishop <email address hidden> http://www.canonical.com/
Canonical Ltd. http://www.ubuntu.com/

This might be old news, but I am in Thailand (transparent proxy) and have been having bad signature problems since I started using Ubuntu, but the "apt-get update -o Acquire::http::No-Cache=True" command fixed it for me.

berg (berg-foss) wrote :

The same thing sometimes happen with me. When happen , the solution give by "ginkhao said on 2006-11-28:" work to me too.

I think there is others bugs had same characteriscs like
https://bugs.launchpad.net/ubuntu/+source/update-manager/+bug/24061

Could implement a routine in apt-get to detect badsig errors and try automatically download again with "apt-get update -o Acquire::http::No-Cache=True" characteriscs in a "fail mode" for that mirrors with badsigs ?

Stuart Bishop (stub) wrote :

Updating status to unblock.

As per ginkhao's message this is still happening at least for a large proportion Thai users. Running "sudo apt-get update -o Acquire::http::No-Cache=True" from the command line seems to be the current work around to use.

Changed in apt:
status: Incomplete → Confirmed
Stuart Bishop (stub) wrote :

Unfortunately there doesn't appear to be anyway to use this work around with update-manager, causing feisty -> grumpy upgrades to fail.

If this failure was indeed caused by this bug, if there is some way of causing update-manager to use those apt flags it would provide a workaround.

description: updated
Changed in apt:
status: Confirmed → Fix Released

I'm closing the update-manager task because this is an issue with the configuration of apt not a bug in update-manager. You can set set these options in /etc/apt/apt.conf or /etc/apt/apt.conf/d/ in order to make it available for every tool using apt

Thanks for your time and don't hesitate to submit any new bug.

Changed in update-manager:
status: New → Won't Fix
mr_white (marco-nightlabs) wrote :

I recommend that everyone experiencing this problem complains to his national parliament *and* the agency running the proxy. For Thailand that agency is CAT - http://www.cat.net.th - (e-mail <email address hidden>).

Since internet censorship is a growing world-wide problem, I recommend furthermore that you support the OpenNet Initiative - http://opennet.net - as well as local NGOs/parties like FACT - http://facthai.wordpress.com - in Thailand, the Pirate Party - http://en.wikipedia.org/wiki/Pirate_Party - in your country or whatever organisation fights for your rights.

Wayne Scott (wsc9tt) wrote :

Complaining to Thailand is all well and good, but you can get stuff like this just trying to use apt-cacher-ng or other local cacheing proxies. The problem isn't that the proxy is caching bad data. The problem is that stuff in /var/lib/apt/lists is
corrupted and even when the proxy is removed the bad data won't get detected or flushed.

I had to delete the contents of that directory to proceed.

Matt C (mvc1095) wrote :

Another use case: I use a DSL modem/router at home which intercepts HTTP requests and returns error pages informing me of the status of my internet connection when my DSL connection goes down. I personally think this is reasonable behaviour, but even if you disagree, the point is that this is a default configuration of many such devices and just telling people to reconfigure their routers is not a very user-friendly solution. Apt should be more robust. Instead, if cron-apt runs when my connection happens to be down, invalid data may get cached in /var/lib/apt/lists, meaning that when the connection is back up I get the BADSIG error when I next try to fetch updates. As Wayne Scott says above, bad data should be detected and flushed from the cache.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Duplicates of this bug

Other bug subscribers