apt-cacher-ng can not deliver package, caused by (maybe) special characters in filename

[S. W.]

Hi,

I have the following entry in my sources.list:
#
deb [arch=amd64] http://dl.bintray.com/dreamcat4/ubuntu master main
#

When I try to install a package, hosted by this repo, apt-cacher is not able to deliver the file to apt.

#
apt-get dist-upgrade
[...]Die folgenden Pakete werden aktualisiert (Upgrade):
  tvheadend
1 aktualisiert, 0 neu installiert, 0 zu entfernen und 0 nicht aktualisiert.
Es müssen 19,5 MB an Archiven heruntergeladen werden.
Nach dieser Operation werden 0 B Plattenplatz zusätzlich benutzt.
Möchten Sie fortfahren? [J/n] J
Fehl:1 http://dl.bintray.com/dreamcat4/ubuntu master/main amd64 tvheadend amd64 4.1-2239~gcf4880d~trusty
  403 Forbidden
E: Fehlschlag beim Holen von http://dl.bintray.com/dreamcat4/ubuntu/4.1-2239~gcf4880d/pool/main/t/tvheadend/tvheadend_4.1-2239~gcf4880d~trusty_amd64.deb 403 Forbidden
#

My apt conf:
#
cat /etc/apt/apt.conf.d/01proxy
// This file is managed by Puppet. DO NOT EDIT.
Acquire::http::proxy "http://apt-cacher.intern.de:3142/";
#

I increased the debug level to 6 and made a tail on /var/log/apt-cacher-ng.* and startet apt-get.
#
==> /var/log/apt-cacher-ng/apt-cacher.err <==
Wed Sep 21 07:43:14 2016|Detected incoming connection from the TCP socket
Wed Sep 21 07:43:14 2016|Client name: 127.0.0.1
Wed Sep 21 07:43:14 2016|Decoded request URI: http://dl.bintray.com/dreamcat4/ubuntu/4.1-2239~gcf4880d/pool/main/t/tvheadend/tvheadend_4.1-2239~gcf4880d~trusty_amd64.deb
Wed Sep 21 07:43:14 2016|Processing new job, GET http://dl.bintray.com/dreamcat4/ubuntu/4.1-2239%7egcf4880d/pool/main/t/tvheadend/tvheadend_4.1-2239%7egcf4880d%7etrusty_amd64.deb HTTP/1.1
Wed Sep 21 07:43:14 2016|Download started, storeHeader for dl.bintray.com/dreamcat4/ubuntu/4.1-2239~gcf4880d/pool/main/t/tvheadend/tvheadend_4.1-2239~gcf4880d~trusty_amd64.deb, current status: 1
Wed Sep 21 07:43:14 2016|Response header to be sent in the next cycle:
HTTP/1.1 403 Forbidden
Content-Length: 0
Date: Wed Sep 21 05:43:14 2016
Server: Debian Apt-Cacher NG/0.9.1
X-Original-Source: http://akamai.bintray.com/30/30724d38fd6e30b66bef6d7ac2c2e28d4fcba7226ba196ed8232331a8e5facf2?__gda__=exp=1474437314~hmac=519bcb47b3b5983f7311cf70755a427013391b1ba89cb5f3b6ba9a36c4111b23&response-content-disposition=attachment;filename="tvheadend_4.1-2239~gcf4880d~trusty_amd64.deb"&response-content-type=application/x-debian-package&requestInfo=U2FsdGVkX18MQiK8d2pJifrulrSNz0ii1PgbWYYysSbRZInFBAgIN8PBBNO6KZPdkRNId3dUIhDGLB_Q4vm5D8GNFZ2DZvfMKxlcxXidm6uyq3lGKGAiveWaqKllveE496mb4dXY0hGFtG0_mLzKCssOEr0M4hTk33CSGrhzLRscuWp4d51bYH9u93gxUDsH
Connection: Keep-Alive

Wed Sep 21 07:43:14 2016|Returning to last state, 6

==> /var/log/apt-cacher-ng/apt-cacher.log <==
1474436594|M|Download of dl.bintray.com/dreamcat4/ubuntu/4.1-2239~gcf4880d/pool/main/t/tvheadend/tvheadend_4.1-2239~gcf4880d~trusty_amd64.deb started
1474436594|M|Download of dl.bintray.com/dreamcat4/ubuntu/4.1-2239~gcf4880d/pool/main/t/tvheadend/tvheadend_4.1-2239~gcf4880d~trusty_amd64.deb finished
1474436595|I|390
1474436595|O|696
#

My version:
#
 dpkg -l | grep apt-cacher-ng
ii apt-cacher-ng 0.9.1-1ubuntu1
#

And the other standard stuff:
#
 lsb_release -rd
Description: Ubuntu 16.04.1 LTS
Release: 16.04
#

Thanks for helping!

Best regards.

[Eduard Bloch (edi-gmx)]

I think it's a bug of Akamai. ACNG does reeencode the URL which was sent back in the intermediate redirect response - however, this is basically legal URL encoding, no magic done there. Simple HTTP clients would use the URL from redirect response as-is and therefore never trigger that condition.

Maybe they do some (broken?) verification against the uber-clever pseudo signature that they append there, maybe without considering URL encoding, I am not sure. Needs closer investigation.

Or I could a special mode (to be enabled with a switch) which would disable any sanity checks on that URLs.

[Eduard Bloch (edi-gmx)]

Well, ok, I checked the code. It's a bit too messy at the moment WRT encoded and unencoded URLs. It should be refactored and for cases where no re-encoding is needed, it could pass-through the original URL no matter which crap was exactly hidden there.

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[VSHN (vshn)]

Any updates on this issue?
We have the same problem with repositories hosted on bintray

[Maksim Malchuk (mmalchuk)]

dl.bintray.com prohibit the download using the some proxy.
configure the UserAgent in /etc/apt-cacher-ng/acng.conf:

UserAgent: Debian APT-HTTP/1.3 (1.6.10)