apt-transport-https reparses URLs, breaking CloudFront signing

[Danny Sauer]

CloudFront URL signing requires that the the whole URL matches the signature, as compared to S3 URLs which prune the querystring before validating the signature. The URL re-parsing which I'm pretty sure was introduced in resolving #1651923 decodes some HTML entities which were encoded before signing. That invalidates the signature and results in permission denied errors. In my case, the equal and semicolon chars from the return-content-disposition header are being decoded, but it applies to any entity which is "optionally" encoded.

Personally, I think that the URL returned in a Location: redirect header should be handled as-is, and if some piece of garbage http server is generating redirects which contain spaces or are otherwise invalid, the problem lies with the web server generating invalid redirects and not in apt failing to follow broken URLs. I say that knowing full well that the http spec says those things should be identical, so CloudFront also falls into the general "garbage" category by breaking the spec by requiring a specific format for identical characters.

Either way, things are what they are. Rather than unnecessarily fully reencoding the URL, I'd suggest that apt-transport-https (and presumably the http transport as well, but I don't know) should at most just replace the spaces with plusses or %20s to keep the original bug resolved without breaking other stuff. :)

PS: I tried to report this against the apt-transport-https package, but the bug tool says that doesn't exist in Ubuntu. That's weird, considering it's definitely a package (https://packages.ubuntu.com/groovy/apt-transport-https), so I assume the apt maintainer will know how to triage. Sorry.

[Julian Andres Klode (juliank)]

This has been fixed in 2.1.15 and newer versions, so starting with hirsute. As the change is too invasive, it has not been backported to older releases.

[Julian Andres Klode (juliank)]

WRT your other comment: apt-transport-https is a binary package, launchpad bugs are against source packages. Also apt-transport-https is an empty transitional package, there is no separate https method anymore.

[Danny Sauer (ubuntu-dannysauer)]

Great, thanks for sharing both answers. :)