ubuntu distro hashes insecure against MITM attacks

[Thomas Mayer]

ubuntu offers an overview to get hashes of download files at https://help.ubuntu.com/community/UbuntuHashes (via HTTPS).

It then takes me to http://releases.ubuntu.com (via HTTP). Which then takes me to e.g. http://releases.ubuntu.com/15.10/ (via HTTP). Which then takes me to e.g. http://releases.ubuntu.com/15.10/SHA256SUMS (via HTTP) which finally contains the hashes.

This HTTP resource is not protected against MITM attackers. Basically that means that when MITM is able to compromize the download, MITM should also be able to compromize the hashes I want to test against. There is also no HTTPS-secured representation of these hashes available (e.g. http://releases.ubuntu.com/15.10/SHA256SUMS).

But there's a http://releases.ubuntu.com/15.10/SHA256SUMS.gpg signature while it is not mentioned in the web pages above, how this can be used to mitigate MITM attacks or who is finally authorized to sign something in the name of ubuntu and which is his or her pgp public key.

I remember that there was once a HTTPS-secured web page containing all the hashes for at least the ubuntu 14.04.x downloads. I think this was really straight forward: Get the hash from a trusted source (via https) and compare with the hash of the download. In that sense, I consider the current state as a regression.

[Thomas Mayer (thomas303)]

There's some documentation about how to check the hash with gpg and which key is authorized at https://help.ubuntu.com/community/VerifyIsoHowto . This page is linked in https://help.ubuntu.com/community/UbuntuHashes . So finally, there is a statement available which key should be valid. Plus, this page is available via HTTPS so it cannot be altered via MITM.

However, this documentation might not be suitable for everybody (e.g. windows users?) and in any case it is ways too complicated and potentially defective for average users. I still think the hashes should again be made available via https, at least for practical reasons.

related: #1460242 #1359836 #1186793

which means the attack vector is relatively high compared how small it could be.

[Seth Arnold (seth-arnold)]

Since we publish a few thousand images it doesn't make sense to put the hashes themselves in the wikipages. What we need is the various GPG keys that we use published somewhere. I've asked the website team to make this list available but it's obviously a very low priority for the team.

In the meantime I've added a new FAQ section to the Ubuntu security team wiki:
https://wiki.ubuntu.com/SecurityTeam/FAQ#GPG_Keys_used_by_Ubuntu
This isn't ideal since it's available for all to change but at least a handful of people receive email when it is changed, and it's otherwise completely impossible to discover this information.

One positive of putting it in the wiki is that others _can_ add to it -- I know these handful of keys need to be published but there are probably more keys that deserve to be publicly published and when people discover them, they can be added here too.

Thanks.

[Peter Mahnke (peterm-ubuntu)]

Seth, how often do these keys ever change?

[Seth Arnold (seth-arnold)]

Peter, these keys rarely change, though we do add to them periodically. (I suspect we should also list keys used for e.g. phone images, snappy images, etc.)

Every few months feels like a realistic worst case.

Thanks

[Iain Lane (laney)]

One part of the fix is to make {releases,cdimage}.ubuntu.com accessible over https. This is independent of publishing the GPG key information - can that be done please?

[Chad Miller (cmiller)]

Even the GPG-related instructions linked from the download pages on ubuntu.com are served in an unvalidated way. An attacker dumber than I am could replace images, links, and instructions on what to verify against. This is a huge deal.

Flip all of releases.ubuntu.com and www.ubuntu.com to be TLS only and redirect plain http to https, to fix everything.

[Thomas Mayer (thomas303)]

Not to forget adding a HSTS header for a https-only site. Maybe combined with HSTS preloading and DNSSEC/DANE in the long run.

[Thomas Mayer (thomas303)]

This issue should see some escalation now. Since I wrote this issue, another distro was tricked into distributing compromised ISOs containing a back door. This was not done via MITM, as the attacker was lucky enough to compromise the distro's servers directly. However, it shows that there is a use case and vulnerabilities will be used whenever available.

http://www.pcworld.com/article/3035682/security/hackers-planted-a-backdoor-inside-a-compromised-version-of-linux-mint.html

Three-letter-agencies, providers, governments and smaller players like internet cafés can easily compromise DNS or use transparent http proxies (among other ways) and downloaded ISO's integrity is gone.

[Chad Miller (cmiller)]

The current scheme of having a GPG sig served over un-validated transport was obviously made by someone who thought it was natural to have met and validated the signing key of someone in the Debian GPG web-of-trust. TLS CAs are the only scheme that are as widely adopted as we want Ubuntu to be.

All future trust of a machine is only as good as the trustability of the downloaded image. It's scary to me to think that almost no one's Ubuntu install is properly validated.

[Mark Shuttleworth (sabdfl)]

Knowing a thing or two about TLS CA's, I beg to differ ;)

[Thomas Mayer (thomas303)]

I fully agree that there's been issues with CA's.

One can still pin a specific X.509 public key with a DNSSEC-secured TLSA RR. Browsers don't support it yet natively, but users could still verify the certificate shown in the browser by hand. Then, there would be no dependency to a CA in terms of trust: DNSSEC's root serves as a trust anchor for DNSSEC's chain of trust. Even the domain owner ist validated (by the TLD's specific registrar this time, not by one of the CA's).

There's also some rumors about DNSSEC/TLSA RRs for a trusted distribution of S/MIME and PGP fingerprints via DNS.

And there's HTTP Public Key Pinning (HPKP) which has seen some browser support at least.

All of that is optional. And it goes well together with another trust anchor (e.g. documentation and somehow trusted download of the signatures or another trust anchor).

Even if the TLS certificate is not pinned and all the CA's serve as a trust anchor: It's still ways safer than plain text http, when a MITM comes into play.

[Thomas Mayer (thomas303)]

@sabdfl

I think PGP's network of trust has some flaws in terms of key management, too. Plus, is usability an issue here (How would you verify an ubuntu ISO with a trustworthy fresh install of windows in an internet café?).

If you want to go for PGP without relying on any sort of CA, there's a new way to publish PGP keys in a safe manner via DNS using DNSSEC and DANE framework using the new resource record OPENPGPKEY

Draft:
https://tools.ietf.org/html/draft-ietf-dane-openpgpkey-12

Of course, this would require the use of DNSSEC in general and DNSSEC-capable resolving on client side. Plus, until widespread usage is visible, should it be documented how retrieve and verify a PGP key against an ISO binary.

I think it is ways to early to go for DNSSEC/OPENPGPKEY as the only way but I'd really like to see this approach happen at some time in the future. But maybe it's an option in addition to the network of trust.

In the meantime, we stuck with the CA system, maybe combined with CA-pinning via DNSSEC/DANE or HPKP. On the other side, we stuck with the network of trust for PGP.

DANE is available for CA/Certificate-pinning (accidently) and will hopefully become available for PGP, too. So there is a way to tighten the screws in the future (or now) without a breaking change.

Why not offer both (X.509 AND PGP), and let the user decide which trust anchor her or she wants to trust? In the end, trust must come from the user. As of now, I can't find a trustworthy way to verify integrity of ubuntu's ISOs (documentation included).

[Thomas Mayer (thomas303)]

Another case of a compromised downloading platform: http://news.softpedia.com/news/hacker-compromises-fosshub-to-distribute-mbr-hijacking-malware-506932.shtml

TLS and checksums would presumably not have prevented a takeover of the whole platform (as it seems), but it illustrates that there is need for authenticated checkout of sources, a trustworthy build platform and verified downloads.

In the end it does not matter which part of the chain is compromised (this issue is about verified downloads only, which is the easiest part).

[Thomas Mayer (thomas303)]

More than a year later, this is still not fixed. Unsandboxed downloads still expose ubuntu users against MITM right from the start. That's as bad as if apt installed unsigned packages from untrusted sources. Even worse.

Another project that moved from a https-secured publication of hash values to an insecure version is the distribution SysRescueCd http://www.system-rescue-cd.org/Download/

That single page demonstrates pretty well that the download is not protected at all against MITM: Put a transparent proxy somewhere in between and redirect download and hash to a manipulated version somewhere else. That's far too easy for attackers.

The sad thing is that both, ubuntu and SysRescueCd once had https-secured hashes in place.

In 2017, we're talking about reproducable builds, but can't protect a download. Sad.

[Mark Shuttleworth (sabdfl)]

Is the fix for this as simple as putting releases.ubuntu.com behind https?

Mark

[Thomas Mayer (thomas303)]

In principle, yes.

[Ben (benjaminrichardwheatley)]

I believe putting releases.ubuntu.com behind https would be sufficient, yes.

[Thomas Mayer (thomas303)]

Putting releases.ubuntu.com behind https or even https-only even secures the download itself against a CA. That's even more than just securing the hashes. Would still be great as this automatically secures downloads even for users who don't verify checksums (real world scenario).

The advantage of an https-only strategy would be that canonical can be "sure" that any future download from canonical's https servers once was trustworthy. Given that "sure" is relative, that could help against future PR disasters at least.

Additionally, should the chain of http-links from the documentation to the hash be https-protected. But that seems more like a nice-to-have compared to the main issue.

[Thomas Mayer (thomas303)]

I have to add that my previous comment only covers canonical's servers to have one path for verification at least. Given that hashes would be secured, that allows users to also verify downloads from mirrors.

Mirrors should not necessarily be considered trustworthy, even when offering checksums and/or downloads via https. Canonical, as the releasing party, is the only party which can say which checksum is right for a release. Everything behind breaks that chain of trust.

That said, it does it not make sense to me to verify against a checksum which is hosted on mirrors, unless I trust the mirror and thereby trust that the mirror verifies downloads. Which I don't (no matter if the checksum and/or download is https-secured on mirrors).

[Wise Melon (wise-melon-deactivatedaccount)]

http://cdimage.ubuntu.com/ where the hashes and ISOs are hosted would also need to become HTTPS-only.

[Eleanor Ellis (eleanor-ac-ellis)]

I have just had a similar discussion on #ubuntu-devel (see https://irclogs.ubuntu.com/2017/07/16/%23ubuntu-devel.html) and I have also posted this concern to #ubuntu-hardened

I agree that cdimage.ubuntu.com and releases.ubuntu.com should be https only.
Hopefully someone will do something about this.

[Eleanor Ellis (eleanor-ac-ellis)]

I also think all the mirrors should be https only. If that is too difficult then perhaps Ubuntu should cease the use of mirrors and promote bittorrent more aggressively or use dns to create mirrors invisible to the users (all the mirrors at the same address).

[Thomas Mayer (thomas303)]

@sabdfl Still not convinced?
https://www.welivesecurity.com/2017/09/21/new-finfisher-surveillance-campaigns/

[Thomas Mayer (thomas303)]

@sabdfl In 2018, I would not download ubuntu in Turkey and Egypt.

https://www.bleepingcomputer.com/news/security/turkish-isp-swapped-downloads-of-popular-software-with-spyware-infected-apps/