wget crash when printing download rate

[halfgaar]

All supported versions of Ubuntu suffer from crashes in wget in printing of the download speed. I've been getting this on various servers. It's been fixed upstream and should probably be included in 'updates' of all supported Ubuntu versions.

https://git.savannah.gnu.org/git/wget.git
Commit 04ab35666997fbb3cd5d72497415fb3dfd62dcc5

https://lists.gnu.org/archive/html/bug-wget/2023-08/msg00001.html

Patch attached.

[Ubuntu Foundations Team Bug Bot (crichton)]

The attachment "0001-Fix-crash-when-printing-download-rate.patch" seems to be a patch. If it isn't, please remove the "patch" flag from the attachment, remove the "patch" tag, and if you are a member of the ~ubuntu-reviewers, unsubscribe the team.

[This is an automated message performed by a Launchpad user owned by ~brian-murray, for any issues please contact him.]

[Marius Gedminas (mgedmin)]

AFAICT from the patch, this only affects users who experience download speeds measured in terabytes per second.

[halfgaar (wiebe-halfgaar)]

The code also says:

if (secs == 0)
  /* If elapsed time is exactly zero, it means we're under the
     resolution of the timer. This can easily happen on systems
     that use time() for the timer. Since the interval lies between
     0 and the timer's resolution, assume half the resolution. */
  secs = ptimer_resolution () / 2.0;

secs is a double. If because of timer issues it's tiny, you can get very large numbers. In me trying to reproduce it, I saw GB/s speeds all the time that were theoretically impossible.

I started getting crashes on various Ubuntu 22.04 servers. There may have been a change somewhere that exposes it, but the problem was always there.

[Mark Esler (eslerm)]

Upstream will not be assigning this issue a CVE [0] and the Ubuntu Security Team does not consider this bug security relevant.

This bug is caused when calculating the download speed, but hitting an out-of-bounds on the table that contains the printable strings. It is hitting GB/s because it is, presumably, not handling the system timer resolution correctly. This is no more security relevant than any other bug that crashes wget.

@wiebe-halfgaar, thank you for raising awareness about this issue and getting it fixed upstream.

[0] https://lists.gnu.org/archive/html/bug-wget/2023-08/msg00008.html

[halfgaar (wiebe-halfgaar)]

Is there any progress in getting it fixed in 'updates'?

[halfgaar (wiebe-halfgaar)]

I'm still getting a bunch of wget crashes from cron jobs I have. Is there any progress on this?

[Mark Esler (eslerm)]

@halfgaar, I've requested that the Foundation's team review the priority of this bug

[Andreas Hasenack (ahasenack)]

I prepared a ppa with the fix here:

https://launchpad.net/~ahasenack/+archive/ubuntu/wget-2029930

Packages are still building as I write this, and should be published in an hour or so.

@halfgaar, would you be able to test it? I can easily upload to noble, but for an SRU, we would require a bit more detail on how you are so easily able to reproduce this crash, and how we could create an environment where the fix could be tested for all affected ubuntu releases.

[halfgaar (wiebe-halfgaar)]

I have just been able to reproduce it by running this on the same server the website example.org is hosted:

cd /tmp
while true; do rm -rf www.example.org/ && wget --mirror --page-requisites https://www.example.org 2>&1 | grep -F GB/s; done

The 'grep GB/s' makes a point of showing the improbable speeds. It's not unusual to see several dozens or hundreds of GB/s there, like this:

2023-11-19 19:33:57 (116 GB/s) - ‘www.example.org/wp-json/oembed/1.0/embed?url=https:%2F%2Fwww.example.org%2F2023%2F06%2F05%2Fredacted’ saved [2362/2362]

That's 2 kB, downloaded at 116 GB/s. Obviously a timer issue. This illustrates it's not hard to occasionally hit some super high TB/s value, which would trigger the crash.

With the PPA, running this grepping on TB/s, it took about 30 minutes, but eventually:

2023-11-19 19:34:16 (4.00 TB/s) - ‘www.example.org/wp-json/oembed/1.0/embed?url=https:%2F%2Fwww.example.org%2F2023%2F08%2F19%2Fredacted%2F’ saved [2201/2201]

This is all crawling a Wordpress site. There seems to be a higher chance of crawling a dynamic site vs static files, possibly because it doesn't start counting bytes until it has seen the first, and dynamic sites tend to only operate from memory at that point. That's probably also the reason it tends to happen on 2 kB-ish files, because it fits in one or two TCP packets.

Probably a simple PHP scripts that prints a few kBytes is enough.