| 2016-06-05 12:23:34 |
Rolf Leggewie |
bug |
|
|
added bug |
| 2016-06-05 12:24:58 |
Rolf Leggewie |
description |
Just found the following in the log:
Checking/Updating mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.gz... (12162KiB)
Checking/Updating mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.xz... (8756KiB)
I guess that once two versions of Packages{,.gz,.bz2,.xz} have made it to acng they will stay there basically forever and their information being downloaded several times. That's a waste of bandwidth.
I believe these days compressors rank from best to worst (purely in terms of compressed size) as xz < bz2 < gz < uncompressed. Furthermore, I assume that serving the uncompressed file to local clients as a fallback should always work.
Would it be possible that as a matter of policy acng tries to download the best compressed first and if that is 404 goes tries xz, bz2, gz and then uncompressed in that order and stores only that Packages file? If a client in the future requests Packages.bz2 and that is indeed available, then this is served, otherwise, fallback to Packages (which acng can get from the compressed file without any issues, recompressing with a different algorithm would possibly result in a different hashsum so I do not propose this). Any reason this would not work? |
Just found the following in the log:
Checking/Updating mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.gz... (12162KiB)
Checking/Updating mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.xz... (8756KiB)
I guess that once two versions of Packages{,.gz,.bz2,.xz} have made it to acng they will stay there basically forever and their information being downloaded several times. That's a waste of bandwidth.
I believe these days compressors rank from best to worst (purely in terms of compressed size) as xz < bz2 < gz < uncompressed. Furthermore, I assume that serving the uncompressed file to local clients as a fallback should always work.
Would it be possible that as a matter of policy acng tries to download the best compressed file first and if that is 404 goes through xz, bz2, gz and then uncompressed in that order and stores only that Packages file? If a client in the future requests Packages.bz2 and that is indeed available, then this is served, otherwise, fallback to Packages (which acng can get from the compressed file without any issues, recompressing with a different algorithm would possibly result in a different hashsum so I do not propose this). Any reason this would not work? |
|
| 2016-06-05 12:27:19 |
Rolf Leggewie |
description |
Just found the following in the log:
Checking/Updating mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.gz... (12162KiB)
Checking/Updating mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.xz... (8756KiB)
I guess that once two versions of Packages{,.gz,.bz2,.xz} have made it to acng they will stay there basically forever and their information being downloaded several times. That's a waste of bandwidth.
I believe these days compressors rank from best to worst (purely in terms of compressed size) as xz < bz2 < gz < uncompressed. Furthermore, I assume that serving the uncompressed file to local clients as a fallback should always work.
Would it be possible that as a matter of policy acng tries to download the best compressed file first and if that is 404 goes through xz, bz2, gz and then uncompressed in that order and stores only that Packages file? If a client in the future requests Packages.bz2 and that is indeed available, then this is served, otherwise, fallback to Packages (which acng can get from the compressed file without any issues, recompressing with a different algorithm would possibly result in a different hashsum so I do not propose this). Any reason this would not work? |
Just found the following in the log:
Checking/Updating mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.gz... (12162KiB)
Checking/Updating mirrordirector.raspbian.org/raspbian/dists/jessie/main/binary-armhf/Packages.xz... (8756KiB)
I guess that once two versions of Packages{,.gz,.bz2,.xz} have made it to acng they will stay there basically forever and their information being downloaded several times. That's a waste of bandwidth.
I believe these days compressors rank from best to worst (purely in terms of compressed size) as xz < bz2 < gz < uncompressed. Furthermore, I assume that serving the uncompressed file to local clients as a fallback should always work.
Would it be possible that as a matter of policy acng tries to download the best compressed file first and if that is 404 goes through xz, bz2, gz and then uncompressed in that order and stores only that Packages file? If a client in the future requests Packages.bz2 and that is indeed available, then this is served, otherwise, fallback to Packages (which acng can get from the compressed file without any issues, recompressing with a different algorithm would possibly result in a different hashsum so I do not propose this). Any reason this would not work?
PS: if this is workable, acng might want to cache for a few weeks the information that xz is not but bz2 is available for a certain Packages file. This is unlikely to change frequently and going straight to downloading bz2 next time around will lessen the number of connections to open. |
|
| 2016-06-06 12:58:58 |
Rolf Leggewie |
bug task added |
|
raspbian |
|
| 2022-01-27 11:26:57 |
Rolf Leggewie |
apt-cacher-ng (Ubuntu): status |
New |
Invalid |
|
