Package lists are fully readed if there are no changes

[Removed by request]

I'm using Ubuntu 12.04 dev with apt 0.8.16~exp12ubuntu3. Since apt was updated from 0.8.16~exp5 to 0.8.16~exp12 the package lists are fully readed ("Reading package lists... Done") on every "apt-get update". This is no problem if there are new package lists available. But apt is doing now such a full reading even if the lists have not changed.

[Torsten Spindler (tspindler)]

I cannot reproduce this problem on 12.04 against archive.ubuntu.com:

...
Hit http://archive.ubuntu.com precise-proposed/restricted Translation-en
Hit http://archive.ubuntu.com precise-proposed/universe Translation-en
Fetched 12.4 MB in 12s (1,022 kB/s)
Reading package lists... Done
...
Hit http://archive.ubuntu.com precise-proposed/restricted Translation-en
Hit http://archive.ubuntu.com precise-proposed/universe Translation-en
Reading package lists... Done
spindler@spitfire:~/Work/apt/debian$ sudo apt-get update
Ign http://security.ubuntu.com precise-security InRelease
Ign http://archive.canonical.com precise InRelease

Full log at http://pastebin.ubuntu.com/860528/

[Removed by request (removed3425744)]

The problem is the line "Reading package lists... Done". If new package lists are downloaded "Reading package lists... Done" can need ~15 seconds +/- a few seconds. If there are no new package lists are available "Reading package lists... Done" should only need ~1 second. The problem is that it still needs a lot of time. Here is an example output which measures the time (you can try this too):

sworddragon@ubuntu:~$ START=$(date +%s) && sudo apt-get update && END=$(date +%s) && echo $(($END-$START))
Ign http://extras.ubuntu.com precise InRelease
Ign http://extras.ubuntu.com oneiric InRelease
Ign http://security.ubuntu.com precise-security InRelease
Ign http://security.ubuntu.com oneiric-security InRelease
Ign http://archive.canonical.com precise InRelease
Ign http://archive.canonical.com oneiric InRelease
Ign http://archive.ubuntu.com precise InRelease
Ign http://archive.ubuntu.com precise-updates InRelease
Ign http://archive.ubuntu.com precise-backports InRelease
Get:1 http://extras.ubuntu.com precise Release.gpg [72 B]
Hit http://security.ubuntu.com precise-security Release.gpg
Hit http://archive.canonical.com precise Release.gpg
Ign http://archive.ubuntu.com oneiric InRelease
Ign http://archive.ubuntu.com oneiric-updates InRelease
Ign http://archive.ubuntu.com oneiric-backports InRelease
Get:2 http://archive.ubuntu.com precise Release.gpg [198 B]
Hit http://archive.ubuntu.com precise-updates Release.gpg
Get:3 http://extras.ubuntu.com oneiric Release.gpg [72 B]
Hit http://security.ubuntu.com oneiric-security Release.gpg
Hit http://archive.canonical.com oneiric Release.gpg
Hit http://archive.ubuntu.com precise-backports Release.gpg
Hit http://archive.ubuntu.com oneiric Release.gpg
Hit http://archive.ubuntu.com oneiric-updates Release.gpg
Hit http://extras.ubuntu.com precise Release
Hit http://security.ubuntu.com precise-security Release
Hit http://archive.canonical.com precise Release
Hit http://archive.ubuntu.com oneiric-backports Release.gpg
Get:4 http://archive.ubuntu.com precise Release [49.6 kB]
Hit http://extras.ubuntu.com oneiric Release
Hit http://security.ubuntu.com oneiric-security Release
Hit http://archive.canonical.com oneiric Release
Hit http://extras.ubuntu.com precise/main Sources
Hit http://extras.ubuntu.com precise/main amd64 Packages ...

[William Lee (wlee753159)]

It still does that in 14.04 LTS. I updated my package lists then did that again, but it still reads the package lists for 5-10 seconds

[Julian Andres Klode (juliank)]

Yes that's true IIRC. And update explicitly rebuilds the cache. That's not a huge issue, unless you are sitting around running apt-get update in loops.

It is not possible to rebuild only changed parts, except for /var/lib/dpkg/status changes. For making that work, a second cache is generated; we cannot do the same for every source list.

[Removed by request (removed3425744)]

> And update explicitly rebuilds the cache.

It would be really nice if we could get back the old behavior here - the cache gets only rebuilded if there are only real changes. But I have found an easier testcase to reproduce this issue:

root@ubuntu:~# time apt-get clean

real 0m0.025s
user 0m0.004s
sys 0m0.021s
root@ubuntu:~# time apt-get check firefox
Reading package lists... Done
Building dependency tree
Reading state information... Done

real 0m6.724s
user 0m3.423s
sys 0m0.148s
root@ubuntu:~# time apt-get check firefox
Reading package lists... Done
Building dependency tree
Reading state information... Done

real 0m0.447s
user 0m0.430s
sys 0m0.017s
root@ubuntu:~# time apt-get clean

real 0m0.018s
user 0m0.004s
sys 0m0.013s
root@ubuntu:~# time apt-get check firefox
Reading package lists... Done
Building dependency tree
Reading state information... Done

real 0m6.516s
user 0m3.371s
sys 0m0.161s

As this example shows "apt-get clean" causes the next command that reads from the package cache to rebuld it while there are even no explicit changes.