Language-specific descriptions not imported

[Michael Nelson]

Currently we import all of the package data from the package cache for each series, using python apt (which rocks!).

It seems that some packages in Oneiric now have language-specific fields such as Description-en (and don't have a Description field) [1]

For these packages we currently end up with a blank description [2]. This seems to be because our server runs on lucid, and the lucid version of the python apt doesn't handle the language-specific fields whereas the oneiric python apt does [3].

I'm not sure if there is a way with the lucid version of python apt that we can work around this, or whether we need to backport/install a custom version on the server. I assume there will be a way to work around it (and mvo will know it :) ).

[1] http://paste.ubuntu.com/714889/
[2] https://apps.ubuntu.com/cat/applications/oneiric/arduino/
[3] http://paste.ubuntu.com/714895/

[Michael Nelson (michael.nelson)]

Just some initial investigation: python-apt on lucid is actually pulling different translated_description files for precice than python-apt on precise.

Python-apt on precise has package.candidate._cand.translated_description.file_list returning a list with one file, like:

archive.ubuntu.com_ubuntu_dists_precise_main_i18n_Translation-en

but python-apt on lucid, has the same api available (seems it supports translations?) package.candidate._cand.translated_description.file_list, but it's just the normal binary packages:

archive.ubuntu.com_ubuntu_dists_precise_main_binary-amd64_Packages

So the next question is can we get the lucid version of python-apt.cache to pull the i18n file when it's available (ie. when it's checking > oneiric)? It doesn't seem like something worth modifying in python-apt for lucid (as normally lucid won't need to deal with precise packages etc.), but is perhaps something we can work around.

[Michael Vogt (mvo)]

Just for english descriptions in oneiric/precise you should be able to get them with:
 apt.apt_pkg.config.set("APT::Acquire::Translation", "en")
before cache.update()

This will force the english translations download even if the (old) apt assumes it does
not need it.

[Michael Vogt (mvo)]

This code works for me in a lucid chroot with a oneiric target in sources.list:

import apt
cache=apt.Cache(rootdir="./aptroot")
apt.apt_pkg.config.set("APT::Acquire::Translation", "en")
cache.update()
cache.open()
print cache["apt"].summary
print cache["apt"].description

[Michael Nelson (michael.nelson)]

Thank-you thank-you thank-you mvo :)

[Michael Vogt (mvo)]

If you want to get multiple languages its going to be a bit slow to import them, but its possible using:

cache=apt.Cache(rootdir="./aptroot")
for lang in ["en", "de"]:
    apt.apt_pkg.config.set("APT::Acquire::Translation", lang)
    cache.update()
    cache.open()
    print cache["apt"].summary
    print cache["apt"].description

As you can see it will run "update" multiple times. It will not re-download the entire Packages files all
over again (as its using if-modified-since) but it will rebuild the cache a lot which takes some seconds.

The langlist needs to be longer of course :) And some will need the long form (e.g. pt_BR, pt_PT).

[Michael Vogt (mvo)]

You should probably set:
  apt.apt_pkg.config.set("APT::List-Cleanup", "0")
too, otherwise libapt will cleanup the lists/ directory with translations it does not appear to need. With that flag
it will leave them there and you can benefit from the if-modified-since goodness of libapt.