No API to manipulate PPAs and buildds

[Cody A.W. Somerville]

There is no API to manipulate PPAs and buildds (ex. copy packages, delete a package, rebuild a package, gather information about packages in the PPA, etc.)

[Celso Providelo (cprov)]

We have to start exposing PPAs via Person.

Then we should probably expose `ArchiveSourcePublication` objects, containing:
 * source dsc fields;
 * builds;
 * source files;
 * dsc_signing_key;
 * upload_record;
 * publishing details;
 * binary publications:
   * control fields;
   * publishing details.

[Julian Edwards (julian-edwards)]

On Thursday 13 November 2008 03:16:46 Celso Providelo wrote:
> We have to start exposing PPAs via Person.

Right, this is trivial.

> Then we should probably expose `ArchiveSourcePublication` objects,
> containing: * source dsc fields;
> * builds;
> * source files;
> * dsc_signing_key;
> * upload_record;
> * publishing details;
> * binary publications:
> * control fields;
> * publishing details.

This needs a design discussion before going any further. Let's discuss when
you're back from Lexington.

[Jamie Strandboge (jdstrand)]

For USN publication of security updates, the security team screen scrapes the following information from various locations:

name of source
version of source
series of source
source changes
binaries' changes

and the following sections from +archive:
'Builds' section
'Available diffs' section (optional, but desired)
'Download files from Librarian' section

[Kees Cook (kees)]

To just be clear, for USN publication we need:
 - all .changes files (including source)
 - full source links (dsc, orig, diff)
 - status of each arch's build
 - all binary files generated from the build

[Julian Edwards (julian-edwards)]

RF 7420 exports source publications on the API. More to come soon, including Kees' requirements.

[Julian Edwards (julian-edwards)]

Extra comments from Kees' duplicate bug 307548:
{{{
I'd like to have access to details in the PPA's list of builds, along with full file lists, build status, build logs, changes, etc. (We'd like to replace our XML scraper with launchpadlib.) There's no rush, since our scraper works currently, but we'd like to make the transition to a supported interface. :)
}}}

[Julian Edwards (julian-edwards)]

From bug 316803, we need to make sure binary publishings are exposed on the API as well. This is likely to happen as part of making the binary files available anyway.

[Julian Edwards (julian-edwards)]

Lots of builds properties are now exposed in RF 7825, including a new getBuilds() custom method on source publications to retrieve the builds.

[Julian Edwards (julian-edwards)]

Kees:

>To just be clear, for USN publication we need:
> - all .changes files (including source)

Why do you need these? Is it just for the file checksums? If so, I can present those in a lot nicer way so you don't re-parse the changes file.

> - all binary files generated from the build

Do you mean the repository URL for them?

Also, do you generate the USNs *after* the package is unembargoed? This would simplify our code a lot if you did :)

[Jamie Strandboge (jdstrand)]

Re changes and binary files>

We parse the changes file so we can generate the following types of entries in our template USN publication script (usn.py is part of usn-tool):
usn.py $DB $USN --release intrepid --package squid --source-version 2.7.STABLE3-1ubuntu2.1
usn.py $DB $USN --release intrepid --package squid --binary-version 2.7.STABLE3-1ubuntu2.1

usn.py $DB $USN --release intrepid --arch source --url http://security.ubuntu.com/ubuntu/pool/main/s/squid/squid_2.7.STABLE3-1ubuntu2.1.dsc --url-size 1252 --url-md5 6953f88d6f4825daabd9e77bd0fa1a88
usn.py $DB $USN --release intrepid --arch source --url http://security.ubuntu.com/ubuntu/pool/main/s/squid/squid_2.7.STABLE3.orig.tar.gz --url-size 1782040 --url-md5 a4d7608696e2b617aa5853c7d23e25b0
usn.py $DB $USN --release intrepid --arch source --url http://security.ubuntu.com/ubuntu/pool/main/s/squid/squid_2.7.STABLE3-1ubuntu2.1.diff.gz --url-size 303042 --url-md5 9132293f589a71ae3f771e1ae6de30f1

usn.py $DB $USN --release intrepid --arch all --url http://security.ubuntu.com/ubuntu/pool/main/s/squid/squid-common_2.7.STABLE3-1ubuntu2.1_all.deb --url-size 495876 --url-md5 b6d1e76b140c792297c14382a06ed3e3
usn.py $DB $USN --release intrepid --arch amd64 --url http://security.ubuntu.com/ubuntu/pool/universe/s/squid/squid-cgi_2.7.STABLE3-1ubuntu2.1_amd64.deb --url-size 119880 --url-md5 27ff06a902debe143acb7b3959fb1c52
usn.py $DB $USN --release intrepid --arch powerpc --url http://ports.ubuntu.com/pool/main/s/squid/squid_2.7.STABLE3-1ubuntu2.1_powerpc.deb --url-size 777958 --url-md5 b9d530e92ad4638fb8d169ef55eb33f4

So what we get from the changes is the release, version, arch, size, and checksum of the debs and source packages. Our script calculates the url using:
1. the architecture in the changes file (we have a mapping of which archs go to security.u.c vs ports.u.c)
2. the component we find based on Sources and Packages files that we locally mirror (icky).

Re USN generation> Yes. We ask a LOSA to unembargo and then we start to generate our USN. You can see our full process at https://wiki.ubuntu.com/SecurityUpdateProcedures#Soyuz.

[Julian Edwards (julian-edwards)]

The changes file text will be available as a changes_file_text property on the source publication as of revision 8085 on edge.

Note that this is able to proxy the file from the restricted librarian so you can see the data before unembargoing.

[Julian Edwards (julian-edwards)]

Unfortunately that last change is part-broken and will OOPS when the changes file text is not UTF-8 compatible. I will fix it in 2.2.4.

[Julian Edwards (julian-edwards)]

There's now a changes_file_url property on the source publication. You can use it to download the changes file separately.

[Julian Edwards (julian-edwards)]

revno 8245 adds binary publications to the webservice.

Kees/Jamie/Cody I think that's everything you need now. Can you verify and I can close this bug (at last!)