Contents generation should be done from the database

Bug #796997 reported by Steve Kowalik on 2011-06-14
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself

Bug Description

The contents generation as it currently stands is only done for the Ubuntu PRIMARY archive, shares a lock file with the publisher, and takes an hour or more. It needs to die horribly.

The rest of this bug report is a brain-dump of my current thought process about this.

This can be fixed iterably, by first creating a table (say, BinaryPackageContents), that contains a bpn id, das id, archive, filename. I have not given much thought to indexing this table, sadly.

The second step would be to write a garbo script that iterates over published BPRs with no expired BPRFs and adds rows to BPC. Expect run time is a week, perhaps less, since unlike PSC, we don't need to extract the source. We can grab the .deb from the librarian, run dpkg-deb -c over it, and parse the output.

The third step would be to hook into the publisher to create and delete rows from BPC when it marks a BPR as SUPERSEDED or PUBLISHED. We may also need to do some trickery in terms of deletion of binary packages.

The fourth step would be to write a hook script for the publisher that generates the contents file for a given archive, and turn off the cronscript for the current script.

Related branches

Steve Kowalik (stevenk) on 2011-06-14
description: updated
William Grant (wgrant) on 2011-06-14
Changed in launchpad:
status: Confirmed → Triaged
tags: added: soyuz-publish
removed: soyuz
Launchpad QA Bot (lpqabot) wrote :
Changed in launchpad:
assignee: nobody → Steve Kowalik (stevenk)
tags: added: qa-needstesting
Changed in launchpad:
status: Triaged → In Progress
Steve Kowalik (stevenk) on 2011-06-22
tags: added: qa-untestable
removed: qa-needstesting
Julian Edwards (julian-edwards) wrote :

Still in progress?

Steve Kowalik (stevenk) wrote :

Currently waiting for all three DB servers to have more disk, then I can land my population branch.

Steve Kowalik (stevenk) on 2012-10-31
Changed in launchpad:
status: In Progress → Triaged
assignee: Steve Kowalik (stevenk) → nobody
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers