Contents generation should be done from the database

Bug #796997 reported by Steve Kowalik
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Launchpad itself
Triaged
Low
Unassigned

Bug Description

The contents generation as it currently stands is only done for the Ubuntu PRIMARY archive, shares a lock file with the publisher, and takes an hour or more. It needs to die horribly.

The rest of this bug report is a brain-dump of my current thought process about this.

This can be fixed iterably, by first creating a table (say, BinaryPackageContents), that contains a bpn id, das id, archive, filename. I have not given much thought to indexing this table, sadly.

The second step would be to write a garbo script that iterates over published BPRs with no expired BPRFs and adds rows to BPC. Expect run time is a week, perhaps less, since unlike PSC, we don't need to extract the source. We can grab the .deb from the librarian, run dpkg-deb -c over it, and parse the output.

The third step would be to hook into the publisher to create and delete rows from BPC when it marks a BPR as SUPERSEDED or PUBLISHED. We may also need to do some trickery in terms of deletion of binary packages.

The fourth step would be to write a hook script for the publisher that generates the contents file for a given archive, and turn off the cronscript for the current script.

Related branches

Steve Kowalik (stevenk)
description: updated
William Grant (wgrant)
Changed in launchpad:
status: Confirmed → Triaged
tags: added: soyuz-publish
removed: soyuz
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :
Changed in launchpad:
assignee: nobody → Steve Kowalik (stevenk)
tags: added: qa-needstesting
Changed in launchpad:
status: Triaged → In Progress
Steve Kowalik (stevenk)
tags: added: qa-untestable
removed: qa-needstesting
Revision history for this message
Julian Edwards (julian-edwards) wrote :

Still in progress?

Revision history for this message
Steve Kowalik (stevenk) wrote :

Currently waiting for all three DB servers to have more disk, then I can land my population branch.

Steve Kowalik (stevenk)
Changed in launchpad:
status: In Progress → Triaged
assignee: Steve Kowalik (stevenk) → nobody
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.