setting partner component using queue override causes carnage

[Colin Watson]

The partner archive team asked us to sync sun-java6 from Debian unstable into the partner archive. I tried to do this in what seemed like the natural way (after an attempt to do so using the LP API failed; I'll file separate bugs about that) and carnage ensued.

I used sync-source.py to do a sync as usual, and flushed that into the NEW queue. I overrode the component to 'partner' using the 'queue' script, and accepted it. This immediately failed to build:

  http://launchpadlibrarian.net/39830889/buildlog_ubuntu-lucid-i386.sun-java6_6.18-2_CHROOTWAIT.txt.gz

More seriously, the pending source publication broke publish-distro.py. In the debug log, I see:

  2010-02-26 20:05:38 DEBUG Added /srv/launchpad.net/ubuntu-archive/ubuntu/pool/partner/s/sun-java6/sun-java6_6.18-2.dsc from library
  2010-02-26 20:05:38 DEBUG Added /srv/launchpad.net/ubuntu-archive/ubuntu/pool/partner/s/sun-java6/sun-java6_6.18-2.diff.gz from library
  2010-02-26 20:05:40 DEBUG Added /srv/launchpad.net/ubuntu-archive/ubuntu/pool/partner/s/sun-java6/sun-java6_6.18.orig.tar.gz from library
  [...]
  2010-02-26 20:25:29 DEBUG * Step D: Generating Release files.
  2010-02-26 20:25:29 DEBUG Writing Release file for lucid/partner/source
  2010-02-26 20:25:29 ERROR Unexpected exception while doing release files
   -> http://launchpadlibrarian.net/39832592/kzwKrh60zH8ijT4IqFaUdAJvpZr.txt ([Errno 2] No such file or directory: u'/srv/launchpad.net/ubuntu-archive/ubuntu/dists.new/lucid/partner/source/Release')
  Traceback (most recent call last):
    File "/srv/launchpad.net/codelines/current/scripts/publish-distro.py", line 32, in <module>
      publishdistro.run_publisher(options, txn)
    File "/srv/launchpad.net/codelines/soyuz-production-rev-8939/lib/lp/soyuz/scripts/publishdistro.py", line 198, in run_publisher
      options.careful or options.careful_apt)
    File "/srv/launchpad.net/codelines/soyuz-production-rev-8939/lib/lp/soyuz/scripts/publishdistro.py", line 92, in try_and_commit
      func(*args)
    File "/srv/launchpad.net/codelines/soyuz-production-rev-8939/lib/lp/archivepublisher/publishing.py", line 350, in D_writeReleaseFiles
      self._writeDistroSeries(distroseries, pocket)
    File "/srv/launchpad.net/codelines/soyuz-production-rev-8939/lib/lp/archivepublisher/publishing.py", line 492, in _writeDistroSeries
      distroseries, pocket, component, architecture, all_files)
    File "/srv/launchpad.net/codelines/soyuz-production-rev-8939/lib/lp/archivepublisher/publishing.py", line 579, in _writeDistroArchSeries
      component, architecture, "Release"), "w")
  IOError: [Errno 2] No such file or directory: u'/srv/launchpad.net/ubuntu-archive/ubuntu/dists.new/lucid/partner/source/Release'

It seems clear from the paths displayed here that sun-java6 ended up in the primary archive with component 'partner', rather than in the partner archive. This should not have been allowed. It seems that the archive is erroneously selected at upload time, rather than at the point of acceptance from the queue.

We removed sun-java6 again as a short-term measure to repair the publisher, and this morning I tried it again, only this time I manually edited the .changes file, replacing "non-free/java" with "partner/java", before flushing the sync into the queue. This has successfully built. The binaries still required an override, so we'll see if that breaks the publisher again.

[Michael Nelson (michael.nelson)]

In addition, this is what has triggered bug 536641 (the same release in two main archives, whereas our traversal currently assumes this will not happen).

[Michael Nelson (michael.nelson)]

So the override is working in that it is setting the source package releases' upload_archive correctly:

12:34 < noodles> bigjools: and one more when you've a minute: http://pastebin.ubuntu.com/393966/ , my suspicion from the code is that it *is* overriding spr.upload_archive correctly, but we're using queue_item.archive when creating the pubs.
12:35 < bigjools> haha
12:35 < bigjools> SPR id | SPR Version | SPR Component | SPR upload_archive name
12:35 < bigjools> --------+-------------+---------------+-------------------------
12:35 < bigjools> 549031 | 6-16-1 | multiverse | primary
12:35 < bigjools> 549031 | 6-16-1 | multiverse | primary
12:35 < bigjools> 468586 | 6-15-1 | multiverse | primary
12:35 < bigjools> 618325 | 6.18-2 | partner | partner <-------
12:35 < bigjools> 620648 | 6.18-2 | partner | partner

We're just not using it when creating the actual publishings, but are defaulting to the queu_items archive attribute, :/

11:29 < noodles> bigjools: could you run this slightly updated query (adds display of the component names for the sun-java6 pkg):
                 http://pastebin.ubuntu.com/393924/
11:29 < noodles> (no rush)
11:30 < bigjools> id | version | (queue item archive) name | name
11:30 < bigjools> --------+---------+---------+------------
11:30 < bigjools> 549031 | 6-16-1 | primary | multiverse
11:30 < bigjools> 468586 | 6-15-1 | primary | multiverse
11:30 < bigjools> 618325 | 6.18-2 | primary | partner <-------
11:30 < bigjools> 620648 | 6.18-2 | partner | partner

[Ursula Junque (ursinha)]

Fixed in stable r10563 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/10563>

[Michael Nelson (michael.nelson)]

QA:
Using db-devel before the change landed:
1. Made sure I had permission to upload to the main archive:
{{{
ubuntu.main_archive.newComponentUploader(me,main)
}}}
2. Set lucid's status to FROZEN so my uploads would not be auto accepted (later realised I could probably have used a different policy when processing uploads):
{{{
ubuntu['lucid'].status = SeriesStatus.FROZEN
}}}
3. dput a new version of pocketsphinx
4. processed uploads
{{{
$LPCURRENT/scripts/process-upload.py -C insecure -v $QUEUE -a <email address hidden>
}}}
5. Used the queue tool to override to partner and then accept:
{{{
scripts/ftpmaster-tools/queue --queue=unapproved --component=partner --exact-match override source pocketsphinx
scripts/ftpmaster-tools/queue --queue=unapproved --exact-match accept source pocketsphinx
}}}
Resulting builds target Ubuntu primary archive:
http://people.canonical.com/~michaeln/qa/529926_before.png

Manually set the status of the source publishing to DELETED, updated db-devel a rev. including the fix, and repeated steps 3-5 which results in builds targeting the partner archive:
http://people.canonical.com/~michaeln/qa/529926-after.png