support dist-upgrader-all pocket copy

[Michael Vogt]

Please provide copying file from "hardy-proposed" to "hardy-updates" for the "dist-upgrader-all" files.

Current the latest update-manager 0.87.27 is in hardy-updates but it does only show up in:
http://archive.ubuntu.com/ubuntu/dists/hardy-proposed/main/dist-upgrader-all/

when it should (also) be available in hardy-updates.

Because it is important to get this to our users, I would like to ask as a workaround to copy it manually in the archive for now. I subscribe ubuntu-archive (assuming they have the required permissions).

Thanks,
 Michael

[Michael Vogt (mvo)]

Please make sure to copy it and not move it (the old location is referenced currently in the meta-release file).

[Martin Pitt (pitti)]

Michael confirmed that we don't need manual copying, u-manager has a workaround. Unsub'ing ubuntu-archive.

[Michael Vogt (mvo)]

Could we please make this higher priority? it would be nice to be able to point to $distro-updates/current instead of the current approach where I manually track $distro-proposed/$explicit-version

[Julian Edwards (julian-edwards)]

Michael - we've talked about this and we think it's not an easy piece of work, but it also has a lot in common with features we need to include for our 3.0 release such as the unembargoing API and native source syncing. Therefore, I'll schedule it for 2.2.2!

[Julian Edwards (julian-edwards)]

I feel bad about slipping this again :( Sorry Michael, other stuff keeps usurping you.

[Julian Edwards (julian-edwards)]

Work is ongoing on this feature, it's mostly done via the infrastructure changes we're implementing here:
https://blueprints.edge.launchpad.net/soyuz/+spec/soyuz-delayed-copies

The finishing touches for this particular feature will be applied soon.

[Julian Edwards (julian-edwards)]

Progess has stalled because we found a problem with the new delayed copy mechanism, where it doesn't handle copying between pockets. We need to fix that first before this bug can be addressed.

[Matthew Revell (matthew.revell)]

Retargeting to 3.0, which is the new name for 2.2.9. Haven't renamed the milestone as the 3.0 milestone is in a different series.

[Julian Edwards (julian-edwards)]

This fails because the same version already exists; PackageUpload.verifyBeforeAccept() blows up.

[Michael Vogt (mvo)]

Is there any news on this? I'm currently working around this by using absolute version numbers in the meta-release file, but that is far from optimal (it means that I need to updates the files manually everytime something goes from -proposed to -updates).

[Julian Edwards (julian-edwards)]

Michael, this turned out to be a very tough problem to solve and it stalled on the problem above. It's not getting any attention now because we have other priorities. If you want to escalate this through Robbie I'd be happy re-prioritise it.

[Colin Watson (cjwatson)]

Michael: In the meantime, I updated archive administration procedures to include copying the custom upload to -updates manually. Feel free to bug an archive administrator if you notice that this hasn't happened after a particular update.

[Launchpad QA Bot (lpqabot)]

Fixed in stable r15491 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/15491>.

[Colin Watson (cjwatson)]

  Module lazr.jobrunner.jobrunner, line 194, in runJobHandleError
    self.runJob(job, fallback)
  Module lp.services.job.runner, line 270, in runJob
    super(BaseJobRunner, self).runJob(IRunnableJob(job), fallback)
  Module lazr.jobrunner.jobrunner, line 162, in runJob
    job.run()
  Module lp.soyuz.model.packagecopyjob, line 495, in run
    self.attemptCopy()
  Module lp.soyuz.model.packagecopyjob, line 562, in attemptCopy
    unembargo=self.unembargo)
  Module lp.soyuz.scripts.packagecopier, line 681, in do_copy
    sponsor=sponsor, packageupload=packageupload)
  Module lp.soyuz.scripts.packagecopier, line 807, in _do_direct_copy
    custom_copier.copyUpload(custom)
  Module lp.soyuz.scripts.custom_uploads_copier, line 134, in copyUpload
    package_upload.setAccepted()
  Module lp.soyuz.model.queue, line 289, in setAccepted
    for source in self.sources:
  Module storm.sqlobject, line 523, in __iter__
    for item in self._result_set:
  Module storm.sqlobject, line 505, in _result_set
    self._finished_result_set = self._finish_result_set()
  Module storm.sqlobject, line 489, in _finish_result_set
    result = self._prepare_result_set()
  Module storm.sqlobject, line 483, in _prepare_result_set
    return store.find(find_spec, *args)
  Module storm.store, line 205, in find
    self.flush()
  Module storm.store, line 503, in flush
    self._flush_one(obj_info)
  Module storm.store, line 540, in _flush_one
    result = self._connection.execute(expr)
  Module storm.databases.postgres, line 260, in execute
    result = Connection.execute(self, Returning(statement), params)
  Module storm.database, line 238, in execute
    raw_cursor = self.raw_execute(statement, params)
  Module storm.databases.postgres, line 276, in raw_execute
    return Connection.raw_execute(self, statement, params)
  Module storm.database, line 322, in raw_execute
    self._check_disconnect(raw_cursor.execute, *args)
  Module storm.database, line 371, in _check_disconnect
    return function(*args, **kwargs)
ProgrammingError: permission denied for relation packageuploadcustom

[Colin Watson (cjwatson)]

r15499 is good. Installer images are copied over from -proposed to -updates exactly as desired. (I haven't explicitly tested dist-upgrader tarballs, but they're very similar to installer images so should be fine too.)

The one flaw I can see is that it copies a little too much - namely, it also copies Rosetta translations, which fail because the copied custom upload doesn't have a .sourcepackagerelease. This is harmless, since we don't actually want or need to copy those translations. I'll prepare another follow-up branch that restricts the copied uploads to only ones in copyable_types.

[Launchpad QA Bot (lpqabot)]

Fixed in stable r15499 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/15499>.