No way to get from binary package to source package

[Martin Mai]

Binary package hint: python-launchpadlib

This is basically the same bug as bug #53171, but for the Launchpad API:

There should be a way to either determine the "source_package" object or the "source_package_publishing_history" object when having a "binary_package_publishing_history" object.

One could (for example) parse the launchpad page https://launchpad.net/ubuntu/maverick/i386/libgtk2.0-0 for the link to https://launchpad.net/ubuntu/+source/gtk+2.0 or run "dpkg-query -W -f='${Source}' libgtk2.0-0", but a API call would be very handy.

Some attribute like "source_package_name" would also suffice, but "getSourcePackage()" or "getSourcePackagePublishingHistory()" would be really great.

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: python-launchpadlib 1.6.0-0ubuntu1
ProcVersionSignature: Ubuntu 2.6.35-4.5-generic 2.6.35-rc3
Uname: Linux 2.6.35-4-generic x86_64
NonfreeKernelModules: nvidia
Architecture: amd64
Date: Mon Jun 21 23:20:44 2010
InstallationMedia: Ubuntu 10.10 "Maverick Meerkat" - Alpha amd64 (20100602.2)
PackageArchitecture: all
ProcEnviron:
 PATH=(custom, user)
 LANG=en_US.utf8
 SHELL=/bin/bash
SourcePackage: python-launchpadlib

[Martin Mai (mrkanister-deactivatedaccount-deactivatedaccount)]

"ubuntu-bug" actually does it somehow, but I was not able to figure out how by digging through the code.

[Stefano Rivera (stefanor)]

ubuntu-dev-tools would also appreciate this for pull-lp-source.

Currently using dde.debian.net instead.

[Colin Watson (cjwatson)]

syncpackage and (especially) the new auto-sync tool I'm writing to replace 'sync-source.py -a' could make good use of this as well, but only if it's done with the right semantics. The reason is a little complex, so bear with me a moment.

When we sync a source package from Debian to Ubuntu, we want to make sure that it does not accidentally override Ubuntu modifications. (It's OK to do that deliberately, but a developer has to check this.) We do this by checking for an "ubuntu" substring in the version number, and if it's present then we require a force option. All well and good so far.

However, there's a wart. Consider the case where we have two source packages like this, generating a different set of binary packages each:

  libfoo 1.0-1ubuntu1
    -> libfoo1 1.0-1ubuntu1
    -> libfoo-dev 1.0-1ubuntu1

  foo 1.0-1
    -> foo 1.0-1

Upstream decides to merge the library and the program that use it into a single release, so the next version we sync from Debian is:

  foo 1.1-1
    -> libfoo1 1.1-1
    -> libfoo-dev 1.1-1
    -> foo 1.1-1

A simple source check wouldn't spot the problem here, but if we check the binaries too then we can see that we may be discarding some Ubuntu changes, and that this needs a manual check in advance of syncing the source.

Now, sync-source.py handles this case by executing a gruesome pile of SQL which roughly amounts to Archive.getPublishedBinaries, checking the Binary field of the .dsc it's syncing to see which binaries it produces, and checking that each of those binaries has a suitable version in the archive at the moment. This mostly works, and I believe we can replicate that with Archive.getPublishedBinaries right now.

However, there's a problem. In the example above, what happens if foo 1.1-1 fails to build? Perhaps Debian fixes that and then we want to sync their fix, which is foo 1.1-2. However, the published binaries in the archive are still 1.0-1ubuntu1, so even though we've already taken an explicit decision to override those Ubuntu modifications we're now going to have to do so again. (There's also another rarer problem related to packages where the binary version isn't the same as the source version; I've seen one source package that doesn't itself have any Ubuntu modifications but which builds binary packages with an ubuntu version substring.)

This is a periodic annoyance for me as the guy running auto-syncs, since I have to stop and figure out what's going on. I'd like to make the tools deal with it for me. I believe that the correct algorithm would be to look up the source package that owns the binary package, by which I mean not necessarily the one linked from the most recent BPPH, but the most recent source package that wants to build that binary package (per SPR.dsc_binaries or similar), and then I could simply check its version to see if I like it.

It would probably be simplest to have a way to return all source packages that want to build a given binary, and then I can look through those to see which ones I care about. I think this feels natural enough in the LP API because lookups for source and binary publications also return collections. How about Archive.getPublishedS...

[James Westby (james-w)]

On Sat, 07 Jan 2012 01:50:43 -0000, Colin Watson <email address hidden> wrote:
> It would probably be simplest to have a way to return all source
> packages that want to build a given binary, and then I can look through
> those to see which ones I care about. I think this feels natural enough
> in the LP API because lookups for source and binary publications also
> return collections. How about
> Archive.getPublishedSources(binary_name=...), which you could then limit
> by series, binary version, etc. as required? Would that work for
> everyone?

The use case that we had for this the other day was just to get the
source package name for the given binary publication, as there no
information in

  https://launchpad.net/+apidoc/devel.html#binary_package_publishing_history

that can provide it.

I don't see a problem with what you want to provide in addition to
that, assuming that dsc_binaries is pretty reliable.

Thanks,

James

[Łukasz Zemczak (sil2100)]

Hi Colin!

Regarding your proposition in comment #4: Archive.getPublishedSources(binary_name=...) would work perfectly for my needs at least. It seems error proof and I could do the rest of validation myself to be sure that it's the source package that I'm actually interested in. Are there any plans on getting something like this implemented soon? I know it's a low priority issue, but would be nice to have something like that in the LP API. I could finally remove those multiple hacks in all my analysis tools that do the binary -> source conversion.

Thanks!

[Colin Watson (cjwatson)]

I've changed my position on this a little: auto-sync in practice copes most of the time, and most users only need a way to get to a single corresponding SPPH, so I think it's better to just introduce BPPH.getPublishedSource() rather than complicating things for a rare corner case.

It looks like I need this as part of the ddeb-retriever work I'm doing at the moment (or, at least, it's probably going to be rather complicated and slow to work around not having this), so I'll take it.

[Colin Watson (cjwatson)]

Following review, this will in fact end up being BPPH.build.getLatestSourcePublication().

[Launchpad QA Bot (lpqabot)]

Fixed in stable r17420 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/17420>.