need API to discover where a given package is copied from

[Robert Bruce Park]

What I have: source package names and versions of packages that exist in -proposed.

What I need: to discover what PPAs these packages were copied from, if any.

The current API exposes package_upload.copy_source_archive_link which contains the information I need:

https://launchpad.net/+apidoc/1.0.html#package_upload

However there is not currently a way to get from "package name & version" to the relevant package_upload, as packageupload_link is always blank in the source_package_publishing_history:

wget -q -O- 'https://api.launchpad.net/1.0/ubuntu/+archive/primary?ws.op=getPublishedSources&pocket=Proposed&source_name=indicator-network&distro_series=https%3A%2F%2Fapi.launchpad.net%2F1.0%2Fubuntu%2Fzesty&version=0.9.0%2B17.04.20161205.1-0ubuntu1'

Ideally there could be a new method on the archive class which gets this information directly, as my preference would be a single server roundtrip. If the packageupload_link field is fixed to contain the package_upload that would be suboptimal as it would require two server roundtrips to get the info I need from the info I start with.

[Robert Bruce Park (robru)]

I believe "most recent hop" is the information I'd want, as opposed to originating archive, because I am trying to group by bileto PPAs, and sometimes bileto PPAs get sources copied into them, in those cases I want to group by the bileto PPA, not the originating PPA.

[Robert Bruce Park (robru)]

Here's a reproducer using lp-shell instead of wget:

>>> lp.load('ubuntu/+archive/primary').getPublishedSources(pocket='Proposed', source_name='lib', distro_series='/ubuntu/zesty')[0].packageupload
>>>

Expected return value: A package_upload object
Actual return value: None

This happens on every source_package_publishing_history, without exception.

[Steve Langasek (vorlon)]

On Fri, Feb 24, 2017 at 06:45:57PM -0000, Robert Bruce Park wrote:
> Here's a reproducer using lp-shell instead of wget:

> >>> lp.load('ubuntu/+archive/primary').getPublishedSources(pocket='Proposed', source_name='lib', distro_series='/ubuntu/zesty')[0].packageupload
> >>>

>>> obj=lp.load('ubuntu/+archive/primary').getPublishedSources(pocket='Proposed', source_name='lib', distro_series='/ubuntu/zesty')[0]
>>> obj
<source_package_publishing_history at https://api.launchpad.net/devel/ubuntu/+archive/primary/+sourcepub/7369578>
>>> obj.display_name
u'afflib 3.7.15-1 in zesty'
>>>

This isn't an upload, it's a sync.

>>> obj=lp.load('ubuntu/+archive/primary').getPublishedSources(pocket='Proposed', source_name='glibc', distro_series='/ubuntu/zesty')[0]
>>> obj.packageupload
<package_upload at https://api.launchpad.net/devel/ubuntu/zesty/+upload/13928233>
>>>

Picking a package that *was* uploaded does result in a link to a
packageupload record.

> This happens on every source_package_publishing_history, without
> exception.

So, not exactly.

But this does appear to affect synced packages, including those that were
synced from ppas.

>>> obj=lp.load('ubuntu/+archive/primary').getPublishedSources(pocket='Proposed', source_name='unity8', exact_match=True, distro_series='/ubuntu/zesty')[0]
>>> obj.packageupload
>>>

But what you can do is:

>>> src=lp.load('ubuntu/+archive/primary').getPublishedSources(pocket='Proposed', source_name='unity8', exact_match=True, distro_series='/ubuntu/zesty')[0]
>>> bin = src.getPublishedBinaries()[0]
>>> bin.build.archive.self_link
u'https://api.launchpad.net/devel/~ci-train-ppa-service/+archive/ubuntu/2405-deletedppa'
>>>

Would this unblock you?

[Robert Bruce Park (robru)]

That does appear to be workable, but it looks like 3 server roundtrips to get the information (getPublishedSources is one, getPublishedBinaries is two, and bin.build.archive looks like a third roundtrip (I'd have to double-check but I suspect that getPublishedBinaries only returns bin.build and then accessing bin.build.archive makes a third roundtrip to load data.

Colin was saying that he'd prefer to solve this by making the info accessible in a single roundtrip and I agree.

[Steve Langasek (vorlon)]

On Sat, Feb 25, 2017 at 02:32:57AM -0000, Robert Bruce Park wrote:
> That does appear to be workable, but it looks like 3 server roundtrips
> to get the information (getPublishedSources is one, getPublishedBinaries
> is two, and bin.build.archive looks like a third roundtrip (I'd have to
> double-check but I suspect that getPublishedBinaries only returns
> bin.build and then accessing bin.build.archive makes a third roundtrip
> to load data.

> Colin was saying that he'd prefer to solve this by making the info
> accessible in a single roundtrip and I agree.

Agreed that this should be fixed, but as there's no fix available today, I
thought the workaround might unblock you despite being less performant.

[Robert Bruce Park (robru)]

Ok, please review workaround:

https://code.launchpad.net/~robru/britney/+git/britney2-ubuntu/+merge/318548

(note that this depends on changes in the email policy which is also awaiting review)

[Robert Bruce Park (robru)]

Haha, turns out the lp-shell way is actually 4 roundtrips because it needs to load .build from build_link and then .archive is loaded from archive_link. Fortunately none of that info is actually required and I'm able to extract the PPA out of build_link URL in only 2 server roundtrips.