initialise-from-parent does not copy upstream packaging links

[Colin Watson]

The process of initialising a new distrorelease based on its parent distrorelease (done by scripts/ftpmaster-tools/initialise-from-parent.py) does not copy upstream packaging links. For example, at the time of writing shortly after opening /ubuntu/gutsy, https://beta.launchpad.net/usb-discover/+packages has a link to /ubuntu/feisty/+source/usb-discover but not to /ubuntu/gutsy/+source/usb-discover. This information should be carried over automatically when opening a new distrorelease; if it later becomes inaccurate, the right approach is to override it in the usual gardening way in the few cases where that happens, rather than requiring lots of work in the common case.

(For what it's worth, this discourages me from bothering to fill in the upstream links. I've spent some time doing so in the past, but why should I spend time on this when the data will all be invalidated in six months? I think it's really important to carry this over automatically from release to release.)

[Celso Providelo (cprov)]

I don't understand the packaging system very well, neither how exactly it should be used in productseries.

However, IIRC, lifeless introduced it in a way it could be recursively found based on distrorelease.parent. This way the Packaging.distrorelease only represents when/where the packaging relationship was established and it continues to be valid for all derivations of that distrorelease context until someone establishes the opposite.

As you can see in: https://beta.launchpad.net/ubuntu/gutsy/+source/usb-discover/+packaging

As far as I could see by checking the ProductSeries and Products, those methods are not following the Packaging rule:

 * Expand packaging across derivations until we find a new explicit Packaging entry.

Anyway, I agree that this rule sounds annoying and very unnatural to follow (and hard & expensive to do using sqlobject).

Perhaps ddaa is in a better position to explain/discuss this.

[Robert Collins (lifeless)]

I dont recall the details offhand. I will spend some time after UDS looking at this as I come back up to speed on Soyuz.

From memory Mark was concerned about the storage cost of explicitly recording each distros packaging relationship versus only recording it when it changes.

As far as SQL cost is concerned, a single query should be able to generate the implicit data, so I dont see why it has to be expensive. Its not per source package upload after all.

[Celso Providelo (cprov)]

Yes, the reasons to have Packaging as it is are legit.

IMHO, what Colin is pointing as 'uncomfortable' is the fact that the information is *implicit* in the UI.

As I said before, we probably only need to expand Packaging presentation to avoid the feeling "Packaging relationships are not derived." or/and add proper documentation explaining the implicit details and their consequences to the system.

[Colin Watson (cjwatson)]

If it indicated that the release was just the earliest release to which this packaging relationship applied, that would be fine. I had no idea it was like that until reading this discussion - the only clue is that https://beta.launchpad.net/ubuntu/gutsy/+source/usb-discover has an "Upstream associations" section.

In fact, https://beta.launchpad.net/ubuntu/edgy/+source/usb-discover has an "Upstream associations" section too, which reinforces my feeling that I have no idea what's going on when https://beta.launchpad.net/usb-discover/+packages says "feisty".

[Robert Collins (lifeless)]

On Mon, 2007-04-30 at 11:36 +0000, Celso Providelo wrote:
> Yes, the reasons to have Packaging as it is are legit.
>
> IMHO, what Colin is pointing as 'uncomfortable' is the fact that the
> information is *implicit* in the UI.
>
> As I said before, we probably only need to expand Packaging presentation
> to avoid the feeling "Packaging relationships are not derived." or/and
> add proper documentation explaining the implicit details and their
> consequences to the system.

Agreed on documentation - the help tab perhaps. In terms of UI, it would
be really nice when looking upstream to see something like:
series branch distrorelease packageversion
...

e.g.
trunk hoary 0.6-1
trunk dapper 0.8-1
0.11 edgy 0.11ubuntu1-1
0.15 feisty 0.15-1

where all those things are linked appropriately.

And looking at the package source it would likewise be nice to see the
series it was published from (at ubuntu/+source/bzr). I wonder if a more
pithy view than the entire publishing history would be reasonable? (Mind
you that is unrelated ;).

Cheers,
Rob
--
GPG key available at: <http://www.robertcollins.net/keys.txt>.

[Jeroen T. Vermeulen (jtv)]

Several Product methods/properties use the packaging links without taking this form of inheritance into account: sourcepackages, distrosourcepackages, getPackage translatable_packages. Similar for ProductSeries.

[Curtis Hovey (sinzui)]

The best example I can see of this issue is gnome-terminal.
It was listed as a translatable package in Jaunty
    https://edge.launchpad.net/ubuntu/jaunt/+packaging

But it is not linked in Karmic, yet we know being a code GNOME app which repo and series it came from
    https://edge.launchpad.net/ubuntu/lkarmic/+packaging

Then Lucid is opened and the lost data is carried forward, and as I can see just a few weeks into the Lucid series, there are very few known upstream packaging links:
    https://edge.launchpad.net/ubuntu/lucid/+packaging

The build system never stops. Lucid was started with the exact same packages as Jaunty final. The Lucid's packing should have been identical Jaunty at the moment the series was started. I image we can do two things to address this

a) Create a script that copies the packaging from from the parent series and run it when the series is created.
b) Use a variation of that script to add the missing links to Lucid. The script can copy the link from Karmic to Lucid if both have the same version.

[Curtis Hovey (sinzui)]

Fixed in launchpad devel r9922.

[Robert Collins (lifeless)]

On Sat, 2009-11-21 at 15:10 +0000, Curtis Hovey wrote:
> Fixed in launchpad devel r9922.

Awesome. Does this fix also go back through existing distros and do the
copy that wasn't done?

-Rob

[Curtis Hovey (sinzui)]

On Sat, 2009-11-21 at 16:41 +0000, Robert Collins wrote:
> On Sat, 2009-11-21 at 15:10 +0000, Curtis Hovey wrote:
> > Fixed in launchpad devel r9922.
>
> Awesome. Does this fix also go back through existing distros and do the
> copy that wasn't done?

That was a separate bug because the data could be fixed without a
release. We ran script in production yesterday. It updated the packing
links from hoary through lucid
    https://edge.launchpad.net/ubuntu/lucid/+packaging

--

__C U R T I S C. H O V E Y_______
Guilty of stealing everything I am.

[Robert Collins (lifeless)]

On Sat, 2009-11-21 at 17:32 +0000, Curtis Hovey wrote:
> On Sat, 2009-11-21 at 16:41 +0000, Robert Collins wrote:
> > On Sat, 2009-11-21 at 15:10 +0000, Curtis Hovey wrote:
> > > Fixed in launchpad devel r9922.
> >
> > Awesome. Does this fix also go back through existing distros and do the
> > copy that wasn't done?
>
> That was a separate bug because the data could be fixed without a
> release. We ran script in production yesterday. It updated the packing
> links from hoary through lucid
> https://edge.launchpad.net/ubuntu/lucid/+packaging

\o/

-Rob

[Curtis Hovey (sinzui)]

Fixed released in launchpad-project 3.1.11.