get-orig-source priority should be higher than uscan

[Martin Pitt]

Binary package hint: bzr-builddeb

I maintain calibre (lp:~calibre-packagers/calibre/debian/). This needs a get-orig-source rule to build a +dfsg.orig.tar.gz (upstream tar.gz with some files removed). If I just bump the version in debian/changelog, then bd figures out that it needs to get a new orig.tar.gz, but uses uscan for this:

$ bzr bd
Building using working tree
Running in merge mode
Looking for a way to retrieve the upstream tarball
Using apt to look for the upstream tarball.
apt could not find the needed tarball.
Using uscan to look for the upstream tarball.
calibre: Version (0.4.143) available on remote site:
  http://calibre.kovidgoyal.net/downloads/calibre-0.4.143.tar.gz
  (local version is 0.4.143+dfsg, mangled local version number 0.4.143)
calibre: Successfully downloaded updated package calibre-0.4.143.tar.gz
    and renamed it as calibre_0.4.143.orig.tar.gz
bzr: ERROR: Unable to find the needed upstream tarball: calibre_0.4.143+dfsg.orig.tar.gz.

I can't think of a situation where it is correct to use uscan if debian/rules provides a get-orig-source method. Thus I believe provide_with_get_orig_source() should be much higher prioritized than provide_with_uscan(), and also than provide_from_self_by_split(). I'm a bit undecided about provide_from_upstream_branch(), since it's not quite clear to me whether get-orig-source or the explicitly configured upstream branch should win (I think the latter).

I currently don't see a way of overriding the priorities, so I have to call get-orig-source by hand.

Thanks for considering!

[James Westby (james-w)]

On Sat, 2009-03-07 at 22:33 +0000, Martin Pitt wrote:
> Public bug reported:
>
> Binary package hint: bzr-builddeb
>
> I maintain calibre (lp:~calibre-packagers/calibre/debian/). This needs a
> get-orig-source rule to build a +dfsg.orig.tar.gz (upstream tar.gz with
> some files removed). If I just bump the version in debian/changelog,
> then bd figures out that it needs to get a new orig.tar.gz, but uses
> uscan for this:

Hi Martin,

Thanks for the bug report.

It is partly this way round as until recently bzr-builddeb was a lot
more dumb about handling the possibility that it could get the tarball
from more than one place.

Can I ask what purpose the watch file serves if the "get-orig-source"
target is needed? Is it just for detecting if there is a new upstream
version, rather than for downloading it?

An added complication is that "get-orig-source" is not a great interface
for doing this. Debian policy defines it as a target that gets the
latest available source, whereas we need to get the source corresponding
to the version being built. This could mean that it does an expensive
operation, just to throw the result away as it does not produce the
right version. This is more of a problem with the idea of a
get-orig-source rule though, and I think it would be silly to prevent
the tool from working well for you because of this.

Perhaps there could be a way to specify the preferred way of getting
the source, though making it work by default would be preferable.

Thanks,

James

[Martin Pitt (pitti)]

James Westby [2009-03-07 23:37 -0000]:
> Can I ask what purpose the watch file serves if the "get-orig-source"
> target is needed? Is it just for detecting if there is a new upstream
> version, rather than for downloading it?

Exactly. Tools like DEHS and the Debian PTS use the watch files to
check whether the version is current.

> An added complication is that "get-orig-source" is not a great interface
> for doing this. Debian policy defines it as a target that gets the
> latest available source, whereas we need to get the source corresponding
> to the version being built.

Oh, hm. I actually use it as "get the source that debian/changelog
mentions", since anything else doesn't make sense to me. This is also
being discussed in http://bugs.debian.org/466550

I agree that g-o-s is underspecified, though. I just don't know a
better replacement.

> Perhaps there could be a way to specify the preferred way of getting
> the source,

Seems with above disambiguity that might indeed be required. I haven't
even thought of that, sorry.

[James Westby (james-w)]

On Sun, 2009-03-08 at 00:01 +0000, Martin Pitt wrote:
> James Westby [2009-03-07 23:37 -0000]:
> > Can I ask what purpose the watch file serves if the "get-orig-source"
> > target is needed? Is it just for detecting if there is a new upstream
> > version, rather than for downloading it?
>
> Exactly. Tools like DEHS and the Debian PTS use the watch files to
> check whether the version is current.

I've just re-arranged things in bzr such that it should now work better
for you. Thanks for the report.

> > An added complication is that "get-orig-source" is not a great interface
> > for doing this. Debian policy defines it as a target that gets the
> > latest available source, whereas we need to get the source corresponding
> > to the version being built.
>
> Oh, hm. I actually use it as "get the source that debian/changelog
> mentions", since anything else doesn't make sense to me. This is also
> being discussed in http://bugs.debian.org/466550

Thanks for the link.

> I agree that g-o-s is underspecified, though. I just don't know a
> better replacement.

I've been thinking about this for the last few days, and I have an idea
about what the solution would look like, I just need to find some time
to experiment.

> > Perhaps there could be a way to specify the preferred way of getting
> > the source,
>
> Seems with above disambiguity that might indeed be required. I haven't
> even thought of that, sorry.

No problem, I've gone for the simple solution for now. If someone else
has requirements that conflict with yours then I will have to develop
another solution.

Thanks,

James

[Reinhard Tartler (siretart)]

Martin Pitt <email address hidden> writes:

> I maintain calibre (lp:~calibre-packagers/calibre/debian/). This needs a
> get-orig-source rule to build a +dfsg.orig.tar.gz (upstream tar.gz with
> some files removed). If I just bump the version in debian/changelog,
> then bd figures out that it needs to get a new orig.tar.gz, but uses
> uscan for this

Martin,

I rather think uscan should be improved to create/mangle the
+dfsg.orig.tar.gz package. This way you could greatly simplify your
get-orig-source rule, I'd think.

--
Gruesse/greetings,
Reinhard Tartler, KeyID 945348A4

[Launchpad Janitor (janitor)]

This bug was fixed in the package bzr-builddeb - 2.1.1

---------------
bzr-builddeb (2.1.1) experimental; urgency=low

  * Make get-orig-source priority higher than uscan. Thanks Martin Pitt.
    (LP: #339343).
  * Correct variable used in provide_with_apt. Thanks also to Martin Pitt.
    (LP: #339333)
  * Use hashlib instead of the sha and md5 modules. Thanks Kees Cook.
    (LP: #338525)

 -- James Westby <email address hidden> Mon, 16 Mar 2009 00:55:03 +0000