git ubuntu export-orig --no-pristine-tar exception

[Rafael David Tinoco]

Currently, if I have:

(c)inaddy@ctdbdisco:~/work/sources/ubuntu/samba$ git remote -v
pkg https://git.launchpad.net/~usd-import-team/ubuntu/+source/samba (fetch)
pkg ssh://<email address hidden>/~usd-import-team/ubuntu/+source/samba (push)
rafaeldtinoco https://git.launchpad.net/~rafaeldtinoco/ubuntu/+source/samba (fetch)
rafaeldtinoco ssh://<email address hidden>/~rafaeldtinoco/ubuntu/+source/samba (push)

If I'm using a local lp1831381-bionic-devel-ctdb based on pkg/samba/bionic-devel:

(c)inaddy@ctdbdisco:~/work/sources/ubuntu/samba$ git branch
* lp1831381-bionic-devel-ctdb
  lp1831381-disco-devel-ctdb
  lp1831381-disco-devel-ctdb-ppa
  lp1831381-eoan-devel-ctdb
  ubuntu/devel

And try to get orig source with --no-pristine-tar:

(c)inaddy@ctdbdisco:~/work/sources/ubuntu/samba$ git ubuntu export-orig --no-pristine-tar
07/12/2019 01:47:53 - INFO:New upstream version detected (4.7.6+dfsg~ubuntu) which is after the last published upstream version (4.10.0+dfsg).
Traceback (most recent call last):
  File "/snap/git-ubuntu/456/bin/git-ubuntu", line 11, in <module>
    load_entry_point('gitubuntu==0.7.4', 'console_scripts', 'git-ubuntu')()
  File "/snap/git-ubuntu/456/lib/python3.6/site-packages/gitubuntu/__main__.py", line 243, in main
    sys.exit(args.func(args))
  File "/snap/git-ubuntu/456/lib/python3.6/site-packages/gitubuntu/exportorig.py", line 122, in cli_main
    if len(tarballs) > 0:
TypeError: object of type 'NoneType' has no len()

The exception above happens.

If I use the regular "git ubuntu export-orig":

(c)inaddy@ctdbdisco:~/work/sources/ubuntu/samba$ git ubuntu export-orig
07/12/2019 01:49:35 - INFO:Successfully fetched:
../samba_4.7.6+dfsg~ubuntu.orig.tar.gz
using fetch_orig_from_pristine_tar(source=ubuntu)

It works correctly.

Note that, because of the exception, I also lose the changes I was going to have added to quilt using "dpkg-source --commit" (which needs the orig source to exist).

As requested by Robie, I'm opening this bug so it can be better tracked.

Thanks!

[Frank Heimes (fheimes)]

I'm running into a similar situation:

$ git ubuntu export-orig --no-pristine-tar
03/02/2021 18:51:02 - INFO:New upstream version detected (3.9.0+dfsg) which is after the last published upstream version (3.15.1+dfsg).
Traceback (most recent call last):
  File "/snap/git-ubuntu/474/bin/git-ubuntu", line 11, in <module>
    load_entry_point('gitubuntu==1.0', 'console_scripts', 'git-ubuntu')()
  File "/snap/git-ubuntu/474/lib/python3.6/site-packages/gitubuntu/__main__.py", line 279, in main
    sys.exit(args.func(args))
  File "/snap/git-ubuntu/474/lib/python3.6/site-packages/gitubuntu/exportorig.py", line 122, in cli_main
    if len(tarballs) > 0:
TypeError: object of type 'NoneType' has no len()

However:
$ git-ubuntu export-orig
03/02/2021 19:08:40 - INFO:Successfully fetched:
../opencryptoki_3.9.0+dfsg.orig.tar.xz
using fetch_orig_from_pristine_tar(source=ubuntu)

[Utkarsh Gupta (utkarsh)]

Hello,

I am hitting the same bug even w/o "--no-pristine-tar".

$ git ubuntu export-orig
04/07/2021 22:34:30 - WARNING:No pristine-tar data found for 1.6.6
04/07/2021 22:34:42 - INFO:New upstream version detected (1.6.6) which is after the last published upstream version (1.6.4).
Traceback (most recent call last):
  File "/snap/git-ubuntu/477/usr/bin/git-ubuntu", line 11, in <module>
    load_entry_point('gitubuntu==1.0', 'console_scripts', 'git-ubuntu')()
  File "/snap/git-ubuntu/477/usr/lib/python3/dist-packages/gitubuntu/__main__.py", line 269, in main
    sys.exit(args.func(args))
  File "/snap/git-ubuntu/477/usr/lib/python3/dist-packages/gitubuntu/exportorig.py", line 117, in cli_main
    if len(tarballs) > 0:
TypeError: object of type 'NoneType' has no len()

[Bryce Harrington (bryce)]

Hi Utkarsh,

That traceback is a bit generic, it gets triggered whenever build.fetch_orig() finds no tarballs, for whatever reason. The actual bug in these cases is the 'whatever reason' part but the traceback doesn't really tell anything about that. The steps to reproduce are important in this case, due to that.

For additional background, build.fetch_orig() is just a wrapper that iterates through a set of different tarball-retrieval mechanisms. It iterates through this set until one of them works. "pristine-tar" is one of the first attempted but (for me at least) it rarely works. (Using "--no-pristine-tar" is kind of irrelevant in such cases since it isn't going to work anyway.)

It also tries to download from launchpad, which is what usually works for me, although it's subject to launchpad outages and so on. Notably, the launchpad mechanism also fails if you're doing a merge, since the new tarball obviously isn't uploaded to launchpad yet. In this case, the established workaround is to do:

  $ git ubuntu export-orig --for-merge

It seems like this should be something git-ubuntu could figure out for itself, but for now just remember to use '--for-merge' when you're doing export-orig for merges.

The other mechanisms attempt to pull from local caches, which can make export-orig succeed even in cases where the above two mechanisms normally would fail.

A workaround I have been using often when git ubuntu export-orig fails, is to run:

  $ uscan -dd

Note that this works *only* for packages that have debian/watch files with valid url patterns, but I've usually been lucky. It occurs to me that adding 'uscan -dd' as another mechanism for fetch_orig() might be worth considering, although Robie might know a reason why this isn't implemented there already...

Anyway, back to the traceback in question; this is actually a side-effect of the above, because exportorig.py's code tries to do len(tarballs) but tarballs==None, for any of the reasons outlined above. LP: #1868631 shows my original idea for papering over this crash, although looking at it now I see the actual coding error is in fetch_orig()... I'll send in an MP for that.

[Bryce Harrington (bryce)]

And the MP is...
https://code.launchpad.net/~bryce/usd-importer/+git/usd-importer/+merge/405422

This doesn't solve the root cause, just the traceback side-effect.

[Bryce Harrington (bryce)]

Fwiw, the original bug report shows:

07/12/2019 01:47:53 - INFO:New upstream version detected (4.7.6+dfsg~ubuntu) which is after the last published upstream version (4.10.0+dfsg).

IOW, the root problem here was that git-ubuntu thought version 4.7.6 > 4.7.10. Frank's repro shows a similar error thinking that 3.9.0+dfsg > 3.15.1+dfsg.

Both of these are instances of the same problem reported in #1868631 which Robie fixed in revision d13680cf5787e4cf6939c60ce71fd089a4c66d0a.

So I think this bug can be closed.