Historically adopted upload tags may not satisfy new rules

[Nish Aravamudan]

We are attempting (or will be) to assert that when an upload tag is integrated into the rich history (by pointing an import tag directly at it), the upload tag has an appropriate changelog parent already present in its ancestry (that is an import tag (or reimport tag) of the parent changelog version is ancestor). This was not strictly enforced in the past, and so there are upload tags in the history (e.g., ntp) that do not satisfy our new rules.

$ git ubuntu merge start --force --verbose pkg/ubuntu/devel pkg/debian/sid
05/31/2018 16:40:48 - DEBUG:Executing: git config gitubuntu.lpuser
05/31/2018 16:40:48 - DEBUG:Executing: grep -q "* -ident" /home/nacc/work/imports/ntp/.git/info/attributes
05/31/2018 16:40:48 - DEBUG:Executing: grep -q "* -text" /home/nacc/work/imports/ntp/.git/info/attributes
05/31/2018 16:40:48 - DEBUG:Executing: grep -q "* -eol" /home/nacc/work/imports/ntp/.git/info/attributes
05/31/2018 16:40:48 - DEBUG:Executing: git config gitubuntu.lpuser
05/31/2018 16:40:48 - DEBUG:Executing: sh -c "dpkg-parsechangelog -l- -n1 -SVersion"
05/31/2018 16:40:48 - DEBUG:Executing: sh -c "dpkg-parsechangelog -l- -n1 -o1 -SVersion"
05/31/2018 16:40:48 - DEBUG:Executing: sh -c "dpkg-parsechangelog -l- -n1 -SVersion"
05/31/2018 16:40:49 - DEBUG:Executing: sh -c "dpkg-parsechangelog -l- -n1 -o1 -SVersion"
05/31/2018 16:40:49 - DEBUG:Executing: git status --porcelain
05/31/2018 16:40:49 - DEBUG:Executing: sh -c "dpkg-parsechangelog -l- --format rfc822 -SVersion --all"
05/31/2018 16:40:49 - DEBUG:Executing: git merge-base --is-ancestor e96ae83aca626b8483940fbd6ae2e014aec09924 pkg/ubuntu/devel
05/31/2018 16:40:49 - DEBUG:Executing: sh -c "dpkg-parsechangelog -l- -n1 -SVersion"
05/31/2018 16:40:49 - DEBUG:Executing: sh -c "dpkg-parsechangelog -l- -n1 -o1 -SVersion"
05/31/2018 16:40:49 - DEBUG:Executing: git tag old/ubuntu pkg/ubuntu/devel -f
05/31/2018 16:40:49 - DEBUG:Executing: sh -c "dpkg-parsechangelog -l- -n1 -SVersion"
05/31/2018 16:40:49 - DEBUG:Executing: sh -c "dpkg-parsechangelog -l- -n1 -o1 -SVersion"
05/31/2018 16:40:49 - INFO:Created tag old/ubuntu for version 1:4.2.8p10+dfsg-5ubuntu7
05/31/2018 16:40:49 - DEBUG:Executing: git tag old/debian e96ae83aca626b8483940fbd6ae2e014aec09924 -f
05/31/2018 16:40:49 - INFO:Created tag old/debian for version 1:4.2.8p10+dfsg-5
05/31/2018 16:40:49 - DEBUG:Executing: git tag new/debian pkg/debian/sid -f
05/31/2018 16:40:49 - INFO:Created tag new/debian for version 1:4.2.8p11+dfsg-1
05/31/2018 16:40:49 - DEBUG:Executing: git checkout e96ae83aca626b8483940fbd6ae2e014aec09924
05/31/2018 16:40:49 - DEBUG:Executing: git rev-list --ancestry-path --reverse e96ae83aca626b8483940fbd6ae2e014aec09924..pkg/ubuntu/devel
05/31/2018 16:40:49 - DEBUG:Executing: git cherry-pick --allow-empty --keep-redundant-commits 3d189227e948ad8824a85c29bd436c8fcb96ab04
05/31/2018 16:40:49 - DEBUG:Executing: git cherry-pick --allow-empty --keep-redundant-commits -m 2 1f212b7ebb448ee5b597ace32b18987426ce3ec1
05/31/2018 16:40:49 - DEBUG:Executing: git cherry-pick --allow-empty --keep-redundant-commits b524a1e5daeffc374b162ad67044c96f76960bbb
05/31/2018 16:40:49 - ERROR:Command exited 1: git cherry-pick --allow-empty --keep-redundant-commits b524a1e5daeffc374b162ad67044c96f76960bbb
05/31/2018 16:40:49 - ERROR:stdout:
05/31/2018 16:40:49 - ERROR:stderr: error: could not apply b524a1e5... Import patches-unapplied version 1:4.2.8p10+dfsg-5ubuntu3 to ubuntu/artful-proposed
  hint: after resolving the conflicts, mark the corrected paths
  hint: with 'git add <paths>' or 'git rm <paths>'
  hint: and commit the result with 'git commit'

Traceback (most recent call last):
  File "/snap/git-ubuntu/429/bin/git-ubuntu", line 11, in <module>
    load_entry_point('gitubuntu==0.7.4', 'console_scripts', 'git-ubuntu')()
  File "/snap/git-ubuntu/429/lib/python3.6/site-packages/gitubuntu/__main__.py", line 243, in main
    sys.exit(args.func(args))
  File "/snap/git-ubuntu/429/lib/python3.6/site-packages/gitubuntu/merge.py", line 108, in cli_main
    subcommand=args.subsubcommand,
  File "/snap/git-ubuntu/429/lib/python3.6/site-packages/gitubuntu/merge.py", line 471, in main
    tag_only,
  File "/snap/git-ubuntu/429/lib/python3.6/site-packages/gitubuntu/merge.py", line 177, in do_start
    do_reconstruct(repo, tag_prefix, commitish, merge_base_commit_hash, force)
  File "/snap/git-ubuntu/429/lib/python3.6/site-packages/gitubuntu/merge.py", line 155, in do_reconstruct
    repo.git_run(args)
  File "/snap/git-ubuntu/429/lib/python3.6/site-packages/gitubuntu/git_repository.py", line 1556, in git_run
    **kwargs,
  File "/snap/git-ubuntu/429/lib/python3.6/site-packages/gitubuntu/git_repository.py", line 204, in git_run
    return run(['git'] + list(args), env=env, **kwargs)
  File "/snap/git-ubuntu/429/lib/python3.6/site-packages/gitubuntu/run.py", line 88, in run
    raise e
  File "/snap/git-ubuntu/429/lib/python3.6/site-packages/gitubuntu/run.py", line 67, in run
    stdout=stdout, stderr=stderr, stdin=stdin)
  File "/snap/git-ubuntu/429/usr/lib/python3.6/subprocess.py", line 418, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['git', 'cherry-pick', '--allow-empty', '--keep-redundant-commits', 'b524a1e5daeffc374b162ad67044c96f76960bbb']' returned non-zero exit status 1.

Taking a look 1f212b7ebb448ee5b597ace32b18987426ce3ec1 (which is the actual error point), it is a merge of:

Merge: 3d189227 6a986e2b

The first is the import tag of 1%4.2.8p10+dfsg-5ubuntu1, while the latter is the upload tag for 1%4.2.8p10+dfsg-5ubuntu2. If you dig into 6a986e2b, though, you'll see it does not have a recent import tag in its history, and in fact was based off an ubuntu/devel merge at some point in the past (which does not even happen anymore). So that ends up causing `merge start` to fail, because the ancestry of 6a986e2b is not in the ancestry-list (it does not have old/debian as an ancestor).

[Nish Aravamudan (nacc)]

I'm doing a no-push reimport of ntp off my importer-fix-tests branch. I suspect in that case, the upload tag above will end up not being integrated into the import tags. Will report back once I get results.

[Nish Aravamudan (nacc)]

Confirmed:

05/31/2018 17:34:30 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu2 to ubuntu/artful-proposed
05/31/2018 17:34:40 - WARNING:An upload tag was found for version 1:4.2.8p10+dfsg-5ubuntu2, however it does not have any appropriate Git ancestor (for the prior changelog version). This is currently unsupported and we will not integrate this upload tag.
05/31/2018 17:34:43 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu2 to ubuntu/artful
05/31/2018 17:34:50 - INFO:Importing patches-unapplied 1:4.2.8p4+dfsg-3ubuntu5.7 to ubuntu/xenial-proposed
05/31/2018 17:35:01 - INFO:Importing patches-unapplied 1:4.2.8p4+dfsg-3ubuntu5.7 to ubuntu/xenial-updates
05/31/2018 17:35:09 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu3 to ubuntu/artful-proposed
05/31/2018 17:35:22 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu3 to ubuntu/artful
05/31/2018 17:35:31 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu3 to ubuntu/bionic
05/31/2018 17:35:39 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu4 to ubuntu/bionic-proposed
05/31/2018 17:35:48 - WARNING:An upload tag was found for version 1:4.2.8p10+dfsg-5ubuntu4, however it does not have any appropriate Git ancestor (for the prior changelog version). This is currently unsupported and we will not integrate this upload tag.
05/31/2018 17:35:51 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu4 to ubuntu/bionic
05/31/2018 17:36:00 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu3.1 to ubuntu/artful-proposed
05/31/2018 17:36:09 - WARNING:An upload tag was found for version 1:4.2.8p10+dfsg-5ubuntu3.1, however it does not have any appropriate Git ancestor (for the prior changelog version). This is currently unsupported and we will not integrate this upload tag.
05/31/2018 17:36:12 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu3.1 to ubuntu/artful-updates
05/31/2018 17:36:21 - INFO:Importing patches-unapplied 1:4.2.8p10+dfsg-5ubuntu5 to ubuntu/bionic-proposed
05/31/2018 17:36:30 - WARNING:An upload tag was found for version 1:4.2.8p10+dfsg-5ubuntu5, however it does not have any appropriate Git ancestor (for the prior changelog version). This is currently unsupported and we will not integrate this upload tag.

And import tags do not have upload tags as their ancestors.

[Christian Ehrhardt  (paelzer)]

Can we check that on "git push" via hook and/or "git tag --upload" to warn
that this tag will not be usable?

[Nish Aravamudan (nacc)]

Possibly, `lint` should check on this of course.

On Thu, May 31, 2018, 21:35  Christian Ehrhardt  <
<email address hidden>> wrote:

> Can we check that on "git push" via hook and/or "git tag --upload" to warn
> that this tag will not be usable?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1774539
>
> Title:
> Historically adopted upload tags may not satisfy new rules
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/usd-importer/+bug/1774539/+subscriptions
>
> Launchpad-Notification-Type: bug
> Launchpad-Bug: product=usd-importer; status=New; importance=Undecided;
> assignee=None;
> Launchpad-Bug-Tags: import merge
> Launchpad-Bug-Information-Type: Public
> Launchpad-Bug-Private: no
> Launchpad-Bug-Security-Vulnerability: no
> Launchpad-Bug-Commenters: nacc paelzer
> Launchpad-Bug-Reporter: Nish Aravamudan (nacc)
> Launchpad-Bug-Modifier:  Christian Ehrhardt  (paelzer)
> Launchpad-Message-Rationale: Subscriber
> Launchpad-Message-For: nacc
>