UDD branch freshness checker breaks on incomplete history

[Jelmer Vernooij]

Several of the UDD imports fail like this:

Traceback (most recent call last):
  File "/srv/package-import.canonical.com/new/scripts/bin/import-package", line 7, in <module>
    run_script()
  File "/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py", line 1149, in run_script
    only_before=options.only_before))
  File "/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py", line 1030, in main
    revid_db, bstore, possible_transports=possible_transports)
  File "/srv/package-import.canonical.com/new/scripts/udd/scripts/import_package.py", line 663, in find_unimported_versions
    possible_transports=possible_transports)
  File "/srv/package-import.canonical.com/new/scripts/udd/icommon.py", line 1213, in get_branch
    possible_transports=possible_transports, readonly=readonly)
  File "/srv/package-import.canonical.com/new/scripts/udd/icommon.py", line 1221, in get_branch_parts
    possible_transports=possible_transports)
  File "/srv/package-import.canonical.com/new/bzr/bzrlib/branch.py", line 178, in open
    return control.open_branch(unsupported=_unsupported)
  File "/srv/package-import.canonical.com/new/bzr/bzrlib/remote.py", line 597, in open_branch
    setup_stacking=not ignore_fallbacks, name=name)
  File "/srv/package-import.canonical.com/new/bzr/bzrlib/remote.py", line 2559, in __init__
    hook(self)
  File "/srv/package-import.canonical.com/new/bzr/bzrlib/plugins/launchpad/__init__.py", line 519, in _check_is_up_to_date
    lp_api_lite.report_freshness(the_branch, verbosity, latest_pub)
  File "/srv/package-import.canonical.com/new/bzr/bzrlib/plugins/launchpad/lp_api_lite.py", line 283, in report_freshness
    latest_ver, branch_ver = _get_newest_versions(the_branch, latest_pub)
  File "/srv/package-import.canonical.com/new/bzr/bzrlib/plugins/launchpad/lp_api_lite.py", line 231, in _get_newest_versions
    best_tag = get_most_recent_tag(tags, the_branch)
  File "/srv/package-import.canonical.com/new/bzr/bzrlib/plugins/launchpad/lp_api_lite.py", line 201, in get_most_recent_tag
    for rev_id in graph.iter_lefthand_ancestry(last_rev, stop_revisions):
  File "/srv/package-import.canonical.com/new/bzr/bzrlib/graph.py", line 1034, in iter_lefthand_ancestry
    parents = get_parents(next_key)
  File "/srv/package-import.canonical.com/new/bzr/bzrlib/graph.py", line 1030, in get_parents
    raise errors.RevisionNotPresent(next_key, self)
bzrlib.errors.RevisionNotPresent: Revision {<email address hidden>} not present in "Graph(StackedParentsProvider(bzrlib.repository._LazyListJoin(([CachingParentsProvider(None)], []))))".

We should either disable this check in the UDD importer or make it non-fatal.

[John A Meinel (jameinel)]

On 11/10/2011 5:23 PM, Jelmer Vernooij wrote:
> Public bug reported:
>
> Several of the UDD imports fail like this:

...

> File "/srv/package-import.canonical.com/new/bzr/bzrlib/graph.py", line 1034, in iter_lefthand_ancestry
> parents = get_parents(next_key)
> File "/srv/package-import.canonical.com/new/bzr/bzrlib/graph.py", line 1030, in get_parents
> raise errors.RevisionNotPresent(next_key, self)
> bzrlib.errors.RevisionNotPresent: Revision {<email address hidden>} not present in "Graph(StackedParentsProvider(bzrlib.repository._LazyListJoin(([CachingParentsProvider(None)], []))))".
>
> We should either disable this check in the UDD importer or make it non-
> fatal.

This certainly sounds like bad data in the branches if it is failing
like this.

John
=:->

[Jelmer Vernooij (jelmer)]

Am 13/11/11 22:39, schrieb John A Meinel:
> On 11/10/2011 5:23 PM, Jelmer Vernooij wrote:
>> Public bug reported:
>>
>> Several of the UDD imports fail like this:
>
> ...
>
>> File "/srv/package-import.canonical.com/new/bzr/bzrlib/graph.py", line
1034, in iter_lefthand_ancestry
>> parents = get_parents(next_key)
>> File "/srv/package-import.canonical.com/new/bzr/bzrlib/graph.py", line
1030, in get_parents
>> raise errors.RevisionNotPresent(next_key, self)
>> bzrlib.errors.RevisionNotPresent: Revision
{<email address hidden>} not present in
"Graph(StackedParentsProvider(bzrlib.repository._LazyListJoin(([CachingParentsProvider(None)],
[]))))".
>>
>> We should either disable this check in the UDD importer or make it non-
>> fatal.
>
> This certainly sounds like bad data in the branches if it is failing
> like this.
>
That's probably true too, but in the case of the importer it seems like
commenting out the freshness-checker works around the issue.

Cheers,

Jelmer

[Vincent Ladeuil (vila)]

I went ahead and set:
   launchpad.packaging_verbosity = off
in jubany's bazaar.conf

We probably want a finer grained setup *and* investigate whether some branches have invalid data or if the freshness is bogus (as in not setting up the branch correctly or some other internal caching issue or whatever).

In the mean time, with this option off, several imports were unblocked leading to different errors (including errors different than the previous import) and even succeeded in some cases (which means the check freshness is doing something weird anyway).

[John A Meinel (jameinel)]

You can do this with: "bzr -Olaunchpad.packaging_verbosity=off" from the commandline for people running into this manually.

Offhand I would guess this happens when you have a stacked branch that doesn't have any revisions in the stacked location, when we try to find any tags that are recent.

[John A Meinel (jameinel)]

For some versions of bzr, you'll have to put the setting into a configuration file, new versions of bzr already fully support the command line option.

[Stuart Bishop (stub)]

$ bzr branch lp:ubuntu/postgresql-9.1
bzr: ERROR: Revision {<email address hidden>} not present in "Graph(StackedParentsProvider(bzrlib.repository._LazyListJoin(([CachingParentsProvider(None)], []))))".

[Buyukkayhan (buyukkayhan-a)]

# bzr branch lp:ubuntu/saucy/firefox
bzr: ERROR: Revision {<email address hidden>} not present in "Graph(StackedParentsProvider(bzrlib.repository._LazyListJoin(([CachingParentsProvider(None)], []))))".

[Anthony Wong (anthonywong)]

Met this problem when checking out lp:ubuntu/trusty/unity-lens-files. The workaround in #4 helps:

bzr -Olaunchpad.packaging_verbosity=off branch lp:ubuntu/trusty/unity-lens-files

[TienFu Chen (ctf)]

also happen on checking out lp:ubuntu/trusty/unity, and option Olaunchpad.packaging_verbosity=off can fix it.

[autra (autra)]

Same fetching lp:ubuntu/utopic/network-manager
Workaround also worked.

[Vardan (vardan-pogosyan)]

Same problem. Workaround also work.

[Ivy Alexander (ivyalexander)]

Same issue when fetching lp:ubuntu/trusty/cups - adding -0launchpad.packaging_verbosity=off worked around it successfully.