option to do less strict fastimport protocol checking

[Eli Zaretskii]

I tried to use the latest bzr-fastimport and python-fastimport to import from git. "bzr fast-import" crashed with the traceback shown below.

Since it seemed to complain about tags, and fast-export-from-git produced warnings about signed tags, I also tried to invoke "git fast-export" directly, passing it the "--signed-tags=strip" option, but the "bzr fast-import" still crashed.

This is bzr 2.1.1 on GNU/Linux.

Here's the traceback:

bzr: ERROR: fastimport.errors.BadFormat: line 1178854: Bad format for section tagger in command tag: found 'Greg McGary'

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 853, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1055, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 661, in run_argv_aliases
    return self.run_direct(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 665, in run_direct
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 122, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/cleanup.py", line 156, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/home/e/eliz/.bazaar/plugins/fastimport/cmds.py", line 299, in run
    info = self._generate_info(source)
  File "/home/e/eliz/.bazaar/plugins/fastimport/cmds.py", line 327, in _generate_info
    return_code = proc.process(p.iter_commands)
  File "/home/e/eliz/lib/python2.6/site-packages/fastimport/processor.py", line 65, in process
    self._process(command_iter)
  File "/home/e/eliz/lib/python2.6/site-packages/fastimport/processor.py", line 69, in _process
    for cmd in command_iter():
  File "/home/e/eliz/lib/python2.6/site-packages/fastimport/parser.py", line 300, in iter_commands
    yield self._parse_tag(line[len('tag '):])
  File "/home/e/eliz/lib/python2.6/site-packages/fastimport/parser.py", line 419, in _parse_tag
    tagger = self._get_user_info('tag', 'tagger', accept_just_who=True)
  File "/home/e/eliz/lib/python2.6/site-packages/fastimport/parser.py", line 473, in _get_user_info
    accept_just_who=accept_just_who)
  File "/home/e/eliz/lib/python2.6/site-packages/fastimport/parser.py", line 529, in _who_when
    self.abort(errors.BadFormat, cmd, section, s)
  File "/home/e/eliz/lib/python2.6/site-packages/fastimport/parser.py", line 187, in abort
    raise exception(self.lineno, *args)
BadFormat: line 1178854: Bad format for section tagger in command tag: found 'Greg McGary'

bzr 2.1.1 on python 2.6.5 (Linux-2.6.32-313-ec2-x86_64-with-Ubuntu-10.04-lucid)
arguments: ['/usr/bin/bzr', 'fast-import', 'idutils.fi', 'idutils']
encoding: 'ANSI_X3.4-1968', fsenc: 'ANSI_X3.4-1968', lang: None
plugins:
  bisect /home/e/eliz/.bazaar/plugins/bisect [1.1.0.pre.0]
  bzrtools /home/e/eliz/.bazaar/plugins/bzrtools [2.0.1]
  diffstat /home/e/eliz/.bazaar/plugins/diffstat [0.2.0]
  etckeeper /usr/lib/python2.6/dist-packages/bzrlib/plugins/etckeeper [unknown]
  fastimport /home/e/eliz/.bazaar/plugins/fastimport [0.10.0dev]
  launchpad /usr/lib/python2.6/dist-packages/bzrlib/plugins/launchpad [2.1.1]
  netrc_credential_store /usr/lib/python2.6/dist-packages/bzrlib/plugins/netrc_credential_store [2.1.1]
  news_merge /usr/lib/python2.6/dist-packages/bzrlib/plugins/news_merge [2.1.1]
  rewrite /home/e/eliz/.bazaar/plugins/rewrite [0.6.2dev]
  svn /usr/lib/python2.6/dist-packages/bzrlib/plugins/svn [1.0.2]
  upload /home/e/eliz/.bazaar/plugins/upload [1.0.0dev]

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.
Command exited with non-zero status 4

[Jelmer Vernooij (jelmer)]

On Mon, 2011-03-07 at 13:04 +0000, Eli Zaretskii wrote:
> Public bug reported:
>
> I tried to use the latest bzr-fastimport and python-fastimport to import
> from git. "bzr fast-import" crashed with the traceback shown below.
>
> Since it seemed to complain about tags, and fast-export-from-git
> produced warnings about signed tags, I also tried to invoke "git fast-
> export" directly, passing it the "--signed-tags=strip" option, but the
> "bzr fast-import" still crashed.
>
> This is bzr 2.1.1 on GNU/Linux.
>
> Here's the traceback:
>
> bzr: ERROR: fastimport.errors.BadFormat: line 1178854: Bad format for
> section tagger in command tag: found 'Greg McGary'
Can you paste the contents of that particular line in the fastimport
file?

Cheers,

Jelmer

[Eli Zaretskii (eliz)]

Here it is:

tagger Greg McGary

There are more lines that look exactly like this one, after it.

The previous "tagger" lines include some kind of time stamp, like this:

tagger Jim Meyering <email address hidden> 1202571650 +0100

[Jelmer Vernooij (jelmer)]

That's odd, as far as I know it's invalid for a git identity to not have an email address. Does git fast import import this stream ok?

[Eli Zaretskii (eliz)]

No, it doesn't:

fatal: Missing > in ident string: Greg McGary

So what do I do? manually fix those tags to include time stamps?

This git repository was imported from CVS at some point in the past, so perhaps these problems come from some bug in the conversion scripts?

[Jelmer Vernooij (jelmer)]

Yes, this would probably be a bug in the conversion scripts.

It's not really the timestamp that is the issue I think but the fact that there is no email address. Try setting it to "Greg McGary <>" instead, that might work better.

I'm inclined to mark this bug Invalid/Won't Fix for python-fastimport as this is invalid and doesn't match git fast-import's behaviour. Alternatively, I guess we could interpret absence of an email address as an empty email address and warn.

[Eli Zaretskii (eliz)]

Yes, adding "<>" to each such line solved the problem. Thanks.

I understand your reluctance to "fix" something that is not supposed to work. How about an optional switch to interpret this as an empty email, issue a warning, but let the import proceed? As long as it doesn't do this by default, it could be an advantage, what with all the imports between different VCSs that go on these days, each import risking to introduce yet another subtle bug.

[Eli Zaretskii (eliz)]

Alternatively, perhaps the --user-map option could be reused for repairing this: the special entry @ could be used to give an email address to such problematic tagger lines.

[Jelmer Vernooij (jelmer)]

Perhaps an option to allow less strict checking in general is an option. Such a thing would require changes in both bzr-fastimport and python-fastimport though.