"ValueError: too many values to unpack" when pushing to github

[Brian de Alwis]

I'm trying to push a single change to a repository on github.com:

$ bzr missing ../tycho-github
You have 1 extra revision(s):
------------------------------------------------------------
revno: 815
committer: Brian de Alwis <email address hidden>
branch nick: trunk
timestamp: Sun 2010-08-29 11:08:00 -0400
message:
  Fix for TYCHO-466: "StringIndexOutOfBoundsException when packaging
  unclean MacOS X product". This exception occurs in ProductExportMojo.java
  should there be detritus remaining from a previous build. This
  patch ensures that any detritus is removed and also adds a safety
  check before any string processing is performed.

$ bzr dpush git+ssh://<email address hidden>/briandealwis/sonatype-tycho.git
bzr: ERROR: exceptions.ValueError: too many values to unpack

Traceback (most recent call last):
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/commands.py", line 911, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/commands.py", line 1111, in run_bzr
    ret = run(*run_argv)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/commands.py", line 689, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/commands.py", line 704, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/foreign.py", line 319, in run
    push_result = source_branch.lossy_push(target_branch)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/branch.py", line 1098, in lossy_push
    return lossy_push(stop_revision)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/git/branch.py", line 704, in lossy_push
    update_refs)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/git/push.py", line 302, in dfetch_refs
    self.source_store.generate_lossy_pack_contents)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/git/remote.py", line 296, in send_pack
    return self._transport.send_pack(get_changed_refs, generate_pack_contents)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/git/remote.py", line 138, in send_pack
    generate_pack_contents)
  File "/usr/pkg/lib/python2.6/site-packages/dulwich/client.py", line 152, in send_pack
    new_refs = determine_wants(old_refs)
  File "/usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/git/push.py", line 293, in determine_wants
    for name, (gitid, revid) in self.new_refs.iteritems():
ValueError: too many values to unpack

bzr 2.2.0 on python 2.6.2 (Darwin-10.4.0-i386-32bit)
arguments: ['/usr/pkg/bin/bzr', 'dpush', 'git+ssh://<email address hidden>/briandealwis/sonatype-tycho.git']
encoding: 'UTF-8', fsenc: 'utf-8', lang: 'en_CA.UTF-8'
plugins:
  automv /Users/bsd/.bazaar/plugins/automv [unknown]
  bash_completion /Users/bsd/.bazaar/plugins/bash_completion [1.1.0dev]
  bzrtools /Users/bsd/.bazaar/plugins/bzrtools [2.1b1]
  clear_exec_bit /Users/bsd/.bazaar/plugins/clear_exec_bit.py [unknown]
  colo /Users/bsd/.bazaar/plugins/colo [0.2.0dev]
  diffstat /Users/bsd/.bazaar/plugins/diffstat [0.2.0]
  externals /Users/bsd/.bazaar/plugins/externals [1.1.0]
  extmerge /Users/bsd/.bazaar/plugins/extmerge [unknown]
  git /usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/git [0.5.2]
  grep /Users/bsd/.bazaar/plugins/grep [0.4.0]
  launchpad /usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/launchpad [2.2.0]
  mail_app /Users/bsd/.bazaar/plugins/mail_app [0.9.0]
  netrc_credential_store /usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/netrc_credential_store [2.2.0]
  news_merge /usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/news_merge [2.2.0]
  rewrite /Users/bsd/.bazaar/plugins/rewrite [0.6.1dev]
  sprout /Users/bsd/.bazaar/plugins/sprout [0.9.0dev]
  svn /usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/svn [1.0.3]
  xmloutput /Users/bsd/.bazaar/plugins/xmloutput [0.8.6.dev]

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.

[Brian de Alwis (slyguy)]

Some more information, in case it matters. My local bzr branch was actually branched from a bzr-git branch made against

   git://github.com/sonatype/sonatype-tycho.git

I subsequently forked that repository on github.com and am trying to dpush my changes into that forked repo.

So the order of operations was:

$ bzr branch git://github.com/sonatype/sonatype-tycho.git tycho-github
$ bzr branch tycho-github tycho-bugfix
$ cd tycho-bugfix
// hack hack
// fork sonatype/sonatype-tycho.git on github.com
$ bzr commit
$ bzr dpush git+ssh://<email address hidden>/briandealwis/sonatype-tycho.git

I doubt it matters since I still receive the ValueError if I first pull from git://github.com/briandealwis/sonatype-tycho.git

[Brian de Alwis (slyguy)]

$ BZR_PDB=1 bzr dpush git+ssh://<email address hidden>/briandealwis/sonatype-tycho.git
[error text deleted]
**** entering debugger
> /usr/pkg/lib/python2.6/site-packages/bzrlib/plugins/git/push.py(293)determine_wants()
-> for name, (gitid, revid) in self.new_refs.iteritems():

(Pdb) x = self.new_refs.iteritems()
(Pdb) dict(x)
{'refs/tags/tycho-0.6.0-git': '813e7b61c9f8dff9894940934651a5591415a512', 'refs/tags/0.1.0': '890115d1b381ede1daf158c384c9f30460ae34ed', 'refs/heads/tycho-0.4.0': '5bac3ddb8934993df15113154123250183bb63d2', 'refs/heads/p2-hosted-repo': '323560f880291208f4d4935ed0f19e177bb24af3', 'refs/heads/tycho-0.7.x': '1dd8c0efd62fe23b0039a612a25ea317aedcad7c', 'refs/heads/TYCHO-478_tycho-p2-extraction': 'f3e68faf67ac7a429a2220a407679a52770005e6', 'refs/heads/maven-3.0-beta-3': 'a7813c97544e318dc807ac183c3cf2af323aea3c', 'refs/tags/tycho-0.6.0': '2f05e036e6165a65d76b2c70c480f325ba7255a7', 'refs/heads/tycho-0.3.0': 'a6bbddb6590582419157a9a56e789852f7c42772', 'refs/tags/tycho-0.8.0': '8f270570c3b0c58ecd461b953ce9376275366394', 'refs/tags/tycho-0.5.0': 'd9b725c77c0e8dfb534ea94a744fac260a8bc7f7', 'refs/heads/tycho-0.9.x': '2d63c8196c110075632882abc1453a11f4685591', 'refs/tags/0.2.0': 'd471370d5794aa68cc3ef6331646a7e415dedf46', 'refs/heads/tycho-0.6.0': '813e7b61c9f8dff9894940934651a5591415a512', 'refs/heads/tycho-0.8.x': 'db048961376765d3b72057c1fcd852cceefd1148', 'refs/heads/master': (None, 'git-v1:facc0487f6780c4a92b374866b38535df1fc0202'), 'refs/tags/tycho-0.7.0': '1dd8c0efd62fe23b0039a612a25ea317aedcad7c'}

[Brian de Alwis (slyguy)]

Looks like the iteriems() can return either git SHA1 ids or a tuple with the bzr id. I've attached a simple patch that gets this to work; my python-fu is underdeveloped, so I'm sure there's likely a better way. But with this, my dpush works.

[Brian de Alwis (slyguy)]

Oops, I left in the stray pdb.set_trace() in that patch. Sorry!

[n[ate]vw (natevw)]

I'm getting a similar error in a different place using latest trunk:

dvorak:ShutterStem.git nathan$ bzr dpush --remember git+ssh://<email address hidden>/natevw/ShutterStem.git
bzr: ERROR: exceptions.ValueError: too many values to unpack

Traceback (most recent call last):
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 911, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 1111, in run_bzr
    ret = run(*run_argv)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 689, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 704, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/foreign.py", line 319, in run
    push_result = source_branch.lossy_push(target_branch)
  File "/Library/Python/2.6/site-packages/bzrlib/branch.py", line 1098, in lossy_push
    return lossy_push(stop_revision)
  File "/Users/nathan/.bazaar/plugins/git/branch.py", line 721, in lossy_push
    update_refs)
  File "/Users/nathan/.bazaar/plugins/git/push.py", line 325, in dfetch_refs
    self.source_store.generate_lossy_pack_contents)
  File "/Users/nathan/.bazaar/plugins/git/remote.py", line 296, in send_pack
    return self._transport.send_pack(get_changed_refs, generate_pack_contents)
  File "/Users/nathan/.bazaar/plugins/git/remote.py", line 138, in send_pack
    generate_pack_contents)
  File "/Library/Python/2.6/site-packages/dulwich-0.6.1-py2.6-macosx-10.6-universal.egg/dulwich/client.py", line 176, in send_pack
    objects = generate_pack_contents(have, want)
  File "/Users/nathan/.bazaar/plugins/git/object_store.py", line 612, in generate_lossy_pack_contents
    lossy=True)
  File "/Users/nathan/.bazaar/plugins/git/object_store.py", line 625, in generate_pack_contents
    (type, (revid, tree_sha)) = ret[commit_sha]
ValueError: too many values to unpack

bzr 2.2.0 on python 2.6.1 (Darwin-10.4.0-i386-64bit)
arguments: ['/usr/local/bin/bzr', 'dpush', '--remember', 'git+ssh://<email address hidden>/natevw/ShutterStem.git']
encoding: 'UTF-8', fsenc: 'utf-8', lang: 'en_US.UTF-8'
plugins:
  bash_completion /Library/Python/2.6/site-packages/bzrlib/plugins/bash_completion [2.2.0]
  git /Users/nathan/.bazaar/plugins/git [0.5.3dev]
  launchpad /Library/Python/2.6/site-packages/bzrlib/plugins/launchpad [2.2.0]
  netrc_credential_store /Library/Python/2.6/site-packages/bzrlib/plugins/netrc_credential_store [2.2.0]
  news_merge /Library/Python/2.6/site-packages/bzrlib/plugins/news_merge [2.2.0]

*** Bazaar has encountered an internal error. Thi...

[n[ate]vw (natevw)]

Regression notes: bzr-git release 0.5.2 can successfully dpush (ignoring issue #640873)

[n[ate]vw (natevw)]

Actually, I take that back, just ran into this issue with bzr-git 0.5.2 too, dulwich 0.6.1:

$ bzr dpush git+ssh://<email address hidden>/calftrail/ShutterStem-bzrtest.git
bzr: ERROR: exceptions.ValueError: too many values to unpack

Traceback (most recent call last):
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 911, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 1111, in run_bzr
    ret = run(*run_argv)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 689, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/Library/Python/2.6/site-packages/bzrlib/commands.py", line 704, in run
    return self._operation.run_simple(*args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/cleanup.py", line 135, in run_simple
    self.cleanups, self.func, *args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/cleanup.py", line 165, in _do_with_cleanups
    result = func(*args, **kwargs)
  File "/Library/Python/2.6/site-packages/bzrlib/foreign.py", line 319, in run
    push_result = source_branch.lossy_push(target_branch)
  File "/Library/Python/2.6/site-packages/bzrlib/branch.py", line 1098, in lossy_push
    return lossy_push(stop_revision)
  File "/Users/nathan/.bazaar/plugins/git/branch.py", line 704, in lossy_push
    update_refs)
  File "/Users/nathan/.bazaar/plugins/git/push.py", line 302, in dfetch_refs
    self.source_store.generate_lossy_pack_contents)
  File "/Users/nathan/.bazaar/plugins/git/remote.py", line 296, in send_pack
    return self._transport.send_pack(get_changed_refs, generate_pack_contents)
  File "/Users/nathan/.bazaar/plugins/git/remote.py", line 138, in send_pack
    generate_pack_contents)
  File "/Library/Python/2.6/site-packages/dulwich-0.6.1-py2.6-macosx-10.6-universal.egg/dulwich/client.py", line 152, in send_pack
    new_refs = determine_wants(old_refs)
  File "/Users/nathan/.bazaar/plugins/git/push.py", line 293, in determine_wants
    for name, (gitid, revid) in self.new_refs.iteritems():
ValueError: too many values to unpack

bzr 2.2.0 on python 2.6.1 (Darwin-10.4.0-i386-64bit)
arguments: ['/usr/local/bin/bzr', 'dpush', 'git+ssh://<email address hidden>/calftrail/ShutterStem-bzrtest.git']
encoding: 'UTF-8', fsenc: 'utf-8', lang: 'en_US.UTF-8'
plugins:
  bash_completion /Library/Python/2.6/site-packages/bzrlib/plugins/bash_completion [2.2.0]
  git /Users/nathan/.bazaar/plugins/git [0.5.2]
  launchpad /Library/Python/2.6/site-packages/bzrlib/plugins/launchpad [2.2.0]
  netrc_credential_store /Library/Python/2.6/site-packages/bzrlib/plugins/netrc_credential_store [2.2.0]
  news_merge /Library/Python/2.6/site-packages/bzrlib/plugins/news_merge [2.2.0]

*** Bazaar has encountered an internal error. This probably indicates a
    bug in Bazaar. You can help us fix it by filing a bug report at
        https://bugs.launchpad.net/bzr/+filebug
    including this traceback and a description of the problem.

[Jelmer Vernooij (jelmer)]

On Thu, 2010-09-16 at 22:12 +0000, n[ate]vw wrote:
> I'm getting a similar error in a different place using latest trunk:
>
> dvorak:ShutterStem.git nathan$ bzr dpush --remember git+ssh://<email address hidden>/natevw/ShutterStem.git
> bzr: ERROR: exceptions.ValueError: too many values to unpack
Are you sure you're running trunk's tip?

Jelmer

[Sabin Iacob (iacobs)]

yep, still happens on tip (lp:bzr-git), attaching crash file

[Sabin Iacob (iacobs)]

ok, made an attempt at fixing this, and the error also pops up in object_store.py/generate_pack_contents, because cache.lookup_git_sha returns (type, (revid, tree sha, verifiers)), where verifiers contains something called testament3-sha1

After I take care of the ValueError exceptions I end up with a bzr(?) error saying something along the lines of:

error: unpack should have generated 0cc600f846009a049c9421772b8253496931e2b9, but I can't find it!
BzrError: refs/heads/master failed to update

... which seems to be a git protocol error, and it also means I'm lost :-/

[Jelmer Vernooij (jelmer)]

On Thu, 2010-09-23 at 19:40 +0000, Sabin Iacob wrote:
> ok, made an attempt at fixing this, and the error also pops up in
> object_store.py/generate_pack_contents, because cache.lookup_git_sha
> returns (type, (revid, tree sha, verifiers)), where verifiers contains
> something called testament3-sha1
I don't see how generate_pack_contents() could be problematic, it
doesn't call cache.lookup_git_sha.

Cheers,

Jelmer

[Sabin Iacob (iacobs)]

self._cache.idmap.lookup_git_sha in lookup_git_shas, more precisely, and all implementations return 3-item tuples as the 2nd item instead of 2-item tuples as the code in generate_pack_contents expects to find

[Jelmer Vernooij (jelmer)]

This appears to already have been fixed - if you can still reproduce this, please reopen.

[Jesse (sbjesse)]

I have exactly this exception thrown during dpush to github! using TIP revision of both bzr-git and dulwich. switching to bzr.dev doesn't help either.

[Sabin Iacob (iacobs)]

I got it too right after I got news about the bug being fixed, but I
refrained from commenting until now because I didn't have the time to
test properly using a new source tree