Importing revisions with submodules is not supported

Bug #402814 reported by Martin Pool
432
This bug affects 78 people
Affects Status Importance Assigned to Milestone
Bazaar
Triaged
High
Unassigned
Bazaar Git Plugin
Fix Released
Medium
Unassigned
Breezy
Fix Released
Medium
Jelmer Vernooij
Launchpad code imports
Triaged
High
Unassigned
cloudfoundry
Invalid
High
Unassigned

Bug Description

bzr-git converts submodules into nested trees, but nested trees aren't very well supported by bzr yet.

This affects importing of commits from Git repositories that contains submodules anywhere in their history. It doesn't matter if the submodules have been removed in the last revision.

Related branches

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

This is blocked by by-reference nested tree support in bzrlib.

Changed in bzr-git:
status: New → Triaged
importance: Undecided → Wishlist
importance: Wishlist → Medium
Revision history for this message
Jeff Fortin Tam (kiddo) wrote :

I think I'm affected by the same problem. When trying to branch pitivi:

jeff@testbed:~$ bzr branch git://git.pitivi.org/git/pitivi.git
Escaped 1 XML-invalid characters in ce745af077aeda4f8d22cae84d5b1f0baf8eb1b7. Will be unable to regenerate the SHA map.
bzr: ERROR: exceptions.NotImplementedError: <function import_git_submodule at 0xa803f7c>

Traceback (most recent call last):
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 835, in exception_to_return_code
    return the_callable(*args, **kwargs)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 1030, in run_bzr
    ret = run(*run_argv)
  File "/usr/lib/python2.6/dist-packages/bzrlib/commands.py", line 647, in run_argv_aliases
    return self.run(**all_cmd_args)
  File "/usr/lib/python2.6/dist-packages/bzrlib/builtins.py", line 1245, in run
    source_branch=br_from)
  File "/usr/lib/python2.6/dist-packages/bzrlib/bzrdir.py", line 1177, in sprout
    result_repo.fetch(source_repository, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/repository.py", line 1544, in fetch
    find_ghosts=find_ghosts, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 378, in fetch
    mapping=mapping, fetch_spec=fetch_spec)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 403, in fetch_refs
    self.fetch_objects(determine_wants, mapping, pb)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 437, in fetch_objects
    store, recorded_wants, pb)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 335, in import_git_objects
    parent_invs, target_git_object_retriever._idmap, lookup_object)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 246, in import_git_tree
    file_id, revision_id, parent_invs, shagitmap, lookup_object)
  File "/usr/lib/python2.6/dist-packages/bzrlib/plugins/git/fetch.py", line 174, in import_git_submodule
    raise NotImplementedError(import_git_submodule)
NotImplementedError: <function import_git_submodule at 0xa803f7c>

bzr 1.17 on python 2.6.2 (linux2)
arguments: ['/usr/bin/bzr', 'branch', 'git://git.pitivi.org/git/pitivi.git', 'pitivi-bazaar']
encoding: 'UTF-8', fsenc: 'UTF-8', lang: 'fr_CA.UTF-8'
plugins:
  bzrtools /usr/lib/python2.6/dist-packages/bzrlib/plugins/bzrtools [1.17]
  dbus /usr/lib/python2.6/dist-packages/bzrlib/plugins/dbus [0.1dev]
  git /usr/lib/python2.6/dist-packages/bzrlib/plugins/git [0.4]
  gtk /usr/lib/python2.6/dist-packages/bzrlib/plugins/gtk [0.96.2.final]
  launchpad /usr/lib/python2.6/dist-packages/bzrlib/plugins/launchpad [1.17]
  netrc_credential_store /usr/lib/python2.6/dist-packages/bzrlib/plugins/netrc_credential_store [1.17]
  svn /usr/lib/python2.6/dist-packages/bzrlib/plugins/svn [0.6.3]

This is probably what is affecting https://code.launchpad.net/~vcs-imports/pitivi/trunk-git too, I presume.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

A proper error message is now printed.

Launchpad could automatically recognize and perhaps better format this error by explicitly catching objects of the class bzrlib.plugins.git.fetch.SubmodulesNotSupported

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

This is blocked by nested tree support in Bazaar. I've marked it as affecting Bazaar, but I'm not sure if that's the right thing to do. I couldn't find any existing bug reports in Bazaar about missing nested tree support.

Changed in bzr:
status: New → Triaged
status: Triaged → Confirmed
importance: Undecided → Wishlist
Jelmer Vernooij (jelmer)
summary: - NotImplementedError importing a branch with submodules
+ Importing revisions with submodules does not work
summary: - Importing revisions with submodules does not work
+ Importing revisions with submodules is not supported
description: updated
Jelmer Vernooij (jelmer)
Changed in bzr-git:
status: Triaged → Fix Released
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Of course, this doesn't mean you can use submodules now - bzr-git can import them and converts them to nested trees, but nested tree support in bzr itself is still incomplete.

Revision history for this message
Benjamin Drung (bdrung) wrote :

The git import of xmms2 [1-2] failed due to missing submodules support:

bzrlib.plugins.git.fetch.SubmodulesNotSupported: Submodules can not yet be imported (requires nested tree support in Bazaar).

[1] https://code.launchpad.net/~vcs-imports/xmms2/stable
[2] https://code.launchpad.net/~vcs-imports/xmms2/trunk

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

A version of bzr-git that supports submodules should land on launchpad within the next two weeks.

Revision history for this message
L.Lopez (ldotlopez) wrote :

Launchpad import still fails:

2010-01-09 12:30:49 INFO 51 bytes transferred |
2010-01-09 12:30:50 INFO Counting objects: 5920, done. 0
2010-01-09 12:30:52 INFO finding revisions to fetch:generating index 0/5920
2010-01-09 12:30:53 INFO finding revisions to fetch 1/1808
2010-01-09 12:31:18 INFO
Traceback (most recent call last):
  File "/srv/importd.launchpad.net/production/launchpad-rev-8806/scripts/code-import-worker.py", line 63, in <module>
    script.main()
  File "/srv/importd.launchpad.net/production/launchpad-rev-8806/scripts/code-import-worker.py", line 58, in main
    import_worker.run()
  File "/srv/importd.launchpad.net/production/launchpad-rev-8806/lib/lp/codehosting/codeimport/worker.py", line 409, in run
    self._doImport()
  File "/srv/importd.launchpad.net/production/launchpad-rev-8806/lib/lp/codehosting/codeimport/worker.py", line 526, in _doImport
    bazaar_tree.branch.pull(foreign_branch, overwrite=True)
  File "<string>", line 4, in pull_write_locked
  File "/home/pqm/for_rollouts/production/eggs/bzr-2.1.0b3-py2.5-linux-x86_64.egg/bzrlib/branch.py", line 949, in pull
  File "/srv/importd.launchpad.net/production/launchpad-rev-8806/optionalbzrplugins/git/branch.py", line 416, in pull
    graph=graph)
  File "/srv/importd.launchpad.net/production/launchpad-rev-8806/optionalbzrplugins/git/branch.py", line 379, in update_revisions
    interrepo.fetch_objects(determine_wants, self.source.mapping)
  File "/srv/importd.launchpad.net/production/launchpad-rev-8806/optionalbzrplugins/git/fetch.py", line 493, in fetch_objects
    store, recorded_wants, pb)
  File "/srv/importd.launchpad.net/production/launchpad-rev-8806/optionalbzrplugins/git/fetch.py", line 363, in import_git_objects
    allow_submodules=getattr(repo._format, "supports_tree_reference", False))
  File "/srv/importd.launchpad.net/production/launchpad-rev-8806/optionalbzrplugins/git/fetch.py", line 261, in import_git_tree
    raise SubmodulesRequireSubtrees()
bzrlib.plugins.git.fetch.SubmodulesRequireSubtrees: The repository you are fetching from contains submodules. To continue, upgrade your Bazaar repository to a format that supports nested trees, such as 'development-subtree'.
Import failed:
Traceback (most recent call last):
Failure: twisted.internet.error.ProcessTerminated: A process has ended with a probable error condition: process ended with exit code 1.

Revision history for this message
Tim Su (tim-todoroo) wrote :

Launchpad fails for me too, same error. I no longer have submodules in the repository, but there was a time in the past when there were submodules. The branch that's failing is "lp:astrid"

Aaron Bentley (abentley)
tags: added: udd
Tim Penhey (thumper)
Changed in launchpad-code:
status: New → Triaged
importance: Undecided → Wishlist
Revision history for this message
Michael Hope (michaelh1) wrote : Re: [Bug 402814] Re: Importing revisions with submodules is not supported

This is fairly important to me - more important than 'Wishlist'. We
can't import GDB or QEMU into Launchpad without this support.

Martin Pool (mbp)
Changed in bzr:
importance: Wishlist → High
Benjamin Drung (bdrung)
description: updated
Revision history for this message
Sebastian Dröge (slomo) wrote :

Also affects pitivi and gstreamer.

Revision history for this message
Jorge Castro (jorge) wrote :

Any update on this? This is really holding us back enabling daily builds.

Revision history for this message
Jorge Castro (jorge) wrote :

Sorry, I should have been more clear, this is holding back our ability to evangelize daily builds to existing upstreams who would be interested in using dailies and PPAs.

Benjamin Drung (bdrung)
description: updated
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

For a more complete list of failing imports because of this issue, see https://dev.launchpad.net/FailingBzrSvnImports

Revision history for this message
Jelmer Vernooij (jelmer) wrote :
Benjamin Drung (bdrung)
description: updated
Revision history for this message
Sebastian Dröge (slomo) wrote :

And also affects banshee

Revision history for this message
Eli Zaretskii (eliz) wrote :

Are there any concrete plans to add this to bzr stable in some specific future version? If not, how about making this higher priority, as it seems to render a valuable bzr feature only half workable.

Revision history for this message
Riccardo 'c10ud' (c10ud) wrote :

affects emesene

Revision history for this message
Vsevolod Velichko (torkvemada) wrote :

Affects qutim

Revision history for this message
Stefan H. (stefan-h) wrote :

Has been affecting Mumble for a long time now. Even being able to import without the submodules would be better than being completely unable to do so.

Any ETA on releasing a fix for this bug?

Revision history for this message
Paulo Dias (paulo-miguel-dias) wrote :

Affects tomahawk too

Revision history for this message
Jaromil (jaromil) wrote :

affects hasciicam and more software

Changed in launchpad:
importance: Wishlist → Low
Revision history for this message
xaav (xaav) wrote : Re: Bazaar needs support for nested trees

I updated the title because it was misleading.

summary: - Importing revisions with submodules is not supported
+ Bazaar needs support for nested trees
Revision history for this message
Martin Pool (mbp) wrote :

@Geoff this bug is for the specific git issue.

summary: - Bazaar needs support for nested trees
+ Importing revisions with submodules is not supported
Revision history for this message
Vsevolod Velichko (torkvemada) wrote :

Added affected software to description :)

description: updated
Revision history for this message
xaav (xaav) wrote :

Bug #769629 is not a duplicate then. Can someone please un-mark it as a duplicate.

Revision history for this message
karen pulsifer (froggy1234) wrote :

how do you fix this problem E:: command not found
root@kpulaifer-desktop:~# Go to the repository dialog to correct the problem.
Go: command not found
root@kpulaifer-desktop:~# E: _cache->open() failed, please repo
bash: syntax error near unexpected token `('
root@kpulaifer-desktop:~# /etc/apt/sources.list.d/ferramroberto
bash: /etc/apt/sources.list.d/ferramroberto: No such file or directory
root@kpulaifer-desktop:~# sudo remove /etc/apt/sources.list.d/ferramroberto
sudo: remove: command not found
root@kpulaifer-desktop:~# /etc/apt/sources.list.d/ferramroberto
bash: /etc/apt/sources.list.d/ferramroberto: No such file or directory
root@kpulaifer-desktop:~# sudo/etc/apt/sources.list.d/ferramroberto
bash: sudo/etc/apt/sources.list.d/ferramroberto: No such file or directory
root@kpulaifer-desktop:~# /etc/apt/sources.list.d/ferramroberto-
bash: /etc/apt/sources.list.d/ferramroberto-: No such file or directory
root@kpulaifer-desktop:~# sudo 'b-src
> sudo deb-src
> sudo E:Type 'b-src' is not known on line 1 in source list /etc/apt/sources.list.d/ferramroberto-java-natty.list

Revision history for this message
Martin Pool (mbp) wrote : Re: [Bug 402814] Re: Importing revisions with submodules is not supported

Hi Karen,

That's a different problem; please go to
<http://askubuntu.com/questions/ask> and describe what it is you want
to accomplish, what you're doing, and what goes wrong, and someone
will help you soon.

Martin

Revision history for this message
Dustin Kirkland  (kirkland) wrote :

This bug is impacting our work with CloudFoundry. Any likelihood of it being worked-on or solved in the near future?

Thanks,
Dustin

Changed in cloudfoundry:
importance: Undecided → High
status: New → Confirmed
Revision history for this message
snd (dns) wrote :
Benjamin Drung (bdrung)
description: updated
Jelmer Vernooij (jelmer)
description: updated
Revision history for this message
chrysn (chrysn) wrote :

as a workaround, i used `git filter-branch` to remove the submodules from the project -- of course, this only works if the submodules are purely optional or otherwise not relevant to the purposes intended (in my case, i need the branch to build daily packages; upstreams might keep library dependencies in a subtree to build and ship them, while in debian/ubuntu, this is patched out anyway to use package dependencies).

the very line i used for filtering was

  git filter-branch -f --index-filter 'git rm --cached --ignore-unmatch .gitmodules libraries/MCAD'

(libraries/MCAD is the only submodule i had). for reference, this affects the lp:openscad project.

Revision history for this message
Benjamin Drung (bdrung) wrote :

chrysn, this workaround doesn't help, because a machine running this command is needed.

Revision history for this message
snd (dns) wrote :

These projects/branches are also affected by this bug:

lp:anarchnet
lp:anubis
lp:autoconf-archive
lp:bison
lp:cflow
lp:coreutils
lp:cpio (lp:~rbose-vcs-imports/cpio/master)
lp:findutils
lp:gnu-radius
lp:grep (lp:~rbose-vcs-imports/grep/master)
lp:mumble
lp:patch
lp:sed

The GDB project you can remove from the list, the branch is fixed, as you can see here: lp:gdb

Benjamin Drung (bdrung)
description: updated
Benjamin Drung (bdrung)
description: updated
Revision history for this message
snd (dns) wrote :

Also affected by this bug:

lp:gzip
lp:parted

Revision history for this message
snd (dns) wrote :

sorry, lp:parted does not exist, i mean: lp:~rbose-vcs-imports/parted/master

Revision history for this message
Martin Pool (mbp) wrote :

Hi DNS,

We appreciate this bug is important and is blocking an increasing
number of projects. We have enough test data, so adding a list of
every affected branch is not especially going to help.

Revision history for this message
snd (dns) wrote :

Ok :-)

Revision history for this message
Heimen Stoffels (vistaus) wrote :

Then can we expect some work to be done, Martin? It's more than a year now and many are affected. Something has to be done.

Revision history for this message
Jani Uusi-Rantala (jani-uusi-rantala) wrote :

Prevents Qt5 base module from being imported from gitorious.. isn't that reason enough to fix this pronto.

Jelmer Vernooij (jelmer)
Changed in bzr:
status: Confirmed → In Progress
assignee: nobody → Jelmer Vernooij (jelmer)
milestone: none → 2.6b1
Vincent Ladeuil (vila)
Changed in bzr:
milestone: 2.6b1 → none
Revision history for this message
Eli Zaretskii (eliz) wrote :

What does the change of the milestone to "none" mean? Does it mean that the bug will never be fixed? That'd be a pity.

Revision history for this message
Vincent Ladeuil (vila) wrote :

@Eli: No, it means whatever was in progress was not finished when 2.6b1 was frozen, nothing more.

Jelmer Vernooij (jelmer)
Changed in bzr:
status: In Progress → Triaged
assignee: Jelmer Vernooij (jelmer) → nobody
Revision history for this message
Riccardo 'c10ud' (c10ud) wrote :

2012-05-19 11:17:49 INFO Unable to import branch because of limitations in Bazaar.
2012-05-19 11:17:49 INFO The repository you are fetching from contains submodules. To continue, upgrade your Bazaar repository to a format that supports nested trees, such as 'development-subtree'.

We're no longer using git submodules, is there a way to tell bzr/vcs-imports to just ignore history and focus on current code?
We'd like to setup a daily ppa for our software without using anything else than github and launchpad.

Thanks

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Hi c1oud,

There is no way to tell the import system to just ignore history. You could provide a custom repository that has the submodules filtered out of the history or filter them out of your history on github.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

I'm not sure why this is marked as affecting cloudfoundry? I don't see what changes would be needed in cloudfoundry for this to be fixed.

Revision history for this message
Juan L. Negron (negronjl) wrote :

CloudFoundry uses github. Getting sub modules supported would allow the
package to be built directly from their repository.

Thanks,

Juan

Sent from my mobile.
Apologies for the brevity of this message.
On Jun 28, 2012 6:05 AM, "Jelmer Vernooij" <email address hidden>
wrote:

> I'm not sure why this is marked as affecting cloudfoundry? I don't see
> what changes would be needed in cloudfoundry for this to be fixed.
>
> --
> You received this bug notification because you are a member of
> Canonical-SIG, which is subscribed to the bug report.
> https://bugs.launchpad.net/bugs/402814
>
> Title:
> Importing revisions with submodules is not supported
>
> Status in Bazaar Version Control System:
> Triaged
> Status in bzr git support plugin:
> Fix Released
> Status in CloudFoundry:
> Confirmed
> Status in Launchpad itself:
> Triaged
>
> Bug description:
> bzr-git converts submodules into nested trees, but nested trees aren't
> very well supported by bzr yet.
>
> Affected projects:
>
> lp:anarchnet
> lp:anubis
> astrid
> lp:autoconf-archive
> banshee
> lp:bison
> lp:cflow
> lp:coreutils
> lp:cpio (lp:~rbose-vcs-imports/cpio/master)
> lp:dico
> lp:diffutils
> emesene
> lp:findutils
> f-spot
> lp:gnu-cssc
> lp:gnu-radius
> lp:grep (lp:~rbose-vcs-imports/grep/master)
> gstreamer
> haasciicam
> lp:mailutils
> lp:mumble
> lp:patch
> pitivi
> QEMU
> qutim
> lp:sed
> tomahawk
> lp:vlc
> lp:xmms2
> lp:zile
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/bzr/+bug/402814/+subscriptions
>

Changed in launchpad:
importance: Low → Critical
importance: Critical → Low
importance: Low → High
Jelmer Vernooij (jelmer)
Changed in bzr:
importance: High → Low
importance: Low → Medium
Changed in launchpad:
importance: High → Low
importance: Low → High
Changed in bzr:
importance: Medium → High
Changed in bzr-git:
assignee: nobody → Jelmer Vernooij (jelmer)
Revision history for this message
Jarl (jarl-dk) wrote :

hmmm, many of the affected packages seems to be way old in recent ubuntu releases due to this bug, e.g. coreutils.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Thu, 2012-10-04 at 19:43 +0000, Jarl wrote:
> hmmm, many of the affected packages seems to be way old in recent ubuntu
> releases due to this bug, e.g. coreutils.
This shouldn't have any effect on what version of these packages is in
Ubuntu. New versions can be packaged without working VCS imports

Cheers,

Jelmer

Curtis Hovey (sinzui)
tags: added: feature git
Revision history for this message
Gianfranco Costamagna (costamagnagianfranco) wrote :

How can we workaround this bug?

I see vlc team is workarounding it [1], but I don't see how

[1] https://code.launchpad.net/~videolan/vlc/master-manual

Revision history for this message
Vsevolod Velichko (torkvemada) wrote :

LocutusOfBorg, I myself created cron script (for lp:qutim) with the following contents:
#!/bin/bash
cd /path/to/qutim-replica
git pull origin master
git submodule update --init --recursive
bzr stat -S|grep ^?|sed 's/^? //'| while read LINE; do bzr add "$LINE"; done
bzr ci -m "Cron update"
bzr push

You also have to add ..git* to your .bzrignore.

Revision history for this message
Benjamin Drung (bdrung) wrote :

I wrote a small hacky script [1] and run it every 15 minutes via Cron.

[1] https://bazaar.launchpad.net/~videolan/vlc/manual-bzr-import/view/head:/manual-bzr-import

Revision history for this message
skybon (skybon-deactivatedaccount-deactivatedaccount) wrote :

So... any ETA on the fix for the bug?

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Tue, Apr 16, 2013 at 01:16:25AM -0000, ?????????? ???????????????????? wrote:
> So... any ETA on the fix for the bug?
Nobody is actively working on this bug at the moment, as far as I am aware.

no longer affects: tribler
description: updated
description: updated
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

> CloudFoundry uses github. Getting sub modules supported would allow the
> package to be built directly from their repository.
Fixing this doesn't require any changes to cloudfoundry though, which is what that field is about (I agree the naming is a bit confusing).

Changed in cloudfoundry:
status: Confirmed → Invalid
Revision history for this message
pataquets (pataquets) wrote :

Also, with a github repo with no submodules at all, LP import fails complaining of submodules:
https://code.launchpad.net/~dnscrypt-proxy/dnscrypt-proxy/master

Revision history for this message
Martin Erik Werner (arand) wrote :

pataquets: That's because there are submodules at some point in the commit history, which will trigger the issue as well.

pataquets (pataquets)
description: updated
Jelmer Vernooij (jelmer)
description: updated
Revision history for this message
hexafraction (rarkenin) wrote :

arand: So you're stating that removing submodules isn't a workaround? Anyway it would be extremely beneficial to getting things running if submodules could be skipped as an option for imports, as in many cases any recipes may not even need to care about submodules.

Revision history for this message
Martin Erik Werner (arand) wrote :

hexafraction:
The only "workaround" (as far as I can tell, provided the way I guess the importer currently works) would be to go through the whole repository history and replace all commits that contained submodules with commits that didn't, and then run the importer on that.

Revision history for this message
hexafraction (rarkenin) wrote :

Thanks. Unfortunately for two repositories I use their design makes this untenable.

The only workaround I found was to install bazaar WITHOUT the git plugin, bzr init inside the existing repo, fix up ignores, and then just push that. The only pitfall is an extra modification to the git repo's state.

Revision history for this message
Matthias Mailänder (mailaender) wrote :

I read about workarounds here, but I don't know how to apply them to lp:openra The web-interface seems very limited in that regard. Is there another way or did some post a step-by-step guide somewhere. Not that the recent commits don't use git submodules anymore. It was many years ago, but still bzr complains because somewhere in the history git submodules have been detected. :(

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Sun, Dec 08, 2013 at 10:27:11PM -0000, Matthias Mailänder wrote:
> I read about workarounds here, but I don't know how to apply them to
> lp:openra The web-interface seems very limited in that regard. Is there
> another way or did some post a step-by-step guide somewhere. Not that
> the recent commits don't use git submodules anymore. It was many years
> ago, but still bzr complains because somewhere in the history git
> submodules have been detected. :(

The workarounds require you to do the import yourself and then
push the result bzr branch up to Launchpad; Launchpad can't
do them for you.

Revision history for this message
Sergey Belov (arikon) wrote :

Is there any progress on this issue?

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Thu, Oct 09, 2014 at 08:10:53PM -0000, Sergey Belov wrote:
> Is there any progress on this issue?
Nobody is currently working on this issue.

Revision history for this message
Sergey Belov (arikon) wrote :

As a workaround we could use github.com svn gateway, but it is not working.
Don't know why:

Example: http://launchpadlibrarian.net/186966468/saltstack-salt-develop.log

svn co https://github.com/saltstack/salt/trunk

It works on my laptop.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Fri, Oct 10, 2014 at 05:08:33AM -0000, Sergey Belov wrote:
> As a workaround we could use github.com svn gateway, but it is not working.
> Don't know why:
>
> Example: http://launchpadlibrarian.net/186966468/saltstack-salt-
> develop.log
>
> svn co https://github.com/saltstack/salt/trunk
>
> It works on my laptop.

Launchpad imports don't just do a checkout, they import the full
history.

GitHubs SVN protocol implementation appears to be behave differently
from "regular" SVN servers. E.g. visiting
https://github.com/saltstack/salt/trunk gives me a 404.

Revision history for this message
dreamcat4 (dreamcat4) wrote :

Github SVN url don't work because it uses the User-agent string to determine whether to serve SVN protocol... And bzr-svn (unlike the official svn client) does not set the User-agent string.

I have requested to Github Support to provide a solution. Because bzr-svn is EOL (for several years now). And it can't be improved / updated anymore.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Even if bzr-svn wasn't unmaintained, it would be wrong for us to change the user agent. Servers should not have different behaviour based on user agent.

On 24 April 2015 5:44:29 pm BST, dreamcat4 <email address hidden> wrote:
>Github SVN url don't work because it uses the User-agent string to
>determine whether to serve SVN protocol... And bzr-svn (unlike the
>official svn client) does not set the User-agent string.
>
>I have requested to Github Support to provide a solution. Because bzr-
>svn is EOL (for several years now). And it can't be improved / updated
>anymore.
>
>--
>You received this bug notification because you are subscribed to the
>bug
>report.
>https://bugs.launchpad.net/bugs/402814
>
>Title:
> Importing revisions with submodules is not supported
>
>To manage notifications about this bug go to:
>https://bugs.launchpad.net/bzr/+bug/402814/+subscriptions

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Even if bzr-svn wasn't EOL, it would be wrong for us to change the user agent. Servers should not have different behaviour based on user agent.

On 24 April 2015 5:44:29 pm BST, dreamcat4 <email address hidden> wrote:
>Github SVN url don't work because it uses the User-agent string to
>determine whether to serve SVN protocol... And bzr-svn (unlike the
>official svn client) does not set the User-agent string.
>
>I have requested to Github Support to provide a solution. Because bzr-
>svn is EOL (for several years now). And it can't be improved / updated
>anymore.
>
>--
>You received this bug notification because you are subscribed to the
>bug
>report.
>https://bugs.launchpad.net/bugs/402814
>
>Title:
> Importing revisions with submodules is not supported
>
>To manage notifications about this bug go to:
>https://bugs.launchpad.net/bzr/+bug/402814/+subscriptions

Revision history for this message
Josh Fogg (jfogg) wrote :

The bug is also preventing Numix Circle daily builds. Someone accidentally submitted a submodule and then it was removed, but it was many many commits later before we realised that our Launchpad builds had broken. It's now such a tangled mess that we can't really unpick the error.

A fix for this would be much appreciated.

Revision history for this message
dreamcat4 (dreamcat4) wrote :

Joshua,

Listen to me: you MUST rewind the git repo(s) for your official priject back to a point before the submodule commit. It's your only hope to recover from this situation on Launchpad.net. Then carefully re-apply any commits since then cleanly. If it's not too late, that's your least - cost option.

You can try messing around with git-->svn-->bzr import but that is simply flat out not gonna work if you're source repo is on github.com. (due to other bug in bzr / launchpad)

The entire bzr toolset is now defunct, including the bzr-git helper tool. So those bugs are never getting fixed / going away. The launchpad team are in a transition period right now, and it's a priority for them to get proper git hosting going (including native git-git imports). But they don't have any solid timeframe by which you can reasonably expect that to become available.

I say that coming from another project, where we already investigated every other possible workaround, including importing via git->svn->bzr, and asking github to help us work around the blocking launchpad bugs / issues. It was all dead ends. Eventually I just had to set up a brand new build system on dockerhub (tvheadend project). With a 'maybe when' can return back to launchpad.net.

For reference: our project the tvheadend builds had been broken on launchpad for about 2 years or something. Way before I got involved with it. So it was already far too late to be rewinding / picking through the commits.

Revision history for this message
Khurshid Alam (khurshid-alam) wrote :

Launchpad beta now supports native git. Although, I think, it can not be used for ppa build.

Revision history for this message
Synapse Developer Team (synapse-network) wrote :

Bump on this issue. I hope launchpad supports native git submodules in the project.

Revision history for this message
Synapse Developer Team (synapse-network) wrote :

Perhaps adding a simple line `git submodule update --init --recursive` solves the problem. Why is this issue not fixed yet? I think the fix is fairly easy.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Wed, Aug 31, 2016 at 04:54:20AM -0000, Synapse Developer Team wrote:
> Perhaps adding a simple line `git submodule update --init --recursive`
> solves the problem. Why is this issue not fixed yet? I think the fix is
> fairly easy.

This bug is about importing revisions with submodules from git into bazaar.
Support for submodules in git->git imported repositories is a separate (Launchpad) issue.

Revision history for this message
Jasem Mutlaq (mutlaqja) wrote :

We accidentally used a git submodule which was removed later on but against now stuck with this bug as the import failed for the last couple of weeks. Is there any ETA on this bug? I created a git import which worked, but then I needed nest-part from another project which was bzr-hosted so that didn't work and I couldn't build my recipe.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Sun, Oct 02, 2016 at 01:09:14PM -0000, Jasem Mutlaq wrote:
> We accidentally used a git submodule which was removed later on but
> against now stuck with this bug as the import failed for the last couple
> of weeks. Is there any ETA on this bug? I created a git import which
> worked, but then I needed nest-part from another project which was bzr-
> hosted so that didn't work and I couldn't build my recipe.

Nobody is working on this bug at the moment, and as far as I am aware
there are no plans to work on it in the near future.

Jelmer

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Your best bet is probably to use the git imports, and use daily builds
from git.

Revision history for this message
Pali (pali) wrote :

@jelmer: How to use that git imports? Can launchpad import/mirror git repository foreign git repository?

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

On Sun, Oct 23, 2016 at 10:09:45AM -0000, Pali wrote:
> @jelmer: How to use that git imports? Can launchpad import/mirror git
> repository foreign git repository?
The launchpad folks should be able to help you with that; I'm just
involved in bzr-git/bzr these days.

Revision history for this message
Colin Watson (cjwatson) wrote :

Launchpad git-to-git imports are nearly done; see bug 1469459 if you want to follow that work.

Revision history for this message
Colin Watson (cjwatson) wrote :

Launchpad git-to-git imports are available now: http://blog.launchpad.net/code/git-to-git-imports

Revision history for this message
Cas (calumlind) wrote :

I can confirm that using git-to-git import works as expected. We annoyingly had old submodules usage in the git history that prevented importing before now. Big thanks Colin! :)

Jelmer Vernooij (jelmer)
Changed in brz:
status: New → Triaged
Jelmer Vernooij (jelmer)
Changed in brz:
importance: Undecided → Medium
tags: added: foreign
Jelmer Vernooij (jelmer)
tags: added: git-support
Jelmer Vernooij (jelmer)
Changed in brz-git:
status: New → Fix Released
importance: Undecided → Medium
assignee: nobody → Jelmer Vernooij (jelmer)
milestone: none → integration
Jelmer Vernooij (jelmer)
no longer affects: brz-git
Changed in brz:
assignee: nobody → Jelmer Vernooij (jelmer)
Changed in bzr-git:
assignee: Jelmer Vernooij (jelmer) → nobody
Jelmer Vernooij (jelmer)
Changed in brz:
status: Triaged → In Progress
Jelmer Vernooij (jelmer)
Changed in brz:
milestone: none → 3.1.0
Revision history for this message
Jelmer Vernooij (jelmer) wrote :

Cloning git repositories with submodules now works in Breezy, and it's possible to pull Git branches with submodules into Breezy branches.

Working tree updates don't work yet - this requires changes to tree transform.

Revision history for this message
Jelmer Vernooij (jelmer) wrote :

The support for nested trees in transform is being tracked in bug 267770.

Changed in brz:
status: In Progress → Fix Committed
Jelmer Vernooij (jelmer)
Changed in brz:
status: Fix Committed → Fix Released
Colin Watson (cjwatson)
affects: launchpad → lp-codeimport
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Related questions

Related blueprints

Remote bug watches

Bug watches keep track of this bug in other bug trackers.