recipe builds for git projects don't work with submodules

Bug #1733603 reported by Alan Griffiths on 2017-11-21
84
This bug affects 17 people
Affects Status Importance Assigned to Milestone
git-build-recipe
High
Corentin Noël
open62541
New
Undecided
Unassigned

Bug Description

When generating the source package git submodules are not checked out.

I think all this requires is "git submodule update --init --recursive" after merging any branches.

Tags: git Edit Tag help

Related branches

Colin Watson (cjwatson) on 2017-11-21
affects: launchpad → git-build-recipe
Otto Kekäläinen (otto) wrote :

This affects our project as well. You can see it in the live here: https://code.launchpad.net/~otto/+recipe/mariadb-10.3-dev-daily

The error it hangs on:

**********
CMake Error at cmake/submodules.cmake:27 (MESSAGE):
  No MariaDB Connector/C! Run

      git submodule update --init

  Then restart the build.
**********

patrick jp (patrickelectric) wrote :

Any update over this issue ?

Colin Watson (cjwatson) on 2018-08-29
Changed in git-build-recipe:
status: New → In Progress
importance: Undecided → High
assignee: nobody → Corentin Noël (tintou)
ezzieyguywuf (ezzieyguywuf) wrote :

The following patch fixes this bug. I cannot figure out how to fork and request a merge on launchpad.

Is anyone maintaining this tool?

diff --git a/gitbuildrecipe/recipe.py b/gitbuildrecipe/recipe.py
index d1700f2..1da7f2a 100644
--- a/gitbuildrecipe/recipe.py
+++ b/gitbuildrecipe/recipe.py
@@ -318,6 +318,8 @@ def pull_or_clone(base_branch, target_path):
         # checkout to interpret its argument as a commit?), but that's
         # their problem.
         base_branch.git_call("checkout", base_branch.commit)
+ # initialize submodules
+ base_branch.git_call("submodule", "update", "--init", "--recursive")
     except subprocess.CalledProcessError as e:
         raise CheckoutFailed(e.output)

Sergey Ponomarev (stokito) wrote :

~tintou it looks like this bug is assigned to you. Can you share with us a status?

This problem also is mentioned here
https://askubuntu.com/questions/954334/how-do-i-build-from-git-repository-with-submodules-on-launchpad

And after the fix it would be good to leave a comment there.

Jasem Mutlaq (mutlaqja) wrote :

Was this bug resolved or Launchpad still fails with submodules?

Murilo Marinho (murlou) wrote :

Is there any workaround for this? Trying to clone using the debian/rules is not working as well. Can someone enlighten me?

DrX@TN (drx.tn) wrote :

It also affects a project I'm running. Is there any progress on this issue?

Sergey Ponomarev (stokito) wrote :

Any update on this?

As I workaround I used nest repositories
https://help.launchpad.net/Packaging/SourceBuilds/Recipes#Nesting

It wasn't so easy to do, because all this subrepos should be imported to LP manually:
1. open you account
2. open Code tab
3. On the right Configure Code
4. Import a Git repository hosted somewhere else: here put an url to github repo and name the same as submodule.
5. Press Update
6. After importing a new LP repository will be created but it will reuse existing name. We'll fix that in 8 step.
7. Do the same for other submodules if any.
8. Fix back main code repo: "Link to a Git repository already on Launchpad" and select an old repo name.
9. Edit recipe and for each submodule create a nest record.

Here is an example recipe:

# git-build-recipe format 0.4 deb-version {debupstream}-{time}
lp:openwrt master
nest libubox lp:~stokito/openwrt/+git/openwrt-libubox libubox
nest uci lp:~stokito/openwrt/+git/openwrt-uci uci
nest ubus lp:~stokito/openwrt/+git/openwrt-ubus ubus
nest ustream-ssl lp:~stokito/openwrt/+git/openwrt-ustream-ssl ustream-ssl
nest uhttpd lp:~stokito/openwrt/+git/openwrt-uhttpd uhttpd
nest rpcd lp:~stokito/openwrt/+git/openwrt-rpcd rpcd
nest jsonpath lp:~stokito/openwrt/+git/openwrt-jsonpath jsonpath
nest mountd lp:~stokito/openwrt/+git/openwrt-mountd mountd

Frederic LOUI (frederic-loui) wrote :

Hi, while Sergey work around is great, my issue is that when the git repository is using a different commit of the submodules, this can break code compilation. Therefore I was wondering when the patch proposed by [ezzieyguywuf] would be accepted and go to mainstream ? For now I'm using nest in order to build the packages but as soon as a new submodule commit is used, this break oackage building as a result of a compilation failure ...

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers