Launchpad itself

import external code into a LP git repo (natively)

Bug #1469459 reported by Timmie
80
This bug affects 13 people
Affects Status Importance Assigned to Milestone
Launchpad itself
Fix Released
High
Colin Watson
turnip
Fix Released
High
Colin Watson

Bug Description

In the light of the announcement that LP can now host git repositories, kindly add support to automatically VcsImports from external repositories/platforms (e.g. github) directly as git repository.

So the import request
https://code.launchpad.net/+code-imports/+new
 would rather create a git repo at LP if git repos are imported

Tags: feature git qa-ok

Related branches

lp:~cjwatson/launchpad/db-git-imports
William Grant (community): Approve (db)
Stuart Bishop: Pending (db) requested
Diff: 31 lines (+27/-0)
1 file modified
database/schema/patch-2209-80-0.sql (+27/-0)
lp:~cjwatson/launchpad/git-repository-type
William Grant (community): Approve (code)
Diff: 808 lines (+258/-46)
18 files modified
database/schema/security.cfg (+1/-0)
lib/lp/app/widgets/doc/launchpad-radio-widget.txt (+2/-2)
lib/lp/code/browser/gitref.py (+5/-1)
lib/lp/code/browser/gitrepository.py (+5/-2)
lib/lp/code/browser/tests/test_gitrepository.py (+10/-1)
lib/lp/code/enums.py (+29/-4)
lib/lp/code/errors.py (+11/-0)
lib/lp/code/interfaces/gitnamespace.py (+3/-3)
lib/lp/code/interfaces/gitrepository.py (+8/-0)
lib/lp/code/model/gitnamespace.py (+7/-6)
lib/lp/code/model/gitrepository.py (+27/-3)
lib/lp/code/model/tests/test_gitnamespace.py (+33/-16)
lib/lp/code/model/tests/test_gitrepository.py (+19/-0)
lib/lp/code/xmlrpc/git.py (+6/-3)
lib/lp/code/xmlrpc/tests/test_git.py (+10/-0)
lib/lp/scripts/garbo.py (+31/-0)
lib/lp/scripts/tests/test_garbo.py (+43/-1)
lib/lp/testing/factory.py (+8/-4)
lp:~cjwatson/launchpad/codeimport-git-model
William Grant (community): Approve (code)
Diff: 1495 lines (+427/-159)
16 files modified
lib/lp/code/configure.zcml (+2/-0)
lib/lp/code/doc/codeimport.txt (+14/-4)
lib/lp/code/emailtemplates/code-import-status-updated.txt (+1/-1)
lib/lp/code/emailtemplates/new-code-import.txt (+1/-1)
lib/lp/code/enums.py (+21/-1)
lib/lp/code/interfaces/codeimport.py (+16/-4)
lib/lp/code/interfaces/gitnamespace.py (+8/-2)
lib/lp/code/mail/codeimport.py (+11/-11)
lib/lp/code/mail/tests/test_codeimport.py (+35/-8)
lib/lp/code/model/codeimport.py (+69/-18)
lib/lp/code/model/codeimportjob.py (+14/-13)
lib/lp/code/model/gitnamespace.py (+4/-1)
lib/lp/code/model/hasbranches.py (+6/-2)
lib/lp/code/model/tests/test_codeimport.py (+200/-85)
lib/lp/registry/browser/product.py (+5/-1)
lib/lp/testing/factory.py (+20/-7)
lp:~cjwatson/launchpad/git-xmlrpc-auth-params
William Grant (community): Approve (code)
Diff: 274 lines (+76/-47)
2 files modified
lib/lp/code/xmlrpc/git.py (+11/-3)
lib/lp/code/xmlrpc/tests/test_git.py (+65/-44)
lp:~cjwatson/launchpad/codeimport-git-auth
William Grant (community): Approve (code)
Diff: 596 lines (+353/-24)
7 files modified
lib/lp/code/configure.zcml (+9/-0)
lib/lp/code/model/codeimportjob.py (+54/-0)
lib/lp/code/model/tests/test_codeimportjob.py (+100/-1)
lib/lp/code/xmlrpc/git.py (+61/-14)
lib/lp/code/xmlrpc/tests/test_git.py (+81/-9)
lib/lp/services/config/schema-lazr.conf (+3/-0)
lib/lp/services/macaroons/interfaces.py (+45/-0)
lp:~cjwatson/launchpad/codeimport-source-details-refactor
William Grant (community): Approve (code)
Diff: 211 lines (+51/-55)
5 files modified
lib/lp/code/xmlrpc/codeimportscheduler.py (+3/-3)
lib/lp/code/xmlrpc/tests/test_codeimportscheduler.py (+3/-3)
lib/lp/codehosting/codeimport/tests/test_worker.py (+37/-42)
lib/lp/codehosting/codeimport/tests/test_workermonitor.py (+4/-4)
lib/lp/codehosting/codeimport/worker.py (+4/-3)
lp:~cjwatson/launchpad/codeimport-worker-refactor
William Grant (community): Approve (code)
Diff: 547 lines (+114/-96)
7 files modified
lib/lp/code/xmlrpc/codeimportscheduler.py (+4/-4)
lib/lp/code/xmlrpc/tests/test_codeimportscheduler.py (+5/-5)
lib/lp/codehosting/codeimport/tests/test_worker.py (+16/-16)
lib/lp/codehosting/codeimport/tests/test_workermonitor.py (+6/-6)
lib/lp/codehosting/codeimport/worker.py (+74/-56)
lib/lp/codehosting/codeimport/workermonitor.py (+5/-5)
lib/lp/testing/factory.py (+4/-4)
lp:~cjwatson/launchpad/codeimport-create-hosting
William Grant (community): Approve (code)
Diff: 206 lines (+35/-8)
6 files modified
lib/lp/code/mail/tests/test_codeimport.py (+2/-0)
lib/lp/code/model/codeimport.py (+3/-0)
lib/lp/code/model/tests/test_codeimport.py (+24/-5)
lib/lp/code/model/tests/test_codeimportjob.py (+2/-0)
lib/lp/code/tests/helpers.py (+1/-0)
lib/lp/snappy/browser/tests/test_hassnaps.py (+3/-3)
lp:~cjwatson/launchpad/codeimport-git-worker
William Grant (community): Approve (code)
Diff: 843 lines (+331/-77)
11 files modified
configs/testrunner-appserver/launchpad-lazr.conf (+3/-0)
lib/lp/buildmaster/model/buildfarmjobbehaviour.py (+3/-15)
lib/lp/codehosting/codeimport/tests/servers.py (+7/-4)
lib/lp/codehosting/codeimport/tests/test_worker.py (+57/-6)
lib/lp/codehosting/codeimport/tests/test_workermonitor.py (+60/-9)
lib/lp/codehosting/codeimport/worker.py (+160/-26)
lib/lp/services/config/fixture.py (+3/-3)
lib/lp/services/config/schema-lazr.conf (+1/-1)
lib/lp/services/utils.py (+14/-1)
lib/lp/testing/factory.py (+7/-4)
scripts/code-import-worker.py (+16/-8)
lp:~cjwatson/launchpad/git-code-import-security-deletion
William Grant (community): Approve (code)
Diff: 436 lines (+126/-21)
11 files modified
lib/lp/_schema_circular_imports.py (+1/-0)
lib/lp/code/interfaces/branch.py (+5/-0)
lib/lp/code/interfaces/gitnamespace.py (+3/-1)
lib/lp/code/interfaces/gitrepository.py (+14/-2)
lib/lp/code/model/branch.py (+3/-4)
lib/lp/code/model/gitcollection.py (+11/-1)
lib/lp/code/model/gitnamespace.py (+3/-3)
lib/lp/code/model/gitrepository.py (+23/-0)
lib/lp/code/model/tests/test_branch.py (+7/-9)
lib/lp/code/model/tests/test_gitrepository.py (+55/-0)
lib/lp/security.py (+1/-1)
lp:~cjwatson/launchpad/codeimport-git-read-only-views
William Grant (community): Approve (code)
Diff: 620 lines (+186/-88)
13 files modified
lib/lp/code/browser/branch.py (+18/-35)
lib/lp/code/browser/codeimport.py (+50/-11)
lib/lp/code/browser/configure.zcml (+5/-2)
lib/lp/code/browser/gitrepository.py (+19/-1)
lib/lp/code/browser/tests/test_codeimport.py (+39/-19)
lib/lp/code/configure.zcml (+1/-0)
lib/lp/code/interfaces/codeimport.py (+10/-4)
lib/lp/code/model/codeimport.py (+7/-0)
lib/lp/code/templates/branch-index.pt (+1/-1)
lib/lp/code/templates/codeimport-machine-index.pt (+2/-2)
lib/lp/code/templates/codeimport-machines.pt (+3/-3)
lib/lp/code/templates/gitrepository-index.pt (+10/-0)
lib/lp/code/templates/import-details.pt (+21/-10)
lp:~cjwatson/launchpad/codeimport-list-git
William Grant (community): Approve (code)
Diff: 299 lines (+111/-38)
5 files modified
lib/lp/code/browser/codeimport.py (+15/-4)
lib/lp/code/interfaces/codeimport.py (+4/-1)
lib/lp/code/model/codeimport.py (+5/-1)
lib/lp/code/stories/codeimport/xx-codeimport-view.txt (+70/-27)
lib/lp/code/templates/codeimport-list.pt (+17/-5)
lp:~cjwatson/launchpad/codeimport-git-edit-views
William Grant (community): Approve (code)
Diff: 699 lines (+202/-144)
11 files modified
lib/lp/code/browser/branch.py (+2/-98)
lib/lp/code/browser/codeimport.py (+111/-17)
lib/lp/code/browser/configure.zcml (+20/-2)
lib/lp/code/browser/gitrepository.py (+9/-9)
lib/lp/code/javascript/tests/test_util.js (+2/-2)
lib/lp/code/javascript/util.js (+2/-2)
lib/lp/code/stories/codeimport/xx-admin-codeimport.txt (+46/-5)
lib/lp/code/stories/codeimport/xx-create-codeimport.txt (+2/-2)
lib/lp/code/templates/import-details.pt (+4/-4)
lib/lp/registry/browser/product.py (+2/-1)
lib/lp/registry/browser/tests/productseries-setbranch-view.txt (+2/-2)
lp:~cjwatson/launchpad/codeimport-git-webservice
William Grant (community): Approve (code)
Diff: 480 lines (+185/-22)
10 files modified
lib/lp/code/errors.py (+21/-0)
lib/lp/code/interfaces/codeimport.py (+4/-0)
lib/lp/code/interfaces/hasbranches.py (+13/-7)
lib/lp/code/model/codeimport.py (+13/-1)
lib/lp/code/model/hasbranches.py (+15/-6)
lib/lp/code/stories/webservice/xx-code-import.txt (+107/-4)
lib/lp/registry/configure.zcml (+5/-0)
lib/lp/registry/interfaces/distributionsourcepackage.py (+3/-2)
lib/lp/registry/interfaces/product.py (+1/-1)
lib/lp/registry/model/distributionsourcepackage.py (+3/-1)
lp:~cjwatson/launchpad/codeimport-git-worker-fixes
William Grant (community): Approve (code)
Diff: 39 lines (+6/-3)
1 file modified
lib/lp/codehosting/codeimport/worker.py (+6/-3)
lp:~cjwatson/launchpad/codeimport-git-new-view
William Grant (community): Approve (code)
Diff: 560 lines (+238/-35)
7 files modified
lib/lp/code/browser/codeimport.py (+59/-16)
lib/lp/code/interfaces/gitnamespace.py (+7/-0)
lib/lp/code/model/codeimport.py (+0/-1)
lib/lp/code/model/gitnamespace.py (+13/-0)
lib/lp/code/model/tests/test_gitnamespace.py (+99/-0)
lib/lp/code/stories/codeimport/xx-create-codeimport.txt (+49/-17)
lib/lp/code/templates/codeimport-new.pt (+11/-1)
lp:~cjwatson/launchpad/codeimport-git-refactor-name-validation
William Grant (community): Approve (code)
Diff: 181 lines (+42/-39)
4 files modified
lib/lp/code/browser/branch.py (+13/-19)
lib/lp/code/browser/codeimport.py (+20/-15)
lib/lp/registry/browser/product.py (+5/-3)
lib/lp/registry/browser/tests/productseries-setbranch-view.txt (+4/-2)
lp:~cjwatson/launchpad/codeimport-git-worker-sync-head
William Grant (community): Approve (code)
Diff: 454 lines (+233/-45)
4 files modified
lib/lp/codehosting/codeimport/tests/servers.py (+50/-3)
lib/lp/codehosting/codeimport/tests/test_workermonitor.py (+78/-33)
lib/lp/codehosting/codeimport/worker.py (+103/-9)
scripts/code-import-worker.py (+2/-0)
lp:~cjwatson/launchpad/codeimport-git-configure-code
William Grant (community): Approve (code)
Diff: 745 lines (+335/-120)
6 files modified
lib/lp/code/javascript/productseries-setbranch.js (+26/-16)
lib/lp/code/javascript/tests/test_productseries-setbranch.html (+1/-1)
lib/lp/code/templates/configure-code.pt (+41/-14)
lib/lp/registry/browser/product.py (+197/-83)
lib/lp/registry/browser/tests/productseries-setbranch-view.txt (+3/-3)
lib/lp/registry/browser/tests/test_product.py (+67/-3)
~cjwatson/turnip:large-http-push
William Grant (community): Approve (code)
Diff: 167 lines (+70/-10)
4 files modified
turnip/pack/git.py (+7/-1)
turnip/pack/http.py (+21/-5)
turnip/pack/tests/test_functional.py (+39/-0)
turnip/pack/tests/test_http.py (+3/-4)
~cjwatson/turnip:set-symbolic-ref
Otto Co-Pilot: Needs Fixing
William Grant (community): Approve (code)
Diff: 497 lines (+297/-26)
8 files modified
README (+16/-0)
turnip/pack/git.py (+83/-9)
turnip/pack/hookrpc.py (+6/-2)
turnip/pack/http.py (+5/-2)
turnip/pack/ssh.py (+2/-1)
turnip/pack/tests/test_functional.py (+65/-11)
turnip/pack/tests/test_git.py (+118/-0)
turnip/pack/tests/test_http.py (+2/-1)
Revision history for this message
Colin Watson (cjwatson) wrote :

Right, we've been talking about the design for this but haven't started implementation yet. It's also useful (though not required) for https://bugs.launchpad.net/launchpad/+bug/1453022.

tags: added: feature git
Changed in launchpad:
status: New → Triaged
importance: Undecided → High
William Grant (wgrant)
no longer affects: debian
Revision history for this message
Timmie (timmie) wrote :

Good day.
any update on this issue?

Thanks.

Revision history for this message
Colin Watson (cjwatson) wrote :

No, no progress to report at the moment. Hopefully we'll get to it next year!

Revision history for this message
Timmie (timmie) wrote :

Good day.
any update on this issue: when it will be tackled?

Thanks.

Colin Watson (cjwatson)
Changed in launchpad:
assignee: nobody → Colin Watson (cjwatson)
status: Triaged → In Progress
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r13561 in db-stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/db-stable/revision/13561) is part of this bug's fix.

tags: added: qa-needstesting
Revision history for this message
Colin Watson (cjwatson) wrote :

2016-10-04 13:38:09,590 INFO 2209-80-0 applied just now in 0.6 seconds

tags: added: qa-ok
removed: qa-needstesting
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18222 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18222) is part of this bug's fix.

tags: added: qa-needstesting
removed: qa-ok
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18223 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18223) is part of this bug's fix.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18225 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18225) is part of this bug's fix.

Revision history for this message
Colin Watson (cjwatson) wrote :

The translatePath changes haven't gone so well:

  https://oops.canonical.com/oops/?oopsid=OOPS-866c482efc3a14306fff7053d7366249

I'll look at soon as I can.

tags: added: qa-bad
removed: qa-needstesting
Revision history for this message
Colin Watson (cjwatson) wrote :

r18227 is better again.

tags: added: qa-ok
removed: qa-bad
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18229 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18229) is part of this bug's fix.

tags: added: qa-needstesting
removed: qa-ok
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18230 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18230) is part of this bug's fix.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18231 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18231) is part of this bug's fix.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18232 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18232) is part of this bug's fix.

Colin Watson (cjwatson)
tags: added: qa-ok
removed: qa-needstesting
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18233 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18233) is part of this bug's fix.

tags: added: qa-needstesting
removed: qa-ok
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18235 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18235) is part of this bug's fix.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18236 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18236) is part of this bug's fix.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18237 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18237) is part of this bug's fix.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18239 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18239) is part of this bug's fix.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18240 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18240) is part of this bug's fix.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18241 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18241) is part of this bug's fix.

Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18242 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18242) is part of this bug's fix.

Colin Watson (cjwatson)
tags: added: qa-ok
removed: qa-needstesting
Colin Watson (cjwatson)
Changed in turnip:
status: New → Triaged
importance: Undecided → High
Colin Watson (cjwatson)
Changed in turnip:
assignee: nobody → Colin Watson (cjwatson)
status: Triaged → In Progress
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18250 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18250) is part of this bug's fix.

tags: added: qa-needstesting
removed: qa-ok
Colin Watson (cjwatson)
tags: added: qa-ok
removed: qa-needstesting
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

r18259 in stable (http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18259) is part of this bug's fix.

tags: added: qa-needstesting
removed: qa-ok
Revision history for this message
Launchpad QA Bot (lpqabot) wrote :

Fixed in stable r18260 <http://bazaar.launchpad.net/~launchpad-pqm/launchpad/stable/revision/18260>.

Changed in launchpad:
status: In Progress → Fix Committed
Colin Watson (cjwatson)
Changed in turnip:
status: In Progress → Fix Released
tags: added: qa-ok
removed: qa-needstesting
Revision history for this message
Colin Watson (cjwatson) wrote :

http://blog.launchpad.net/code/git-to-git-imports

Changed in launchpad:
status: Fix Committed → Fix Released
Revision history for this message
Timmie (timmie) wrote :

Just for the sake of giving this a bit historical background:

the original bug report was triggered by the following Q&A:
code import from GitHub failing - https://answers.launchpad.net/launchpad/+question/264432

Being the original report, I received the status updates over the last months picking up pace.

So I just want to send a very huge thank you to Colin Watson for addressing this issue, working on it constantly and finally releasing a fix.

The very nice consequence is that we can now re-enable the automatic builds for any software developed & hosted primary on Github!
(maybe you'd want to add this to your blog post)

Thanks so much!

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.