[Packaging] Sync source packages from upstream to our Git repository

[Ivan Udovichenko]

Hello,

At this moment we're adding packages one by one from the same source (Debian repositories [1] and Alioth [2]).
We need a Jenkins job/tool/script/whatever which will sync source packages to our Git repository.
As input we may provide list of packages with the actual sources.
A mechanism which will automatically create projects for required packages in Gerrit is also required.
We also need a Jenkins job which will automatically trigger build for a list of packages against the source package repository (for example, source packages in 8.0 branch of packages/distro/pkg_name projects).
All of these mechanisms will increase the speed of packaging for new releases of MOS and overall quality of packages.

If you need help, please ask Thomas Goirand or MOS Packaging Team.

Thank you.

[1] Stable, Unstable, Testing
[2] http://alioth.debian.org/

[Thomas Goirand (thomas-goirand)]

As per Dmitry B., here's more info:

* paths to git projects (http or ssh) and branches to use:

All packages are stored at:
git clone git://anonscm.debian.org/openstack/FOO.git

Branches are always the default one when cloning, except for a list which is described here:
http://anonscm.debian.org/cgit/openstack/openstack-pkg-tools.git/tree/etc/pkgos/packages-list?h=debian/liberty

For example, if a package is listed at http://anonscm.debian.org/cgit/openstack/openstack-pkg-tools.git/tree/etc/pkgos/packages-list/openstack-release-branch?h=debian/liberty, then we should use the debian/liberty branch.

If a pakcage is listed in http://anonscm.debian.org/cgit/openstack/openstack-pkg-tools.git/tree/etc/pkgos/packages-list/experimental-branch?h=debian/liberty then we should use the debian/experimental branch.

if a package is listed at: http://anonscm.debian.org/cgit/openstack/openstack-pkg-tools.git/tree/etc/pkgos/packages-list/pkg-javascript-repo?h=debian/liberty then we should clone it from git://anonscm.debian.org/git/pkg-javascript/FOO.git

On top of all packages within the PKG OpenStack Alioth group, there's those which I just backport from Sid. Those are listed at in here: http://liberty-jessie.pkgs.mirantis.com/debian/dists/jessie-liberty-backports-nochange/main/source/Sources though this is a list of needed backports for Jessie, there may be a little bit more things to backport for Trusty (like python-netaddr, python-requests and such).

* build target distro:
Currently Trusty, though it would be nice to also have Jessie

If you have more question, let me know.

[Thomas Goirand (thomas-goirand)]

What was discussed with Alexandra Fedorova (and others) during a hangout meeting, is that what we need is a parametrized script in Jenkins to create an initial branch, which would create an initial Debian branch, which would be continuously synched. Then an initial MOS 8.0 branch would be created *once*, containing what's in the debian branch. Later on, merging from the Debian branch to the MOS 8.0 branch would still be a manual job.

[Fuel Devops McRobotson (fuel-devops-robot)]

Related fix proposed to branch: master
Change author: Alexander Charykov <email address hidden>
Review: https://review.fuel-infra.org/13939

[Fuel Devops McRobotson (fuel-devops-robot)]

Reviewed: https://review.fuel-infra.org/13939
Submitter: Igor Shishkin <email address hidden>
Branch: master

Commit: e09cbb3d327c8abc8f5f1fa532b0f5c3472536c3
Author: Alexander Charykov <email address hidden>
Date: Mon Nov 16 13:34:21 2015

Adding python-yaml and python-git to all slaves

Change-Id: I115ccb779a190a9e78096dc84682650263e8e738
Related-Bug: #1497275

[Igor Shishkin (teran)]

Commit is merged, assigning back.