Enable git clone depth override in opentstack-origin-git

[Nuno Santos]

This bug has been updated from it's original intent. The original bug description details are at the end of this section.

The deploy from source support in the OpenStack charms performs shallow git clones (e.g. git clone --depth 1) by default. This increases the speed of a git clone significantly. However, there is a side-effect of shallow clones when they are pip installed, in that it causes pip to list the package version as 0.0.0, when in actuality the package version may have been 11.0.0. The next time a pip installed package needs a minimum version of that package (e.g. let's say it needs >= 10.0.0), it'll see that 0.0.0 is installed and it'll install the latest package from pypi.

Non-shallow git clones enable pip to see the actual version and prevent this issue. The openstack-origin-git yaml should support an option to override the default depth of git clones on a per-repo basis.

-------------------original bug details-------------------------------

Here's the original bug description, which is really a dup of upstream bug https://bugs.launchpad.net/pbr/+bug/1375048.

"Missing dependency on testrepository"

I had the source bundle at https://pastebin.canonical.com/147053/
(adapted from http://bazaar.launchpad.net/~ost-maintainers/openstack-charm-testing/trunk/files/head:/bundles/source/
, basically flattened to v4 format to allow for machine allocation/placement) working before, and yesterday noticed that it's now failing with a missing dependency (pip package testrepository) for neutron-fwaas (which is brought in by neutron-api).
The relevant unit error log is at https://pastebin.canonical.com/147054/

[Nuno Santos (nunosantos)]

Should add that if I manually run "pip install testrepository", and then manually retry the failed command ("pip install /mnt/openstack-git/neutron") it works.

[Corey Bryant (corey.bryant)]

This is in the midst of being fixed upstream in pbr. It looks like it'll land soon in stable/kilo.

Here's the upstream bug: https://bugs.launchpad.net/pbr/+bug/1375048
And here's the upstream patch review: https://review.openstack.org/#/c/263928/

I'm not sure why we started hitting this all of a sudden. I couldn't find a recent commit in stable/kilo.

To get the fix is a little tricky at the moment, unfortunately, because the charms perform shallow git clones (for performance), and therefore the pip version of packages installed from git become 0.0.0.

To pick up the fix in the charms, I cherry-picked the upstream fix from the patch review above to my own repo (git://github.com/coreycb/pbr), and added it to neutron-kilo.yaml as follows:

repositories:
  - {name: requirements,
     repository: 'git://github.com/openstack/requirements',
     branch: stable/kilo}
  - {name: pbr,
     repository: 'git://github.com/coreycb/pbr',
     branch: stable/kilo}
  - {name: neutron-fwaas,
     repository: 'git://github.com/openstack/neutron-fwaas',
     branch: stable/kilo}
  - {name: neutron-lbaas,
     repository: 'git://github.com/openstack/neutron-lbaas',
     branch: stable/kilo}
  - {name: neutron-vpnaas,
     repository: 'git://github.com/openstack/neutron-vpnaas',
     branch: stable/kilo}
  - {name: neutron,
     repository: 'git://github.com/openstack/neutron',
     branch: stable/kilo}
directory: /mnt/openstack-git
http_proxy: http://squid.internal:3128
https_proxy: http://squid.internal:3128

This *should* fix the issue and run with the new pbr, *however* the charms do shallow git clones, so the version of the pip installed pbr results in 0.0.0, and the ensuing install of neutron-api re-installs the bad pbr because it requires a higher version that 0.0.0.

For this bug I want to add an option to the openstack-origin-git yaml for all of the charms to enable full git clones, which will allow pip to see the real version of pbr in this situation.

[Corey Bryant (corey.bryant)]

Here's sample openstack-origin-git yaml from the fix I have in progress. This overrides the default shallow clone of pbr to a normal git clone (see pbr depth):

repositories:
  - {name: requirements,
     repository: 'git://github.com/openstack/requirements',
     branch: stable/kilo}
  - {name: pbr,
     repository: 'git://github.com/coreycb/pbr',
     branch: stable/kilo,
     depth: 0}
  - {name: neutron-fwaas,
     repository: 'git://github.com/openstack/neutron-fwaas',
     branch: stable/kilo}
  - {name: neutron-lbaas,
     repository: 'git://github.com/openstack/neutron-lbaas',
     branch: stable/kilo}
  - {name: neutron-vpnaas,
     repository: 'git://github.com/openstack/neutron-vpnaas',
     branch: stable/kilo}
  - {name: neutron,
     repository: 'git://github.com/openstack/neutron',
     branch: stable/kilo}

[Nuno Santos (nunosantos)]

Just tested with the latest changes, and am still getting the error: https://pastebin.canonical.com/147359/
This is the neutron-kilo.yaml I have in place: https://pastebin.canonical.com/147360/

[Corey Bryant (corey.bryant)]

Are you using the next charms? These fixes have only landed in the next charms. Note that the fix for nova-compute hasn't landed, but should land by eod today.

Also note that you should be able to use git://github.com/openstack/pbr now that the upstream fix has landed. And you'll need to add pbr to each of the *-kilo.yaml files, placed after the requirements repo.

[Corey Bryant (corey.bryant)]

Sorry, it's not git://github.com/openstack/pbr . Rather it is git://github.com/openstack-dev/pbr .

[Edward Hope-Morley (hopem)]

looks like this went out with 16.01