git.openstack.org/cgit should not be used to serve git directly

[Clint Byrum]

cgit will happilly serve (dumb) fetch requests from git. However, it occasionally loses track of commits, as evidenced in this (extremely large) tarball of the heat repo:

https://region-b.geo-1.objects.hpcloudsvc.com/v1/42841271810042/clint-fewbar/heat-missing-1bcd6625b44b70b1fddef4b7d46ccf32a7fa7a26.tar.xz

The remote there is:

https://git.openstack.org/cgit/openstack/heat

Which worked fine, until it didn't:

$ git fetch --verbose origin
error: Unable to find 5bfc1e2dade1be6ddfb581e9f99259e0e0b2b76e under https://git.openstack.org/cgit/openstack/heat
Cannot obtain needed tree 5bfc1e2dade1be6ddfb581e9f99259e0e0b2b76e
while processing commit 5235a3197a89f7c30d38bc7985cdc395e83a9d15.
error: Fetch failed.

Suggested remedy:

I would suggest that we should just use mod_rewrite to 301 any actual git requests like this to a base url of https://git.openstack.org/openstack/heat.git , which will let apache do the serving and result in working fetches.

[Logan Paschke (lkpaschke)]

I would like to help with this, although I don't know where to start.

[Bhuvan Arumugam (bhuvan)]

@jeblair, is it still an issue? fwiw, the problem don't seem to exist anymore:

rainbow:heat bhuvan$ git fetch --verbose origin
From https://git.openstack.org/cgit/openstack/heat
 = [up to date] master -> origin/master
 = [up to date] stable/grizzly -> origin/stable/grizzly
 = [up to date] stable/havana -> origin/stable/havana
rainbow:heat bhuvan$ git pull origin master
From https://git.openstack.org/cgit/openstack/heat
 * branch master -> FETCH_HEAD
Already up-to-date.
rainbow:heat bhuvan$ git remote -v
origin https://git.openstack.org/cgit/openstack/heat (fetch)
origin https://git.openstack.org/cgit/openstack/heat (push)

Moroever, the new url return 404.

[bhuvan@build stats]$ curl -w 'Response code: %{http_code}\n' https://git.openstack.org/openstack/heat.git
Response code: 404

[Jeremy Stanley (fungi)]

The more general issue isn't any one particular project, but rather that the cgit built-in git service seems to be less robust than what we're serving directly via Apache and so we should see if we can overlay, disable, or at least reproduce and document its issues. Users are unlikely to realize that cloning from http://git.openstack.org/openstack/nova.git and http://git.openstack.org/cgit/openstack/nova.git are coming from completely separate underlying services presentations of the same repository on disk.

[Chetan Mishra (chetan-s115)]

I assume this hasn't been fixed, as its been three years since the Assignee said anything.

Is the triaged solution still viable? I note that https://git.openstack.org/openstack/heat.git no longer is a valid location whereas https://git.openstack.org/cgit/openstack/heat/ is.

I didn't entirely understand what Jeremy meant by "we should see if we can overlay, disable, or at least reproduce and document its issues". Intuitively, that makes sense but doesn't seem to mesh with the solution recommended by the original poster "mod_rewrite to 301 any actual git requests like this to a base url of https://git.openstack.org/openstack/heat.git".

[Clint Byrum (clint-fewbar)]

Excerpts from Chetan Mishra's message of 2017-03-17 19:14:36 UTC:
> I assume this hasn't been fixed, as its been three years since the
> Assignee said anything.
>
> Is the triaged solution still viable? I note that
> https://git.openstack.org/openstack/heat.git no longer is a valid
> location whereas https://git.openstack.org/cgit/openstack/heat/ is.
>

Actually we want the cgit one to _not_ be viable, because that is the
one that has flaws.

> I didn't entirely understand what Jeremy meant by "we should see if we
> can overlay, disable, or at least reproduce and document its issues".
> Intuitively, that makes sense but doesn't seem to mesh with the solution
> recommended by the original poster "mod_rewrite to 301 any actual git
> requests like this to a base url of
> https://git.openstack.org/openstack/heat.git".
>

Yeah I think that's the ideal solution, though we could also just mark
this Won't Fix.