unhandled PBR Exception "Versioning for this project requires either an sdist tarball" with older (<0.7.2) setuptools

[Zack M. Davis]

Recent versions (>=2.2.0) of swiftclient have been observed to die with the puzzling message "Versioning for this project requires either an sdist tarball, or access to an upstream git repository. Are you sure that git is installed?" E.g., the stack trace excerpt below is from a Django project I have (but, perhaps surprisingly, although this error prevents the site from functioning at all, I can successfully import swiftclient at the Python REPL):

```
  31. import swiftclient
File "/usr/local/lib/python2.7/dist-packages/swiftclient/__init__.py" in <module>
  20. from .client import *
File "/usr/local/lib/python2.7/dist-packages/swiftclient/client.py" in <module>
  34. from swiftclient import version as swiftclient_version
File "/usr/local/lib/python2.7/dist-packages/swiftclient/version.py" in <module>
  28. version_string = str(pbr.version.VersionInfo('python-swiftclient'))
File "/usr/local/lib/python2.7/dist-packages/pbr/version.py" in __str__
  39. return self.version_string()
File "/usr/local/lib/python2.7/dist-packages/pbr/version.py" in version_string
  78. for part in self.release_string().split('.'):
File "/usr/local/lib/python2.7/dist-packages/pbr/version.py" in release_string
  70. self.release = self._get_version_from_pkg_resources()
File "/usr/local/lib/python2.7/dist-packages/pbr/version.py" in _get_version_from_pkg_resources
  62. return packaging.get_version(self.package)
File "/usr/local/lib/python2.7/dist-packages/pbr/packaging.py" in get_version
  864. raise Exception("Versioning for this project requires either an sdist"

Exception Type: Exception at /
Exception Value: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. Are you sure that git is installed?
```

It turns out that this can be fixed by upgrading setuptools (thanks to notmyname in #openstack-swift on Freenode for help debugging this the other month), and binary-searching the versions of setuptools available to install via Pip, I observe that the error occurs with setuptools 0.6c9 but not setuptools 0.7.2.

This is perhaps not the most important bug, since it has such an easy fix ("pip install --upgrade setuptools"), but it is really confusing if you happen to have old Python packages and run into it.

[John Dickinson (notmyname)]

I'm not sure if this needs to be better messages in swiftclient or if something needs to be added to pbr.

[Jeremy Stanley (fungi)]

A very similar error was observed with jenkins-job-builder and setuptools 0.9.8 according to bug 1384238.

[Doug Hellmann (doug-hellmann)]

We can't really add a direct version-based requirement from pbr and setuptools, because setuptools is used to install pbr and you do not want setuptools to upgrade itself as a side-effect of installing a package (that tends to leave you with either a broken setuptools or none at all).

We could document that pbr needs the newest setuptools, but I'm not sure that's really going to help the situation. Maybe if we include this error message in the docs as well, that would make it easy to find via search? OTOH, this bug should also show up in search results for the same reason.

[Kenneth Koski (knkski)]

If anyone else runs into this issue and is unable to resolve it via upgrading setuptools, you can set the environment variable PBR_VERSION as a workaround. For example, this is how I run a setup.py script that was failing:

    PBR_VERSION=X.Y.Z python setup.py build

[Robert Collins (lifeless)]

@dhellman I think we can safely introspect the package database within pbr and bail out on too-old.

[[e-mail] (dave-dittrich)]

Actually, the reported "fix" that many people suggest, like installing distutils, seems to in reality be a manifestation of my all-time favorite logical fallacy, "Post Hoc, Ergo, Propter Hoc", which means "after this, therefore, because of this."

I believe that PBR reports the version of the program to which you gave the --version flag to be whatever version Git reports for the repo of the current working directory, not the program you installed with setup.py. In other words, the runtime determination of version is relatively to the runtime current working directory, not the as-installed program code.

Here is how I determined this (should be easy to replicate if you have a couple Git repos handy.)

$ cd /path/to/dimscli_source_repo
$ dimscli --version
dimscli 0.2.1
$ python setup.py --version
0.2.1.dev2
$ git describe
0.2-2-gf368f27

OK, that is what is expected.

$ cd
$ dimscli --version
Traceback (most recent call last):
File "/path/to/envs/dimscli/bin/dimscli", line 6, in <module>
from dimscli.shell import main
File "/path/to/git/python-dimscli/dimscli/__init__.py", line 20, in <module>
__version__ = version_info.version_string()
File "/path/to/envs/dimscli/lib/python2.7/site-packages/pbr/version.py", line 457, in version_string
return self.semantic_version().brief_string()
File "/path/to/envs/dimscli/lib/python2.7/site-packages/pbr/version.py", line 452, in semantic_version
self._semantic = self._get_version_from_pkg_resources()
File "/path/to/envs/dimscli/lib/python2.7/site-packages/pbr/version.py", line 439, in _get_version_from_pkg_resources
result_string = packaging.get_version(self.package)
File "/path/to/envs/dimscli/lib/python2.7/site-packages/pbr/packaging.py", line 668, in get_version
raise Exception("Versioning for this project requires either an sdist"
Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. Are you sure that git is installed?

OK, now I get the bug behavior (and upgrading distutils *did not* change the output.) But now...

$ cd /path/to/ansible-playbooks
$ dimscli --version
dimscli 1.2.1
$ cd /path/to/dims-devguide/
$ dimscli --version
dimscli 0.0.1

So as you can see, the "fix" is actually changing directory, not upgrading distutils.

Is there a way to fix the version in the installed site-packages directory for the main program? I've been looking and haven't found how to do that yet.

[[e-mail] (dave-dittrich)]

Oh, yeah. Forgot to add:

pbr==1.8.1
setuptools-19.4

[Joe D'Andrea (jdandrea)]

dave-dittrich: This explains why, when I would cd to within the git repo, "it just worked."

This doesn't help when I go to make a sdist tar.gz though. Is pbr just not compatible with PyPI?

[Joe D'Andrea (jdandrea)]

Also, FWIW, "pip install --upgrade setuptools" does not help, nor does "PBR_VERSION=X.Y.Z python setup.py sdist" (with an appropriate version string). I'm trying pbr 1.8.1 as well, to no avail.

Like dave-dittrich, I did notice that things "just work" if I'm somewhere in the git repo hierarchy.

[Joe D'Andrea (jdandrea)]

Pro tip: It helps if the package name in the call to pbr.version.VersionInfo() is up-to-date. *embarrassed* ... well, on the bright side, it now works without any need for PBR_VERSION=X.Y.Z!

[chanux (chanux)]

This is likely not the place to report this but I'll go on anyway.

I'm condemned to work in an enterprise environment that doesn't see the light of Internet and I'm trying to run the swift tests as ops runbook recommends. When I run setup.py install (latest master from both openstack/swift and openstack/os-testr) I get this issue. Passing PBR_VERSION=X.Y.Z puts an end to days of hair pulling.

However this is not the case on an environment blessed with Internet. So this is probably an issue just 3 people face and they are all cursed beyond help anyway :D.

[Stephen Finucane (stephenfinucane)]

setuptools 0.7.2 is basically pre-historic at this point, so I'm going to close this as a won't fix (sorry @chanux :)). If someone really cares about this, feel free to re-open (ideally with suggestions on how to fix it).