Currently all devstack jobs using a current version of git (focal is known to have released a version listed below) will fail to install repositories checked out locally due to a security fix[1] in git.
The error reported will look like this (but it is not limited to keystone):
2022-04-12 22:13:00.471769 | controller | Obtaining file:///opt/stack/keystone
2022-04-12 22:13:00.942756 | controller | ERROR: Command errored out with exit status 1:
2022-04-12 22:13:00.942912 | controller | command: /usr/bin/python3.8 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/opt/stack/keystone/setup.py'"'"'; __file__='"'"'/opt/stack/keystone/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info
2022-04-12 22:13:00.943022 | controller | cwd: /opt/stack/keystone/
2022-04-12 22:13:00.943097 | controller | Complete output (16 lines):
2022-04-12 22:13:00.943172 | controller | Error parsing
2022-04-12 22:13:00.943242 | controller | Traceback (most recent call last):
2022-04-12 22:13:00.943312 | controller | File "/usr/local/lib/python3.8/dist-packages/pbr/core.py", line 111, in pbr
2022-04-12 22:13:00.943381 | controller | attrs = util.cfg_to_args(path, dist.script_args)
2022-04-12 22:13:00.943493 | controller | File "/usr/local/lib/python3.8/dist-packages/pbr/util.py", line 272, in cfg_to_args
2022-04-12 22:13:00.943568 | controller | pbr.hooks.setup_hook(config)
2022-04-12 22:13:00.943637 | controller | File "/usr/local/lib/python3.8/dist-packages/pbr/hooks/__init__.py", line 25, in setup_hook
2022-04-12 22:13:00.943703 | controller | metadata_config.run()
2022-04-12 22:13:00.943772 | controller | File "/usr/local/lib/python3.8/dist-packages/pbr/hooks/base.py", line 27, in run
2022-04-12 22:13:00.943839 | controller | self.hook()
2022-04-12 22:13:00.943907 | controller | File "/usr/local/lib/python3.8/dist-packages/pbr/hooks/metadata.py", line 25, in hook
2022-04-12 22:13:00.943973 | controller | self.config['version'] = packaging.get_version(
2022-04-12 22:13:00.944041 | controller | File "/usr/local/lib/python3.8/dist-packages/pbr/packaging.py", line 872, in get_version
2022-04-12 22:13:00.944108 | controller | raise Exception("Versioning for this project requires either an sdist"
2022-04-12 22:13:00.944188 | controller | Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name keystone was given, but was not able to be found.
2022-04-12 22:13:00.944267 | controller | error in setup command: Error parsing /opt/stack/keystone/setup.cfg: Exception: Versioning for this project requires either an sdist tarball, or access to an upstream git repository. It's also possible that there is a mismatch between the package name in setup.cfg and the argument given to pbr.version.VersionInfo. Project name keystone was given, but was not able to be found.
2022-04-12 22:13:00.944346 | controller | ----------------------------------------
2022-04-12 22:13:00.944428 | controller | ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
The issue is that the repository /opt/stack/keystone is owned by stack:stack but installed under sudo which triggers this new security feature in git to not access the directory.
clarkb was able to reproduce the issue and chown -R /opt/stack/keystone to root allowed the installation.
On Ubuntu focal,
git 1:2.25.1-1ubuntu3.2 will run fine
git 1:2.25.1-1ubuntu3.3 will fail
Installing 3.2 in a test job showed a successful install here: https://review.opendev.org/c/openstack/devstack/+/837634
Related documentation in git can be found here: https://github.com/git/git/blob/master/Documentation/config/safe.txt
[1] https://github.com/git/git/commit/8959555cee7ec045958f9b6dd62e541affb7e7d9
https:/
is one potential solution; that chowns the checked-out directories as root:root. it may be a bit whack-a-mole finding places that want to fiddle with the source directories for various reasons.
using "git config" to mark the directories as "safe" is also an option. it doesn't feel like the type of thing we should be doing to a users machine -- but then again devstack takes over everything else anyway ...
Unfortunately, per the docs in the original commit, you can *not* just use "-c" to set one directory as safe from the git command-line. That basically rules out PBR being able to override this (the theory being, if you're telling PBR to install a source tree, then you trust that directory).
Other suggestion has been to install from a shallow clone. Some question over how PBR would cope with this.
The other approach is to completely ditch all this global install and use virtualenvs. Which I think we can all agree is probably a good idea, but has many implications for a) just getting that working in devstack and then b) bringing along every project and its plugins ...