Devstack setup failed due to the package of python-pcre isn't installed successfully

[Ke Xu]

While CI to setup the ENV by devstack, the devstacklog displayed:

running bdist_wheel
2018-01-26 09:47:38.079 | running build
2018-01-26 09:47:38.079 | running build_py
2018-01-26 09:47:38.079 | creating build
2018-01-26 09:47:38.079 | creating build/lib.linux-x86_64-2.7
2018-01-26 09:47:38.079 | copying python/pcre.py -> build/lib.linux-x86_64-2.7
2018-01-26 09:47:38.079 | running build_ext
2018-01-26 09:47:38.079 | building '_pcre' extension
2018-01-26 09:47:38.079 | creating build/temp.linux-x86_64-2.7
2018-01-26 09:47:38.079 | creating build/temp.linux-x86_64-2.7/src
2018-01-26 09:47:38.079 | x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c src/pcremodule.c -o build/temp.linux-x86_64-2.7/src/pcremodule.o -fno-strict-aliasing
2018-01-26 09:47:38.079 | src/pcremodule.c:32:18: fatal error: pcre.h: No such file or directory
2018-01-26 09:47:38.079 | compilation terminated.
2018-01-26 09:47:38.079 | error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

Apparently the package python-pcre was built failed. According to the patch https://review.openstack.org/#/c/522565/, libpcre3-dev should be installed automatically but I found it wasn't. Only when I installed the libpcre3-dev manually and the python-pcre can be built successfully. Then devstack would set up normally.

[Rui Chen (kiwik-chenrui)]

I face the same issue on Ubuntu 16.04 and latest devstack.

[Rui Chen (kiwik-chenrui)]

But i think the bug should not be report to cinder team.

[Rui Chen (kiwik-chenrui)]

Add some information:
    I use openstack-infra/devstack-gate to install local OpenStack deployment, like this:

```bash
cp devstack-gate/devstack-vm-gate-wrap.sh ./safe-devstack-vm-gate-wrap.sh
./safe-devstack-vm-gate-wrap.sh
```

If I use openstack-dev/devstack to execute ./stack.sh script, libpcre3-dev will be installed automatically.

[Ke Xu (nicolexu)]

Thanks Rui for your information. But I need to use devstack-gate to deploy my openstack since this is for CI.

[Rui Chen (kiwik-chenrui)]

Yes, me too. I use devstack-gate in our CI system, like you do. Currently I add a script to install libpcre3-dev before devstack start as workaround, it worked but ugly. I hope this issue can be fixed in upstream, so add some background information at here.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/538806

[nikeshkm (nike-niec)]

Kaminario CI is also failing due to this

[Jianghua Wang (wjh-fresh)]

Copying some investigation(was in bug/1745935 which is now marked as duplicated) result which may be useful for understanding this problem. JFYI:

When setup package for nova, it will also install the package from test-requirements.txt which has whereto (“whereto>=0.3.0 # Apache-2.0”);
2018-01-29 04:19:41.019 | +inc/python:_setup_package_with_constraints_edit:558 ^[[m^[(B /opt/stack/requirements/.venv/bin/edit-constraints /opt/stack/requirements/upper-constraints.txt -- nova '-e file:///opt/stack/nova#egg=nova'
2018-01-29 04:19:43.115 | +inc/python:pip_install:359 ^[[m^[(B sudo -H http_proxy= https_proxy= no_proxy= PIP_FIND_LINKS= SETUPTOOLS_SYS_PATH_TECHNIQUE=rewrite /usr/local/bin/pip2.7 install -c /opt/stack/requirements/upper-constraints.txt -r /opt/stack/nova/test-requirements.txt -e /opt/stack/nova

https://github.com/openstack-dev/devstack/blob/master/inc/python#L348

https://github.com/openstack/nova/blob/ec8b9a59c018c41bb18d0e712ad2868706ac9db7/test-requirements.txt#L40

And whereto requires python-pcre (http://git.openstack.org/cgit/openstack/whereto/tree/requirements.txt); but python-pcre depends on libpcre3-dev.

Due to the above dependence, python-pcre was added as upper-constraints by the following commit:
https://github.com/openstack/requirements/commit/89cebce27b7bd84260ea8e01a3fff1b64851e41d

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/538806
Committed: https://git.openstack.org/cgit/openstack-dev/devstack/commit/?id=744a829ce0b0a61a9b3be5afa8dfb4a258c69f11
Submitter: Zuul
Branch: master

commit 744a829ce0b0a61a9b3be5afa8dfb4a258c69f11
Author: shutingm <email address hidden>
Date: Sun Jan 28 23:06:40 2018 -0800

    Add the dependent package for python-pcre installation

    This patch add libpcre3-dev and pcre-devel package for python-pcre
    installation.

    Closes-Bug: #1745606

    Change-Id: I59fc688519341c90dc33b79d536f0625a6c4dd17