barbican_api fails to deploy

Bug #1634010 reported by bjolo
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
kolla
Fix Released
High
Magnus Lööf
Ocata
Fix Released
Undecided
Unassigned
Pike
Fix Released
High
Magnus Lööf

Bug Description

env: kolla stable/newton branch; centos binary

deploy is successfull, but docker ps shows barbican_api stuck in restarting.

from the logs

docker logs barbican_api
Running command: 'uwsgi --master --emperor /etc/barbican/vassals --logto /var/log/kolla/barbican/barbican-api.log'
/usr/local/bin/kolla_start: line 24: exec: uwsgi: not found
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Copying service configuration files
INFO:__main__:Removing existing destination: /etc/barbican/barbican.conf
INFO:__main__:Copying /var/lib/kolla/config_files/barbican.conf to /etc/barbican/barbican.conf
INFO:__main__:Setting permissions for /etc/barbican/barbican.conf
INFO:__main__:Removing existing destination: /etc/barbican/vassals/barbican-api.ini
INFO:__main__:Copying /var/lib/kolla/config_files/vassals/barbican-api.ini to /etc/barbican/vassals/barbican-api.ini
INFO:__main__:Setting permissions for /etc/barbican/vassals/barbican-api.ini
INFO:__main__:Removing existing destination: /etc/barbican/barbican-api-paste.ini
INFO:__main__:Copying /var/lib/kolla/config_files/barbican-api-paste.ini to /etc/barbican/barbican-api-paste.ini
INFO:__main__:Setting permissions for /etc/barbican/barbican-api-paste.ini
INFO:__main__:Writing out command to execute
Running command: 'uwsgi --master --emperor /etc/barbican/vassals --logto /var/log/kolla/barbican/barbican-api.log'
/usr/local/bin/kolla_start: line 24: exec: uwsgi: not found

Changed in kolla:
status: New → Confirmed
importance: Undecided → High
status: Confirmed → Triaged
Revision history for this message
Chris Liles (christopherliles) wrote :

A simple workaround is to add 'uwsgi' to barbican_api_packages_append and rebuild.

Revision history for this message
Jeffrey Zhang (jeffrey4l) wrote :

This is a bug need to be fix.

tags: added: low-hanging-fruit
Revision history for this message
Jeffrey Zhang (jeffrey4l) wrote :

the uwsgi is installed. I tested this in master, it works.

Changed in kolla:
status: Triaged → Incomplete
Revision history for this message
Christian Berendt (berendt) wrote :

Works on master, closing as invalid.

tags: removed: low-hanging-fruit
Changed in kolla:
status: Incomplete → Invalid
Revision history for this message
Magnus Lööf (magnus-loof) wrote :

I am on Kolla 3.0.2, binary centos release.

If I run:

$ docker run --rm -ti kolla/centos-binary-barbican-api:3.0.2 /bin/bash
()[barbican@61a7fb825779 /]$ yum list installed | grep uwsgi

uwsgi is not installed, and it exibits the same error as above.

Revision history for this message
Magnus Lööf (magnus-loof) wrote :

And I see now in https://github.com/openstack/kolla/blob/master/docker/barbican/barbican-base/Dockerfile.j2 it is installed by pip.

But:

$ docker run --rm -ti kolla/centos-binary-barbican-api:3.0.2 /bin/bash
()[barbican@61a7fb825779 /]$ pip freeze | grep uwsgi

It is not installed.

Revision history for this message
Magnus Lööf (magnus-loof) wrote :

and:

()[barbican@61a7fb825779 /]$ pip install uwsgi
Collecting uwsgi
  Downloading uwsgi-2.0.14.tar.gz (788kB)
    100% |################################| 798kB 822kB/s
Installing collected packages: uwsgi
  Running setup.py install for uwsgi ... error
    Complete output from command /usr/bin/python2 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-08bqs1/uwsgi/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-vVPOCb-record/install-record.txt --single-version-externally-managed --compile:
    running install
    using profile: buildconf/default.ini
    detected include path: ['/usr/include', '/usr/local/include']
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-build-08bqs1/uwsgi/setup.py", line 124, in <module>
        distclass=uWSGIDistribution,
      File "/usr/lib64/python2.7/distutils/core.py", line 152, in setup
        dist.run_commands()
      File "/usr/lib64/python2.7/distutils/dist.py", line 953, in run_commands
        self.run_command(cmd)
      File "/usr/lib64/python2.7/distutils/dist.py", line 972, in run_command
        cmd_obj.run()
      File "/tmp/pip-build-08bqs1/uwsgi/setup.py", line 77, in run
        conf = uc.uConf(get_profile())
      File "uwsgiconfig.py", line 742, in __init__
        raise Exception("you need a C compiler to build uWSGI")
    Exception: you need a C compiler to build uWSGI

Revision history for this message
Jeffrey Zhang (jeffrey4l) wrote :

@magnus-loof, could u fix the newton branch?

Revision history for this message
Magnus Lööf (magnus-loof) wrote :

Some more investigation into this:

in docker/barbican/barbican-api/Dockerfile.js one can see that for ubuntu, the `package uwsgi-plugin-python` is installed in addition to `barbican-api`. For centos, the only package that is installed is `openstack-barbican-api`

Conclusion: `uwsgi-plugin-python` should be installed for Centos as well.

pip is only used to install uwsgi if it is built from source - as can be seen in barbican-base/Dockerfile.js - and this problem is related to binary release.

Changed in kolla:
assignee: nobody → Magnus Lööf (magnus-loof)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/435687

Changed in kolla:
status: Invalid → Confirmed
milestone: none → 3.0.3
Revision history for this message
Magnus Lööf (magnus-loof) wrote :

I have examined the source code for master, as well as building `kolla/centos-binary-barbican-api` from master.

```
[root@08a168a921e6 src]# git show head
commit f0d272b8aa1257c56537f5900623b795e474dafc
Merge: 9890fb2 ffdfac2
Author: Jenkins <email address hidden>
Date: Sat Feb 18 01:46:09 2017 +0000

    Merge "build: sort list of built/failed images before printing"

[root@08a168a921e6 src]#
```

The problem with uwsgi in binary centos containers exists even in master.

Changed in kolla:
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.openstack.org/435776

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (master)

Reviewed: https://review.openstack.org/435776
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=df2e5d5e23030de57824c4d46df7af1db462c511
Submitter: Jenkins
Branch: master

commit df2e5d5e23030de57824c4d46df7af1db462c511
Author: Magnus Lööf <email address hidden>
Date: Sat Feb 18 10:26:04 2017 +0100

    Install uwsgi in Centos binary

    Installs `uwsgi-plugin-python` and dependencies in `barbican-api` image.

    This will resolve problems bootstrapping barbican-api with binary Centos
    images which otherwise fail to start due to:

    ```
    Running command: 'uwsgi --master --emperor /etc/barbican/vassals --logto /var/log/kolla/barbican/barbican-api.log'
    /usr/local/bin/kolla_start: line 24: exec: uwsgi: not found
    ```

    Tested with:

    ```
    docker run --rm -ti --entrypoint=/bin/bash
    kolla/centos-binary-barbican-api:3.0.3
    $ uwsgi

    *** Starting uWSGI 2.0.14 (64bit) on [Sat Feb 18 09:32:12 2017] ***
    compiled with version: 4.8.5 20150623 (Red Hat 4.8.5-11) on 13 January 2017 23:06:53
    os: Linux-4.9.4-moby #1 SMP Wed Jan 18 17:04:43 UTC 2017
    nodename: b9063757e276
    machine: x86_64
    clock source: unix
    pcre jit disabled

    ...
    ```

    Closes-Bug: 1634010
    Signed-off-by: Magnus Lööf <email address hidden>
    Change-Id: I77613bbeb19d89e6a25e52f24f3275d309ceb032

Changed in kolla:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (stable/ocata)

Fix proposed to branch: stable/ocata
Review: https://review.openstack.org/443253

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (stable/ocata)

Reviewed: https://review.openstack.org/443253
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=8cae6ef8602d952e9212d2bdf2b128da65e18c8e
Submitter: Jenkins
Branch: stable/ocata

commit 8cae6ef8602d952e9212d2bdf2b128da65e18c8e
Author: Magnus Lööf <email address hidden>
Date: Sat Feb 18 10:26:04 2017 +0100

    Install uwsgi in Centos binary

    Installs `uwsgi-plugin-python` and dependencies in `barbican-api` image.

    This will resolve problems bootstrapping barbican-api with binary Centos
    images which otherwise fail to start due to:

    ```
    Running command: 'uwsgi --master --emperor /etc/barbican/vassals --logto /var/log/kolla/barbican/barbican-api.log'
    /usr/local/bin/kolla_start: line 24: exec: uwsgi: not found
    ```

    Tested with:

    ```
    docker run --rm -ti --entrypoint=/bin/bash
    kolla/centos-binary-barbican-api:3.0.3
    $ uwsgi

    *** Starting uWSGI 2.0.14 (64bit) on [Sat Feb 18 09:32:12 2017] ***
    compiled with version: 4.8.5 20150623 (Red Hat 4.8.5-11) on 13 January 2017 23:06:53
    os: Linux-4.9.4-moby #1 SMP Wed Jan 18 17:04:43 UTC 2017
    nodename: b9063757e276
    machine: x86_64
    clock source: unix
    pcre jit disabled

    ...
    ```

    Closes-Bug: 1634010
    Signed-off-by: Magnus Lööf <email address hidden>
    Change-Id: I77613bbeb19d89e6a25e52f24f3275d309ceb032
    (cherry picked from commit df2e5d5e23030de57824c4d46df7af1db462c511)

Revision history for this message
Magnus Lööf (magnus-loof) wrote :

Metadata for this issue seems incorrect.

Suggest changing

Affects Pike -> Affects Newton

Importance -> High for Ocata Release

Milestone -> Ocata 4.0.1 for Ocata Release

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to kolla (stable/newton)

Reviewed: https://review.openstack.org/435687
Committed: https://git.openstack.org/cgit/openstack/kolla/commit/?id=9c46a15e6472b531fcc658dcdfb13c97ff52c843
Submitter: Jenkins
Branch: stable/newton

commit 9c46a15e6472b531fcc658dcdfb13c97ff52c843
Author: Magnus Lööf <email address hidden>
Date: Sat Feb 18 10:26:04 2017 +0100

    Install uwsgi in Centos binary

    Installs `uwsgi-plugin-python` and dependencies in `barbican-api` image.

    This will resolve problems bootstrapping barbican-api with binary Centos
    images which otherwise fail to start due to:

    ```
    Running command: 'uwsgi --master --emperor /etc/barbican/vassals --logto /var/log/kolla/barbican/barbican-api.log'
    /usr/local/bin/kolla_start: line 24: exec: uwsgi: not found
    ```

    Tested with:

    ```
    docker run --rm -ti --entrypoint=/bin/bash
    kolla/centos-binary-barbican-api:3.0.3
    $ uwsgi

    *** Starting uWSGI 2.0.14 (64bit) on [Sat Feb 18 09:32:12 2017] ***
    compiled with version: 4.8.5 20150623 (Red Hat 4.8.5-11) on 13 January 2017 23:06:53
    os: Linux-4.9.4-moby #1 SMP Wed Jan 18 17:04:43 UTC 2017
    nodename: b9063757e276
    machine: x86_64
    clock source: unix
    pcre jit disabled

    ...
    ```

    Closes-Bug: 1634010
    Signed-off-by: Magnus Lööf <email address hidden>
    Change-Id: I77613bbeb19d89e6a25e52f24f3275d309ceb032

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla 4.0.1

This issue was fixed in the openstack/kolla 4.0.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/kolla 5.0.0.0b2

This issue was fixed in the openstack/kolla 5.0.0.0b2 development milestone.

Revision history for this message
Yuvaraja Mariappan (ymariappan) wrote :

Hi,

where can i find the docker image for 4.0.1 release?
would it be fixed in 4.0.0?

Thanks,
Yuvaraja

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.