Deployment failed with connection to keystone endpoint error

[Timur Nurlygayanov]

Snapshot link (200+ Mb): https://copy.com/9t4yHwA2yL8dqndh

ISO custom iso MOS 5.1, should reproduced on vanilla MOS 5.1 too:
{"build_id": "2014-10-24_17-40-46", "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346", "build_number": "125", "auth_required": true, "api": "1.0", "nailgun_sha": "0acc6912f0613d358d8a2b5b274f1d228509fe31", "production": "docker", "fuelmain_sha": "8ef433e939425eabd1034c0b70e90bdf888b69fd", "astute_sha": "52f99f0c897a0e329431b5befa2987e33f28fbdb", "feature_groups": ["mirantis"], "release": "5.1", "release_versions": {"2014.1.1-5.1": {"VERSION": {"build_id": "2014-10-24_17-40-46", "ostf_sha": "64cb59c681658a7a55cc2c09d079072a41beb346", "build_number": "125", "api": "1.0", "nailgun_sha": "0acc6912f0613d358d8a2b5b274f1d228509fe31", "production": "docker", "fuelmain_sha": "8ef433e939425eabd1034c0b70e90bdf888b69fd", "astute_sha": "52f99f0c897a0e329431b5befa2987e33f28fbdb", "feature_groups": ["mirantis"], "release": "5.1", "fuellib_sha": "947eba8622baeb498aa260147a5a0128b666a0c9"}}}, "fuellib_sha": "947eba8622baeb498aa260147a5a0128b666a0c9"}

Steps To Reproduce:
1. Configure cloud with 3 controllers and 1 compute (virtual machines, 8Gb RAM on each VM), vCenter, Nova network, VMWare vCenter for Cinder, select 'install Ceilometer & Murano & Sahara'.
2. Start deployment.

Observed Result:
Deployment failed (please see snapshot https://copy.com/9t4yHwA2yL8dqndh), in logs we can see strange error like:
____________

2014-10-28T05:10:54 debug: [441] 2074eb2e-ad05-4acb-ac9c-beccb4921023: cmd: . /root/openrc && /usr/bin/glance index
                                               stdout:
                                               stderr: /usr/lib/python2.6/site-packages/eventlet/hubs/__init__.py:8: UserWarning: Module backports was already imported from /usr/lib64/python2.6/site-packages/backports/__init__.pyc, but /usr/lib/python2.6/site-packages is being added to sys.path
  import pkg_resources
Authorization Failed: Unable to establish connection to http://192.168.100.2:5000/v2.0/tokens

                                               exit code: 1
2014-10-28T05:10:54 err: [441] 2074eb2e-ad05-4acb-ac9c-beccb4921023: Disabling the upload of disk image because glance was not installed properly

____________

[Alexei Sheplyakov (asheplyakov)]

Apparently keystone database is not initialized properly:

[root@fuel ~]# dockerctl shell postgres /bin/sh -c 'cd /; echo \\dt | sudo -u postgres psql keystone'
No relations found.

Attempt to initialize it manually fails:

[root@fuel ~]# dockerctl shell keystone keystone-manage db_sync
Traceback (most recent call last):
  File "/usr/bin/keystone-manage", line 37, in <module>
    from keystone import cli
  File "/usr/lib/python2.6/site-packages/keystone/cli.py", line 20, in <module>
    import pbr.version
ImportError: No module named pbr.version

[Alexei Sheplyakov (asheplyakov)]

Installing the python-pbr package into the keystone container solves db initialization problem:

# dockerctl shell keystone yum -y install python-pbr

However keystone still fails to start:

# dockerctl shell keystone service openstack-keystone start
Starting keystone: /usr/bin/python: No module named keystone.openstack.common.systemd
                                                           [FAILED]

[Sergii Golovatiuk (sgolovatiuk)]

Hi,

We need to add package http://osci-obs.vm.mirantis.net:82/centos-fuel-5.0.3-stable-730/centos/noarch/ to fuel-repository. That's required as BVT build for CentOS is broken.

[Andrey Sledzinskiy (asledzinskiy)]

Is it going to be fixed for 5.0.3?
We have red bvt tests because of that

[Alexander Makarov (amakarov)]

Please update the status of the issue: I see all tests OK in http://jenkins-product.srt.mirantis.net:8080/view/5.0.3/job/5.0.3.centos.bvt_1/ for #57

[Andrey Sledzinskiy (asledzinskiy)]

Yes, issue is fixed now