error when prechecks

Bug #1580368 reported by lebo
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
kolla
Invalid
High
lebo

Bug Description

I always get the error when run the "kolla-ansible -i ansible/inventory/multinode prechecks" command.

The error messages is as folows:

TASK: [prechecks | Checking free port for iscsi] ******************************
fatal: [compute01] => error while evaluating conditional: inventory_hostname in groups['tgtd']
fatal: [controller01] => error while evaluating conditional: inventory_hostname in groups['tgtd']
fatal: [compute02] => error while evaluating conditional: inventory_hostname in groups['tgtd']
fatal: [compute03] => error while evaluating conditional: inventory_hostname in groups['tgtd']
fatal: [controller02] => error while evaluating conditional: inventory_hostname in groups['tgtd']
fatal: [controller03] => error while evaluating conditional: inventory_hostname in groups['tgtd']

FATAL: all hosts have already failed -- aborting

PLAY RECAP ********************************************************************
           to retry, use: --limit @/root/prechecks.retry

compute01 : ok=1 changed=0 unreachable=1 failed=0
compute02 : ok=1 changed=0 unreachable=1 failed=0
compute03 : ok=1 changed=0 unreachable=1 failed=0
controller01 : ok=20 changed=0 unreachable=1 failed=0
controller02 : ok=20 changed=0 unreachable=1 failed=0
controller03 : ok=20 changed=0 unreachable=1 failed=0

--------------------------------------------------------------------------------------------------

3 hosts as controlers and 3 hosts as compute.The system is centos 7 and kernal is 3.10.0-327.el7.x86_64.

I tried 8 times to deplay the openstack by kolla,but always failed.

lebo (lebowebs)
tags: added: kolla openstack prechecks
Revision history for this message
Hui Kang (huikang27) wrote :

Do you have storage node in your inventory?

Do you mind sharing your inventory file so others can try reproducing the error?

Changed in kolla:
status: New → Incomplete
Revision history for this message
lebo (lebowebs) wrote :

hi,I finally solved this problem.
I replace the hostname by ipaddress in the file "ansible/inventory/multinode ".
then, precheks command runs ok.

But now,I encountered another problem.

-------------------------------------------------------------------------------------
TASK: [haproxy | Ensuring latest haproxy config is used] **********************
failed: [172.16.16.30] => {"changed": false, "cmd": ["docker", "exec", "haproxy", "/usr/local/bin/kolla_ensure_haproxy_latest_config"], "delta": "0:00:00.027372", "end": "2016-05-11 10:56:17.216540", "rc": 1, "start": "2016-05-11 10:56:17.189168", "stdout_lines": [], "warnings": []}
stderr: Error response from daemon: Container 092b17d868248d52a950616b7533e155b55b6e9942c36e5a0e28d75efa7b3e7d is restarting, wait until the container is running
failed: [172.16.16.32] => {"changed": false, "cmd": ["docker", "exec", "haproxy", "/usr/local/bin/kolla_ensure_haproxy_latest_config"], "delta": "0:00:00.026664", "end": "2016-05-11 10:56:14.785294", "rc": 1, "start": "2016-05-11 10:56:14.758630", "stdout_lines": [], "warnings": []}
stderr: Error response from daemon: Container 718092627d3e8e183b89a756e51aa434ada34bacee92540dcf6d0e29b76a8627 is restarting, wait until the container is running
failed: [172.16.16.31] => {"changed": false, "cmd": ["docker", "exec", "haproxy", "/usr/local/bin/kolla_ensure_haproxy_latest_config"], "delta": "0:00:00.025538", "end": "2016-05-11 10:56:18.731540", "rc": 1, "start": "2016-05-11 10:56:18.706002", "stdout_lines": [], "warnings": []}
stderr: Error response from daemon: Container 563a5470c7d859099d197e63535ae7056bd663242be61d6c669b620bed0e340d is restarting, wait until the container is running

FATAL: all hosts have already failed -- aborting

---------------------------------------------------------------------------

and this messages appeared when deploy

Revision history for this message
lebo (lebowebs) wrote :

now the ansible/inventory/multinode file content:

[control]
172.16.16.30
172.16.16.31
172.16.16.32

[compute]
172.16.16.30
172.16.16.31
172.16.16.32
172.16.16.26
172.16.16.27
172.16.16.28

[storage]
172.16.16.30
172.16.16.31
172.16.16.32
172.16.16.26
172.16.16.27
172.16.16.28

[kibana:children]
control
[elasticsearch:children]
control
[haproxy:children]
control
[mariadb:children]
control
[rabbitmq:children]
control
[mongodb:children]
control
[keystone:children]
control
[glance:children]
control
[nova:children]
control
[neutron:children]
control
[cinder:children]
control
[memcached:children]
control
[horizon:children]
control
[swift:children]
control
[heat:children]
control
[murano:children]
control
[ironic:children]
control
[ceph-mon:children]
control
[ceph-rgw:children]
control
[ceph-osd:children]
storage
[magnum:children]
control
[mistral:children]
control
[manila:children]
control
[glance-api:children]
glance
[glance-registry:children]
glance
[nova-api:children]
nova
[nova-conductor:children]
nova
[nova-consoleauth:children]
nova
[nova-novncproxy:children]
nova
[nova-scheduler:children]
nova
[nova-spicehtml5proxy:children]
nova
[nova-compute-ironic:children]
nova
[neutron-server:children]
control
[neutron-dhcp-agent:children]
neutron
[neutron-l3-agent:children]
neutron
[neutron-metadata-agent:children]
neutron
[cinder-api:children]
cinder
[cinder-backup:children]
control
[cinder-scheduler:children]
cinder
[cinder-volume:children]
control
[manila-api:children]
manila
[manila-scheduler:children]
manila
[manila-share:children]
storage
[swift-proxy-server:children]
swift
[swift-account-server:children]
storage
[swift-container-server:children]
storage
[swift-object-server:children]
storage
[heat-api:children]
heat
[heat-api-cfn:children]
heat
[heat-engine:children]
heat
[murano-api:children]
murano
[murano-engine:children]
murano
[ironic-api:children]
ironic
[ironic-conductor:children]
ironic
[ironic-inspector:children]
ironic
[ironic-pxe:children]
ironic
[magnum-api:children]
magnum
[magnum-conductor:children]
magnum
[mistral-api:children]
mistral
[mistral-executor:children]
mistral
[mistral-engine:children]
mistral

Revision history for this message
lebo (lebowebs) wrote :

I found this container:
092b17d868248d52a950616b7533e155b55b6e9942c36e5a0e28d75efa7b3e7d 718092627d3e8e183b89a756e51aa434ada34bacee92540dcf6d0e29b76a8627 563a5470c7d859099d197e63535ae7056bd663242be61d6c669b620bed0e340d ,

the status is always Restarting (0) 20 minutes ago.

And i want to start it manual but failed too.

Revision history for this message
lebo (lebowebs) wrote :

The container logs :

[WARNING] 131/161137 (1) : config : 'option forwardfor' ignored for proxy 'mariadb' as it requires HTTP mode.
INFO:__main__:Kolla config strategy set to: COPY_ALWAYS
INFO:__main__:Loading config file at /var/lib/kolla/config_files/config.json
INFO:__main__:Validating config file
INFO:__main__:Copying service configuration files
INFO:__main__:Removing existing destination: /etc/haproxy/haproxy.cfg
INFO:__main__:Copying /var/lib/kolla/config_files/haproxy.cfg to /etc/haproxy/haproxy.cfg
INFO:__main__:Setting permissions for /etc/haproxy/haproxy.cfg
INFO:__main__:/var/lib/kolla/config_files/haproxy.pem does not exist, but is not required
INFO:__main__:Writing out command to execute
Running command: '/usr/sbin/haproxy -f /etc/haproxy/haproxy.cfg -p /run/haproxy.pid'
[WARNING] 131/161320 (1) : config : 'option forwardfor' ignored for proxy 'mariadb' as it requires HTTP mode.

lebo (lebowebs)
Changed in kolla:
assignee: nobody → lebo (lebowebs)
status: Incomplete → Fix Committed
Revision history for this message
Rahul Sharma (rahulsharmaait) wrote :

Is there any gerrit link where I can see the committed fix? I am hitting something similar and want to see if my code contains the fix or not.

Thanks.

Changed in kolla:
status: Fix Committed → Triaged
importance: Undecided → High
milestone: none → newton-3
Changed in kolla:
milestone: newton-3 → newton-rc1
Changed in kolla:
milestone: newton-rc1 → newton-rc2
Revision history for this message
Paul Bourke (pauldbourke) wrote :

The original cause for this bug was a faulty inventory (missing the tgtd group). It seems like OP has solved this in their setup and I can confirm the inventory is correct in the codebase. If there are further problems around restarting containers and the like please open a new bug.

Changed in kolla:
status: Triaged → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.