[pike][ocata][24 hrs][periodic]CREATE_FAILED ResourceInError: resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. , Code: 500"

[chandan kumar]

Pike and Ocata 24 hrs openstack-periodic jobs are failing at overcloud deploy by giving following errors:
2018-07-10 06:06:05 | 2018-07-10 06:05:35Z [overcloud.Controller.0.Controller]: CREATE_FAILED ResourceInError: resources.Controller: Went to status ERROR due to "Message: No valid host was found. , Code: 500"
2018-07-10 06:06:05 | 2018-07-10 06:05:35Z [overcloud.Controller.0.Controller]: DELETE_IN_PROGRESS state changed
2018-07-10 06:06:05 | 2018-07-10 06:05:37Z [overcloud.Controller.0.Controller]: DELETE_COMPLETE state changed
2018-07-10 06:06:05 | 2018-07-10 06:05:52Z [overcloud.CephStorage.0.CephStorage]: CREATE_IN_PROGRESS state changed
2018-07-10 06:06:05 | 2018-07-10 06:05:53Z [overcloud.CephStorage.0.CephStorage]: CREATE_FAILED ResourceInError: resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. , Code: 500"
2018-07-10 06:06:05 | 2018-07-10 06:05:54Z [overcloud.Controller.0.Controller]: CREATE_IN_PROGRESS state changed
2018-07-10 06:06:05 | 2018-07-10 06:05:54Z [overcloud.CephStorage.0]: CREATE_FAILED Resource CREATE failed: ResourceInError: resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
2018-07-10 06:06:05 | 2018-07-10 06:05:55Z [overcloud.CephStorage.0]: CREATE_FAILED ResourceInError: resources[0].resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
2018-07-10 06:06:05 | 2018-07-10 06:05:55Z [overcloud.CephStorage]: UPDATE_FAILED ResourceInError: resources[0].resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
2018-07-10 06:06:05 | 2018-07-10 06:05:55Z [0.NovaCompute]: CREATE_IN_PROGRESS state changed
2018-07-10 06:06:05 | 2018-07-10 06:05:55Z [overcloud.CephStorage]: CREATE_FAILED resources.CephStorage: ResourceInError: resources[0].resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
2018-07-10 06:06:05 | 2018-07-10 06:05:55Z [overcloud.Compute]: CREATE_FAILED CREATE aborted
2018-07-10 06:06:05 | 2018-07-10 06:05:55Z [overcloud.Controller]: CREATE_FAILED CREATE aborted
2018-07-10 06:06:05 | 2018-07-10 06:05:55Z [overcloud]: CREATE_FAILED Resource CREATE failed: resources.CephStorage: ResourceInError: resources[0].resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. There are not enough hosts available., Code: 500"
2018-07-10 06:06:05 | 2018-07-10 06:05:56Z [overcloud.Controller.0.Controller]: CREATE_FAILED ResourceInError: resources.Controller: Went to status ERROR due to "Message: No valid host was found. , Code: 500"
2018-07-10 06:06:05 | 2018-07-10 06:05:56Z [overcloud.Controller.0.Controller]: DELETE_IN_PROGRESS state changed
2018-07-10 06:06:05 | 2018-07-10 06:05:56Z [overcloud.Controller.0]: CREATE_FAILED CREATE aborted
2018-07-10 06:06:05 | 2018-07-10 06:05:56Z [overcloud.Controller]: UPDATE_FAILED Operation cancelled
2018-07-10 06:06:05 | 2018-07-10 06:05:56Z [0]: CREATE_FAILED CREATE aborted
2018-07-10 06:06:05 | 2018-07-10 06:05:56Z [overcloud.Compute]: UPDATE_FAILED Operation cancelled
2018-07-10 06:06:05 | 2018-07-10 06:05:58Z [overcloud.Compute.0.NovaCompute]: CREATE_FAILED ResourceInError: resources.NovaCompute: Went to status ERROR due to "Message: No valid host was found. , Code: 500"
2018-07-10 06:06:05 | 2018-07-10 06:05:58Z [overcloud.Controller.0.Controller]: DELETE_COMPLETE state changed
2018-07-10 06:06:05 | 2018-07-10 06:05:59Z [overcloud.Compute.0]: CREATE_FAILED Resource CREATE failed: Operation cancelled
2018-07-10 06:06:05 |
2018-07-10 06:06:05 | Stack overcloud CREATE_FAILED
2018-07-10 06:06:05 |
2018-07-10 06:06:05 | overcloud.Controller.0:
2018-07-10 06:06:05 | resource_type: OS::TripleO::Controller
2018-07-10 06:06:05 | physical_resource_id: 0e6045df-747a-4d87-bb46-5c65e7b86944
2018-07-10 06:06:05 | status: CREATE_FAILED
2018-07-10 06:06:05 | status_reason: |
2018-07-10 06:06:05 | CREATE aborted
2018-07-10 06:06:05 | overcloud.Compute.0.NovaCompute:
2018-07-10 06:06:05 | resource_type: OS::TripleO::ComputeServer
2018-07-10 06:06:05 | physical_resource_id: e8902a0d-72df-474d-9ef1-99603a773ff4
2018-07-10 06:06:05 | status: CREATE_FAILED
2018-07-10 06:06:05 | status_reason: |
2018-07-10 06:06:05 | ResourceInError: resources.NovaCompute: Went to status ERROR due to "Message: No valid host was found. , Code: 500"
2018-07-10 06:06:05 | overcloud.CephStorage.0.CephStorage:
2018-07-10 06:06:05 | resource_type: OS::TripleO::CephStorageServer
2018-07-10 06:06:05 | physical_resource_id: 850308c9-a5c4-4ba1-817b-d54e894b126c
2018-07-10 06:06:05 | status: CREATE_FAILED
2018-07-10 06:06:05 | status_reason: |
2018-07-10 06:06:05 | ResourceInError: resources.CephStorage: Went to status ERROR due to "Message: No valid host was found. , Code: 500"
2018-07-10 06:06:05 | + status_code=1
2018-07-10 06:06:05 | + openstack stack list
2018-07-10 06:06:05 | + grep -q overcloud
2018-07-10 06:06:08 | + openstack stack list
2018-07-10 06:06:08 | + grep -Eq '(CREATE|UPDATE)_COMPLETE'
2018-07-10 06:06:11 | + openstack stack failures list overcloud --long
2018-07-10 06:06:18 | + openstack overcloud failures
2018-07-10 06:06:19 | + true
2018-07-10 06:06:19 | ++ openstack stack resource list --nested-depth 5 overcloud
2018-07-10 06:06:19 | ++ grep FAILED
2018-07-10 06:06:19 | ++ grep 'StructuredDeployment '
2018-07-10 06:06:19 | ++ cut -d '|' -f3
2018-07-10 06:06:31 | + exit 1

Below is the logs:
http://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp-featureset020-ocata/194f438/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz

http://logs.rdoproject.org/openstack-periodic-24hr/git.openstack.org/openstack-infra/tripleo-ci/master/legacy-periodic-tripleo-ci-centos-7-ovb-1ctlr_1comp_1ceph-featureset024-pike/6b2de67/logs/undercloud/home/zuul/overcloud_deploy.log.txt.gz

[Sagi (Sergey) Shnaidman (sshnaidm)]

Doesn't happen again