Puppet does not detect that RabbitMQ is up in time

Bug #1615053 reported by Dmitry Mescheryakov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Invalid
High
Fuel Sustaining
Mitaka
Invalid
High
Fuel Sustaining

Bug Description

Version: 10.0

Steps to reproduce:
1. Deploy env consisting of 3 controllers and 3 computes

The deployment fails with message
AssertionError: Task 'deploy' has incorrect status. error != ready, 'Deployment has failed. All nodes are finished. Failed tasks: Task[primary-rabbitmq/3] Stopping the deployment process!'

In the puppet.log on the node-3 one can see that astute killed task while puppet was waiting for RabbitMQ master to come up: http://paste.openstack.org/show/561434/

At the same time on the reverted environment it could be clearly seen that RabbitMQ master is up, while Puppet is still trying to check that. After some time passes, puppet finally sees that RabbitMQ is up and successfully finishes (puppet timeout is 30 minutes, while astute's is 15 minutes, that is why puppet succeeds, while astute fails).

Attached is snapshot and 'cibadmin -Q' output before puppet detected that RabbitMQ master is up (cibadmin.before) and after that (cibadmin.after). That output is used by puppet to find out about RabbitMQ master.

Failed job: https://product-ci.infra.mirantis.net/view/10.0/job/10.0.main.ubuntu.bvt_2/563/console

Tags: area-library
Revision history for this message
Dmitry Mescheryakov (dmitrymex) wrote :
Revision history for this message
Dmitry Mescheryakov (dmitrymex) wrote :
description: updated
Revision history for this message
Dmitry Mescheryakov (dmitrymex) wrote :
Changed in fuel:
milestone: none → 10.0
tags: added: area-library
Changed in fuel:
status: New → Confirmed
importance: Undecided → High
assignee: nobody → Fuel Sustaining (fuel-sustaining-team)
Revision history for this message
Maksim Malchuk (mmalchuk) wrote :

related or even duplicate of the: https://bugs.launchpad.net/fuel/+bug/1606258

Revision history for this message
Kyrylo Galanov (kgalanov) wrote :

Is this issue still reproduced? It should be already fixed by multiple commits.

Changed in fuel:
status: Confirmed → Incomplete
Revision history for this message
Oleksiy Molchanov (omolchanov) wrote :

Marking as Invalid, because of no update for more than a month.

Changed in fuel:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.