kestone bootstrap fails if previous keystone running

Bug #1483259 reported by Vladislav Belogrudov
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
kolla
Invalid
Medium
Michał Jastrzębski

Bug Description

Sometimes keystone service is running on a bootstrapping host due to previous ansible attempts (failed/misconfigured or old service). Bootstrap container creates database but fails to create endpoints/users because it tries to bind temporary keystone to already bound port. Ansible should delete running keystone before bootstrapping if the last one is necessary

Changed in kolla:
assignee: nobody → Vladislav Belogrudov (vlad-belogrudov)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.openstack.org/211158

Changed in kolla:
status: New → In Progress
Revision history for this message
Steven Dake (sdake) wrote :

I think the proper way to handle this scenario is through a new integration point rather then just deleting in the current playbooks. The proposed patch exposes the scenario where a rerun of the playbooks would result in lost data without the operator's knowledge. This would be highly surprising.

Changed in kolla:
importance: Undecided → Medium
milestone: none → liberty-3
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on kolla (master)

Change abandoned by Vladislav Belogrudov (<email address hidden>) on branch: master
Review: https://review.openstack.org/211158
Reason: Too much control over failed task

Steven Dake (sdake)
Changed in kolla:
status: In Progress → Triaged
Revision history for this message
Vladislav Belogrudov (vlad-belogrudov) wrote :

I ran kestone initially and it ran OK but did not create anything due to 'su' problem - some kernels don't allow 'su' inside of a container (also the same can happen during table migration and many other possibilities - lack of space, memory,...). To reinit the service I had to delete user/database and run ansible playbook again.

Status from bootstrapping container is not tracked so far so normal keystones started after the former exited. If status of bootstrapping could be taken into account the problem would not occur.

Revision history for this message
Sam Yaple (s8m) wrote :

I believe we discussed this on IRC at one point.

If the bootstrap process failed initially and the container never properly bootstrapped, it is reasonable to require the operator to remove that database (that is empty) before continuing. We do _not_ want to accidentally start bootstrapping again.

Since this is only an issue that only occurs during an initial bootstrap, I believe this is best left to documentation. I am open for other suggestions.

Changed in kolla:
importance: Medium → Low
Revision history for this message
Vladislav Belogrudov (vlad-belogrudov) wrote :

agree, not a bug

Changed in kolla:
status: Triaged → Invalid
milestone: liberty-3 → none
Revision history for this message
Michał Jastrzębski (inc007) wrote :

I'm reoppening this bug as we will want to run bootstrap during upgrade, as we need to do db migrate prior to upgrade of containers

Changed in kolla:
status: Invalid → Triaged
importance: Low → High
importance: High → Medium
assignee: Vladislav Belogrudov (vlad-belogrudov) → Michał Jastrzębski (inc007)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to kolla (master)

Fix proposed to branch: master
Review: https://review.openstack.org/257567

Changed in kolla:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on kolla (master)

Change abandoned by Michal Jastrzebski (inc0) (<email address hidden>) on branch: master
Review: https://review.openstack.org/257567
Reason: it's done in playbooks already

Steven Dake (sdake)
Changed in kolla:
status: In Progress → Incomplete
Revision history for this message
Christian Berendt (berendt) wrote :

Related review was abandoned because it is already done in the playbooks. Closing as invalid.

Changed in kolla:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.