CI rdocloud 295 orphaned nodes have exhausted tenant resources

Bug #1728135 reported by Gabriele Cerami on 2017-10-27
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Critical
Gabriele Cerami

Bug Description

During the rdocloud nodes shortage yesterday, te-broker lost contact with it, and the 295 instances running were left without control, with nothing able to delete them.
When te-broker reestablished contact this morning, it was too late, there is no other choice than to delete 295 nodes by hand to free resources.

Gabriele Cerami (gcerami) wrote :

Deleted most of the stacks. There are some leftovers we need to look at more carefully, but there's no emergency anymore, resources from 300 instances freed.

Changed in tripleo:
status: In Progress → Fix Released
tags: removed: alert
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers