Promoter server is impacted by networking issues and is offline (running private instance presently)

Bug #1774220 reported by Arx Cruz
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
In Progress
Critical
Arx Cruz

Bug Description

The server that runs our promoter scripts are taking too long due networking performance issues that were causing the promoter to stall out when tagging images. This could be related to DNS but has not been fully root caused.

Since Friday evening (2018-05-25) we have been running a promoter instance on a machine managed by Matt Young (halcyondude) until we can resolve the issues in our tripleo-infra tenant of RDO Cloud.

This bug is to track the root cause and fix.

Tags: ci quickstart
Matt Young (halcyondude)
description: updated
Changed in tripleo:
importance: High → Critical
summary: - Promoter server is having DNS issues
+ Promoter server is impacted by networking issues and is offline (running
+ private instance presently)
Revision history for this message
Matt Young (halcyondude) wrote :

The promoter being used presently is running on sol.REDACTED.com.

The logs are exposed here:

http://sol.REDACTED.com/promoter_logs

Ruck/rover have creds to access. IMHO we need to resolve this as what we're doing now to keep promotions flowing is suboptimal and a stopgap at best.

Revision history for this message
Ronelle Landy (rlandy) wrote :

Should be resolved by:
    https://review.rdoproject.org/r/14060 (patch for above using supersede in /etc/dhclient.conf)

Changed in tripleo:
assignee: nobody → Arx Cruz (arxcruz)
milestone: none → rocky-2
status: Triaged → In Progress
Revision history for this message
Ronelle Landy (rlandy) wrote :

Promoter server is back on line

Changed in tripleo:
milestone: rocky-2 → rocky-3
Revision history for this message
Matt Young (halcyondude) wrote :

note details on how this was resolved are here: https://bugs.launchpad.net/tripleo/+bug/1770860/comments/4

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.