RDO jobs failing with POST_FAILURE

Bug #1854517 reported by Marios Andreou on 2019-11-29
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tripleo
Critical
Unassigned

Bug Description

RDO jobs are failing with POST_FAILURE and there are no logs generated. There have been many examples today like at [1][2] and just now the train container-push job (periodic promotion) failed with the same at [3][4] rhel/centos so this also blocks promotions.

[1] https://review.rdoproject.org/r/23882
[2] https://review.rdoproject.org/r/#/c/23873
[3] https://review.rdoproject.org/zuul/build/f220f10c7bce4051ade3bdc6d4914fe5
[4] https://review.rdoproject.org/zuul/build/1af2d63ebd3e468db53fc995d54d319d

Marios Andreou (marios-b) wrote :

internal sf-ops folks are looking into that (via chandan earlier on irc) fbo/mhu are at least aware of the issue

Changed in tripleo:
status: New → Triaged
importance: Undecided → Critical
milestone: none → ussuri-1
tags: added: promotion-blocker
tags: added: ci
Marios Andreou (marios-b) wrote :

17:57 < jpena> raukadah: the ceph cluster it's running on is currently having I/O issues (some disks failed and it is rebuilding)
17:57 < raukadah> jpena: and reagrding rdo log server perf issue?
17:57 < raukadah> we have got hitten by post failures
17:57 < jpena> raukadah: same thing, it's running on the same ceph

tags: added: alert
chandan kumar (chkumar246) wrote :

https://review.rdoproject.org/zuul/builds?result=POST_FAILURE from this the post failure is gone, it was not on weekend, we can wait for today and close it.

Marios Andreou (marios-b) wrote :

ack looks like 29th was the last time we saw that so marking invalid (closest thing i could find to 'closed' :/) we can revisit if we see it again

Changed in tripleo:
status: Triaged → Invalid
Changed in tripleo:
status: Invalid → Triaged
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers