[Fuel CI] Fuel Jenkins is unavailable time to time.

Bug #1570322 reported by Andrey Nikitin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Andrey Nikitin

Bug Description

Hello!

Fuel Jenkins -- ci.fuel-infra.org -- is unavailable time to time (April, 13 - ~15.50; April, 14 - ~13.40).
I've found that all cores of CPU were utilized on 100 percent on the server on that time. Looks like, Jenkins was overloaded by any job or another activity.

Please investigate it from application side. We're going to help to find root cause of the problem, if you need our help.

Priority is High due to Fuel Jenkins is a production service and this problem affects many people.

Thanks in advance!

Tags: area-ci
Revision history for this message
Aleksandra Fedorova (bookwar) wrote :

There are no unusual activities on Jenkins side

Consider tuning hardware resources

Changed in fuel:
assignee: Fuel CI (fuel-ci) → Fuel DevOps (fuel-devops)
Changed in fuel:
status: New → Confirmed
Revision history for this message
Igor Shishkin (teran) wrote :

Andrey, please research if have any confirmations for such problems on atop side.
If yes - problem is about SW on the node, there also we could determine the pattern to track
Otherwise - it's mostly possible about networking so will require further investigation.

Changed in fuel:
assignee: Fuel DevOps (fuel-devops) → Andrey Nikitin (heos)
Andrey Nikitin (heos)
Changed in fuel:
status: Confirmed → In Progress
Revision history for this message
Andrey Nikitin (heos) wrote :

As far I can see, that all cores of the CPU were utilized on 100%, no other problems were found. Looks like, that problem occured due to errors in Postfix configuration, which were fixed.

Related messages in the logs:
Apr 13 12:55:44 fuel-jenkins01 postfix/smtp[17589]: connect to ASPMX.L.GOOGLE.com[2a00:1450:400c:c0b::1a]:25: Network is unreachable
Apr 13 12:55:44 fuel-jenkins01 postfix/smtp[17589]: connect to ALT1.ASPMX.L.GOOGLE.com[2a00:1450:4010:c09::1b]:25: Network is unreachable

Changed in fuel:
status: In Progress → Fix Committed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.