EC2 builds pile-up due to lack of instance slots
Bug #1021191 reported by
Paul Sokolovsky
This bug affects 1 person
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Linaro AWS Tools |
Fix Released
|
Critical
|
Paul Sokolovsky |
Bug Description
This (EU) morning we had a case of pretty big pending builds pile-up in Jenkins service: ~15 build on ci.linaro.org, ~5 builds on android-build. Both hosts are set at 28 instance cap.
On a quick look, there were 6 cbuild instances running at that time (most for more than a day) which took bunch of lots. On closer look, we also have few undocumented long-running instances (some for more than a month), which account for other slots. More detailed info to be provided below.
Changed in linaro-aws-tools: | |
status: | Confirmed → Triaged |
summary: |
- EC2 build pile-up + EC2 builds pile-up due to lack of instance slots |
To post a comment you must log in.
Output of ./check- aws-resources :
FAIL: test_no_ unknown_ old_instance (__main_ _.TestEC2Instan ces) ------- ------- ------- ------- ------- ------- ------- ------- ------- aws-resources" , line 527, in test_no_ unknown_ old_instance assertEqual( [], no_match) i-91eee8f7, Instance...
-------
Traceback (most recent call last):
File "./check-
self.
AssertionError: Lists differ: [] != [Instance:
Second list contains 8 additional elements.
First extra element 0:
Instance:i-91eee8f7
- [] i-91eee8f7, i-6bf7f10d, i-1f150379, i-91cddbf7, i-dae678a3, i-fcc8ad85, i-aeddb8d7, i-18dcba61]
+ [Instance:
+ Instance:
+ Instance:
+ Instance:
+ Instance:
+ Instance:
+ Instance:
+ Instance: