test_runner can fail while acquiring the console log when the test bed instance can't be created

Bug #1292461 reported by Vincent Ladeuil
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Ubuntu CI Engine
Fix Released
High
Vincent Ladeuil
Ubuntu CI Services
Fix Released
High
Vincent Ladeuil

Bug Description

adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ builds done.
adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ tests ...
adt-run: @@@@@@@@@@@@@@@@@@@@ test dsc dsc0
adt-run: @@@@@@@@@@ read control ...
adt-run: @@@@@@@@@@ run_tests ...
* SKIP no tests in this package
adt-run: @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@ tests done.
INFO:test_runner:saving subunit results for libgmp10
INFO:test_runner:saving testbed console}
INFO:urllib3.connectionpool:Resetting dropped connection: region-a.geo-1.compute.hpcloudsvc.com
INFO:ci_utils.amqp_worker:on_message: {"progress_trigger": "lander_master-6-testrunner", "state": "WAITING", "package_list": ["libgsl0ldbl", "libgmp10"], "image_id": ["4f271b1e-ab38-11e3-9e2b-fa163ead66bc.img", "https://region-a.geo-1.objects.hpcloudsvc.com/v1/11293005633044/ticket-5/new.img"], "cancel_url": "http://10.0.0.43:8080/job/lander_test_runner/2/api/json", "ticket_id": "5"}
INFO:urllib3.connectionpool:Starting new HTTPS connection (1): region-a.geo-1.identity.hpcloudsvc.com
INFO:urllib3.connectionpool:Starting new HTTPS connection (1): region-a.geo-1.identity.hpcloudsvc.com
INFO:urllib3.connectionpool:Starting new HTTPS connection (1): region-a.geo-1.identity.hpcloudsvc.com
INFO:urllib3.connectionpool:Starting new HTTPS connection (1): region-a.geo-1.compute.hpcloudsvc.com
INFO:test_runner:saving testbed console}
ERROR:test_runner:unable to upload testbed console
Traceback (most recent call last):
  File "./test_runner/tstrun/run_worker.py", line 53, in save_testbed_console
    console = test_bed.get_cloud_init_console()
  File "/srv/tr_rabbit_worker/test_runner/tstrun/testbed.py", line 242, in get_cloud_init_console
    return self.instance.get_console_output(length)
AttributeError: 'NoneType' object has no attribute 'get_console_output'
ERROR:test_runner:Unexpected exception occurred

We're trying to acquire the console after the test run, this can succeed if the testbed deletion is slow.

Tags: airline

Related branches

Vincent Ladeuil (vila)
summary: - test_runner can fail while acquiring the console log *after* the test
- succeeded
+ test_runner can fail while acquiring the console log when the test bed
+ instance can't be created
Revision history for this message
Vincent Ladeuil (vila) wrote :

I misread the log, only the last part is relevant, the part about the test succeeding is for a previous ticket.

Upon investigation it appeared that the image_id received was a list (instead of a scalar), due to some wip by Chris.

This led to a failed call to nova.servers.create() but the corresponding exception was masked by the attempt to upload the console (which didn't exist anyway in this case since there was no instance).

I'm re-targetting this bug at providing a more robust code and better error reporting in light of the above scenario (the error misled Chris instead of telling him what the problem was).

Andy Doan (doanac)
Changed in ubuntu-ci-services-itself:
importance: Critical → High
Vincent Ladeuil (vila)
Changed in ubuntu-ci-services-itself:
status: In Progress → Fix Committed
Vincent Ladeuil (vila)
Changed in ubuntu-ci-services-itself:
status: Fix Committed → Fix Released
Ursula Junque (ursinha)
Changed in uci-engine:
assignee: nobody → Vincent Ladeuil (vila)
importance: Undecided → High
milestone: none → phase-0
status: New → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.