Test Runner returns a success too quickly, did it run tests

[Francis Ginther]

While manual testing, the test-runner returns a SUCCESS very quickly. If the testing was skipped, do we know why?

[jenkins log]
Started by upstream project "lander_master" build number 1
originally caused by:
 Started by remote host 10.55.32.145
Building in workspace /var/lib/jenkins/jobs/lander_test_runner/workspace
[workspace] $ /bin/bash /tmp/hudson7924327328721655416.sh
+ '[' -d results ']'
+ mkdir results
+ echo '{"bsbuilder":' '{"status":' '{"building":' '"set([])",' '"failed":' '"set([])",' '"pending":' '"set([])"},' '"source_packages":' '["https://swift.canonistack.canonical.com/v1/AUTH_a1412d7034ce49b2a980b84b15c04372/ticket-1//home/fginther/work/engine/ppa-test/lintian4python/lintian4python_0.24.4ubuntu1~fginther.20140226.1_source.changes",' '"https://swift.canonistack.canonical.com/v1/AUTH_a1412d7034ce49b2a980b84b15c04372/ticket-1//home/fginther/work/engine/ppa-test/lintian4python/lintian4python_0.24.4ubuntu1~fginther.20140226.1.tar.gz",' '"https://swift.canonistack.canonical.com/v1/AUTH_a1412d7034ce49b2a980b84b15c04372/ticket-1//home/fginther/work/engine/ppa-test/lintian4python/lintian4python_0.24.4ubuntu1~fginther.20140226.1.dsc"],' '"artifacts":' '[],' '"ppa":' '"ppa:fginther/ci-pool-001",' '"state":' '"COMPLETED",' '"exit":' true, '"result":' '"PASSED"},' '"master":' '{"ppa_assigner_url":' '"http://10.55.32.150:8080",' '"progress_trigger":' '"lander_master-1",' '"imgbuilder_url":' '"http://10.55.32.120:8080",' '"tr_url":' '"http://10.55.32.157:8080",' '"bsb_url":' '"http://10.55.32.107:8080",' '"request_id":' '"1",' '"ts_url":' '"http://10.55.32.151:8080",' '"request_parameters":' '{"status":' '"Not' 'started",' '"bug_id":' null, '"updated":' '"Wed,' 26 Feb 2014 20:18:15 '+0000",' '"current_workflow_step":' '"Queued",' '"description":' '"Ticket' lintian4ptyhon '1",' '"created":' '"Wed,' 26 Feb 2014 20:18:05 '+0000",' '"series":' '"saucy",' '"subticket":' '[{"status":' '"New",' '"current_workflow_step":' '"New",' '"artifact":' '[{"resource_uri":' '"/api/v1/fullsubticketartifact/3/",' '"type":' '"SPU",' '"id":' 3, '"reference":' '"https://swift.canonistack.canonical.com/v1/AUTH_a1412d7034ce49b2a980b84b15c04372/ticket-1//home/fginther/work/engine/ppa-test/lintian4python/lintian4python_0.24.4ubuntu1~fginther.20140226.1_source.changes",' '"name":' '"lintian4python_0.24.4ubuntu1~fginther.20140226.1_source.changes"},' '{"resource_uri":' '"/api/v1/fullsubticketartifact/2/",' '"type":' '"SPU",' '"id":' 2, '"reference":' '"https://swift.canonistack.canonical.com/v1/AUTH_a1412d7034ce49b2a980b84b15c04372/ticket-1//home/fginther/work/engine/ppa-test/lintian4python/lintian4python_0.24.4ubuntu1~fginther.20140226.1.tar.gz",' '"name":' '"lintian4python_0.24.4ubuntu1~fginther.20140226.1.tar.gz"},' '{"resource_uri":' '"/api/v1/fullsubticketartifact/1/",' '"type":' '"SPU",' '"id":' 1, '"reference":' '"https://swift.canonistack.canonical.com/v1/AUTH_a1412d7034ce49b2a980b84b15c04372/ticket-1//home/fginther/work/engine/ppa-test/lintian4python/lintian4python_0.24.4ubuntu1~fginther.20140226.1.dsc",' '"name":' '"lintian4python_0.24.4ubuntu1~fginther.20140226.1.dsc"}],' '"assignee":' '"fginther",' '"source_package_upload":' '{"version":' '"0.24.4ubuntu1~fginther.20140226.1",' '"sourcepackage":' '{"resource_uri":' '"/api/v1/sourcepackage/1/",' '"id":' 1, '"name":' '"lintian4python"},' '"id":' 1, '"resource_uri":' '"/api/v1/spu/1/"},' '"id":' 1, '"resource_uri":' '"/api/v1/fullsubticket/1/"}],' '"added_binaries":' '"lintian4python",' '"owner":' '"fginther",' '"master_ppa":' '"ppa:ci-engineering-airline/ci-archive",' '"title":' '"Ticket' lintian4ptyhon '1",' '"base_image":' '"http://cloud-images.ubuntu.com/releases/13.10/release-20131015/ubuntu-13.10-server-cloudimg-amd64-disk1.img",' '"removed_binaries":' null, '"id":' 1, '"resource_uri":' '"/api/v1/next/1/"}},' '"image_builder":' '{"image_id":' '"ac5a7e48-9f26-11e3-821a-fa163e3ecf56.img",' '"state":' '"COMPLETED",' '"exit":' true, '"result":' '"PASSED"},' '"ppa_assigner":' '{"ppa":' '"ppa:fginther/ci-pool-001",' '"result":' '"PASSED",' '"location":' '"http://10.55.32.150:8080/api/v1/ppa/1/"}}'
+ cat
++ sed 's/"//g'
++ grep -Po '(?<=ts_url": )"http:.*?"'
++ cat params.json
+ /srv/lander_jenkins_sub/lander/bin/lander_service_wrapper.py --input-file params.json --output-file results/params.json --service test_runner
20:43:47 DEBUG lander_service_wrapper: Executing service wrapper: test_runner
20:43:47 INFO lander_service_wrapper: Binaries all: [u'lintian4python']
20:43:47 INFO lander_service_wrapper: Calling via POST [http://10.55.32.157:8080/api/v1/test_image]: {'progress_trigger': u'lander_master-1-testrunner', 'image_id': u'ac5a7e48-9f26-11e3-821a-fa163e3ecf56.img', 'ticket_id': u'1', 'package_list': [u'lintian4python']}
20:43:47 INFO lander_service_wrapper: starting progress handler...
20:43:48 DEBUG amqplib: Start from server, version: 8.0, properties: {u'information': u'Licensed under the MPL. See http://www.rabbitmq.com/', u'product': u'RabbitMQ', u'copyright': u'Copyright (C) 2007-2011 VMware, Inc.', u'capabilities': {}, u'platform': u'Erlang/OTP', u'version': u'2.7.1'}, mechanisms: [u'PLAIN', u'AMQPLAIN'], locales: [u'en_US']
20:43:48 DEBUG amqplib: Open OK! known_hosts []
20:43:48 DEBUG amqplib: using channel_id: 1
20:43:48 DEBUG amqplib: Channel open
20:43:48 INFO root: Waiting for messages. ^C to exit.
20:43:48 INFO PROGRESS_TRIGGER: WAITING: {u'progress_trigger': u'lander_master-1-testrunner', u'image_id': u'ac5a7e48-9f26-11e3-821a-fa163e3ecf56.img', u'ticket_id': u'1', u'package_list': [u'lintian4python']}
20:43:48 INFO PROGRESS_TRIGGER: STATUS: {u'progress_trigger': u'lander_master-1-testrunner', u'image_id': u'ac5a7e48-9f26-11e3-821a-fa163e3ecf56.img', u'ticket_id': u'1', u'package_list': [u'lintian4python']}
20:43:48 INFO PROGRESS_TRIGGER: COMPLETED: {u'exit': True, u'result': u'PASSED'}
20:43:48 DEBUG amqplib: Closed channel #1
Archiving artifacts
Triggering a new build of lander_archiver #3
Finished: SUCCESS

[Evan (ev)]

8:06 AM <vila> ev: couldn't reproduce yet and still no idea about the cause

[Paul Larson (pwlars)]

This should be fixed by: https://bugs.launchpad.net/ubuntu-ci-services-itself/+bug/1285787
Which has already been merged. The problem was that we are now using an older version of keystoneclient due to using the package. It is supposed to be the same api version, but they are incompatible. So the older servicecatalog call to url_for didn't let you specify the region, and I had to go about it a different way.

[Paul Larson (pwlars)]

One thing - to be clear - I *do* have it still picking the appropriate glance endpoint for the specified region. It's just that now you have to filter it out of a list of all known glance endpoints rather than specifying the region in the call. It is also forward compatible with the newer revisions of keystoneclient, so it shouldn't break if we get a newer version later.

[Vincent Ladeuil (vila)]

Marking fixed released based on Paul's feedback, re-open if you can reproduce.