| 2017-03-06 12:50:41 |
Sylvain Pineau |
description |
Documentation states:
Overrides are applied in order and the last applied override is the effective override in a given test plan.
But I've found that the Bluez after-suspend jobs are not always getting the right category override.
I'm using the following field in one on our snappy test plan:
category_overrides:
apply "2013.com.canonical.plainbox::suspend" to "2013.com.canonical.certification::after-suspend-.*"
apply "2013.com.canonical.plainbox::suspend" to "after-suspend-.*"
apply "2013.com.canonical.certification::after-suspend-bluetooth_bluez5_selftests" to "2013.com.canonical.certification::after-suspend-bluetooth/bluez-internal.*"
When running checkbox-cli, sometimes I see the 250 bluez tests after suspend properly grouped into their own category sometimes part of the Suspend one (the plainbox one).
Looking at plainbox code in details, we are creating an override map based on collections.defaultdict(list) and looping over items(). This is not a deterministic order.
A solution used OrderedDict is needed. |
Documentation states:
Overrides are applied in order and the last applied override is the effective override in a given test plan.
But I've found that the Bluez after-suspend jobs are not always getting the right category override.
I'm using the following field in one of our snappy test plan:
category_overrides:
apply "2013.com.canonical.plainbox::suspend" to "2013.com.canonical.certification::after-suspend-.*"
apply "2013.com.canonical.plainbox::suspend" to "after-suspend-.*"
apply "2013.com.canonical.certification::after-suspend-bluetooth_bluez5_selftests" to "2013.com.canonical.certification::after-suspend-bluetooth/bluez-internal.*"
When running checkbox-cli, sometimes I see the 250 bluez tests after suspend properly grouped into their own category sometimes part of the Suspend one (the plainbox one).
Looking at plainbox code [1] in details, we are creating an override map based on collections.defaultdict(list) and looping over items(). This is not a deterministic order.
A solution used OrderedDict is needed.
[1] https://git.launchpad.net/plainbox/tree/plainbox/impl/session/state.py#n586 |
|
