We should filter data on the collector side. Additional configuration of rules for filtration sould be able to load from external config. If filtration rules are not provided, then all data will be marked as not filtered by collector.
Filtering rules structure: {release_id: {build_id: from_dt}}. If from_dt is not present, then stats from build_id is not marked as filtered. If from_date is present, then stats from build_id is marked as non filtered only after datetime greater than from_dt. Format of from_dt is ISO 8601.
from_dt timezone is UTC.
If filtering rules are present and release is not listed in filtering rules, data should be marked as filtered.
If build_id is not listed for release in filtering rules, data should be marked as filtered.
All other data will be marked as filtered.
Filtering rules example:
{'6.1':
{
# 6.1 build 2015-04-13_234_13-12-31 became not filtered only
# after 2015-04-30T23:00:18 UTC
'2015-04-13_234_13-12-31': '2015-04-30T23:00:18',
# 6.1 build 2015-04-13_06-18-10 not filtered
'2015-04-13_06-18-10': None
},
'6.1.1': {}, # All builds of 6.1.1 filtered
'7.0': None # All builds of 7.0 not filtered
}
Work items:
- Handle filtration on the fuel-collector side,
- Handle filtering status in the CSV exporter.
- Handle filtering status in the fuel-stats web UI.
Fix proposed to branch: master
Review: https:/