template unit doesn't create jobs if a nonexistent attribute is referenced in the _description

[Daniel Manrique]

I found this bug while triaging bug 1361071.

How to reproduce:

$ plainbox run -i 2013.com.canonical.certification::staging/udev

Expected: Not sure, but output with no ERRORs.

Actual result:

$ plainbox run -i 2013.com.canonical.certification::staging/udev
===============================[ Analyzing Jobs ]===============================
=============================[ Session Statistics ]=============================
This session is about 0.00% complete
Estimated duration is 2.26 for automated jobs.
Estimated duration cannot be determined for manual jobs.
Size of the desired job list: 1
Size of the effective execution plan: 2
===========================[ Running Selected Jobs ]============================
----------------------------------[ package ]-----------------------------------
ID: 2013.com.canonical.certification::package
(Command output hidden)
Outcome: job passed
------------------------[ Collect unfiltered udev data ]------------------------
ID: 2013.com.canonical.certification::staging/udev
(Command output hidden)
ERROR plainbox.ctrl: Ignoring invalid instantiated unit Disk statistics for /dev/sda: Problem with field _description: wrong
ERROR plainbox.ctrl: Ignoring invalid instantiated unit Disk statistics for /dev/dm-0: Problem with field _description: wrong
Outcome: job passed
==================================[ Results ]===================================
2013.com.canonical.certification::package: pass
2013.com.canonical.certification::staging/udev: pass

Despite one of the devices having a weird name (dm-0) which has caused problems in the id field before, notice the problem is in the _description which should be free-form text.

The job template in question is:

unit: template
template-unit: job
template-imports: from 2013.com.canonical.certification import staging/udev as udev
template-resource: udev
template-filter: udev.attr_DEVTYPE == 'disk' and 'ram' not in udev.name and 'loop' not in udev.name
plugin: shell
id: staging/disk/stats_{name}
requires:
 block_device.{name}_state != 'removable'
user: root
command: disk_stats_test {name}
_summary: Disk statistics for {attr_DEVNAME}
_description:
 This test checks disk stats, generates some activity and rechecks stats to
 verify they've changed. It also verifies that disks appear in the various
 files they're supposed to.
 .
 This test will inspect the following disk:
 .
     product vendor: {attr_ID_VENDOR}
     product name: {attr_ID_MODEL}
     serial number: {attr_ID_SERIAL_SHORT}
     sysfs path: {path}
     device node path: {attr_DEVNAME}

The problem is that the description has attr_ID_VENDOR which is NOT present in the udev data for these devices so I get this error.

I'd understand if we refuse to instantiate templated jobs if the attributes used in vital fields are missing (e.g. if it has no {name} or if we were using the {path} or {attr_DEVNAME} and it had none). But substitution in the description should be more lenient; if an attribute referenced in the description does not exist, it should still instantiate the job, only with that portion left blank or with a suitable placeholder.

[Zygmunt Krynicki (zyga)]

Hey. Thanks for filing this. Currently this is the indented behaviour. I will think about how we can improve this to meet implicit expectations.

[Zygmunt Krynicki (zyga)]

We may improve this in the new validation core branch

[Daniel Manrique (roadmr)]

Sounds good. A workaround to get the template working would be to remove attr_ID_VENDOR from the description. This would be urgentish because this is already affecting systems in the field, but since the job/template is not in use in-production yet, I won't go ahead with this workaround since we can wait to define this a bit better.