usb/usb3_read_performance test executed in the wrong place

[Jeff Lane ]

The job definition for usb3_read_performance_test is currently this:
plugin: local
name: usb/usb3_read_performance
requires:
 device.category == 'DISK'
_description: Verify USB3 external storage performs at or above baseline performance
command:
 cat <<'EOF' | run_templates -t -s 'udev_resource | filter_templates -w "category=DISK"'
 plugin: shell
 name: usb/usb3_read_performance_`ls /sys$path/block`
 requires:
  device.path == "$path"
  block_device.`ls /sys$path/block`_state == 'removable' and block_device.`ls /sys$path/block`_usb3 == 'supported'
 description: USB3 read performance test for $product
 user: root
 command: disk_read_performance_test `ls /sys$path/block | sed 's|!|/|'`
 EOF

This is a local job that creates individual jobs to run the performance test against devices. Here is the problem.

First, the metajob creates job definitions for any disk it finds, so you could end up with several false starts like this:

usb/usb3_read_performance_sda usb/usb3_read_performance Unsupported Job requirement not met: 'block_device.sda_state == 'removable' and block_device.sda_usb3 == 'supported''

There is NO reason why this should be running against any non-removable hard drive. IN fact, the job should never have been created in the first place.

Next, the jobs are created at run time via the local plugin. The expectation is that this will verify the performance of usb3 devices. HOWEVER, at run time, there are no USB3 devices plugged in. This does not occur until well into testing when we get to the USB tests. At this point, the real usb3 devices we are interested in are inserted and tested.

So in the current configuration, there is no way this test will ever run against a usb3 device, unless the deviec is plugged in prior to testing and left in the whole time.

Finally, the test really should have a depends on usb/usb3_insert so that it runs between usb3_insert and usb3_remove, just like usb3_storage_automated.

[Daniel Manrique (roadmr)]

The requirement to have a plugged-in USB3 drive was expressed at the time this test was added. While the test (mostly) correctly reuses the read performance testing we already had, admittedly the need tor a preinserted USB3 device makes it more complicated to use.

That's the reason why, later, extensive work was done on adding USB3 support to the insert/storage/remove test chain. Those tests now do performance measuring and even fail USB 3.0 devices if they are too slow.

Based on this, a possible solution would be to remove this test altogether. It *may* be in use in OEM whitelists, but really, since the need to insert USB3 devices prior to testing is not communicated to the user anywhere, I doubt they're getting much useful data out of it.

I'll set to Confirmed until we can have a quick chat about what to do with this test.

[Daniel Manrique (roadmr)]

OK, so let's remove this job from checkbox, the implementation in storage_test seems to be working well and I don't think we're even using this anymore.

Setting to triaged and tagging job.

[Po-Hsu Lin (cypressyew)]

I can't find this job anywhere in the checkbox provider now.
I will close this bug as Fix-released, thanks!