fio test doesn't adjust block size

Bug #1869958 reported by Chris Sanders on 2020-03-31
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Lee Trager
Lee Trager
Lee Trager
Lee Trager

Bug Description

Fio testing for attached drives is expected from a user standpoint to check if the drives are performing to spec. From my experience, if you're lucky, the vendor will provide max bandwith and IOPS for the device.

Using the MAAS fio tests I was surprised to find that a set of new machines were severely under performing in disk throughput tests. After reviewing the test, I now see that *all* fio tests are using 4k block sizes to test. These are not the way drives are specified or tested for bandwith maximums. Here's a direct example of the difference vs a 4M block size.

Mass Results: READ: bw=628MiB/s (658MB/s)
My own fio: READ: bw=1080MiB/s (1132MB/s)

The MAAS results seemed to imply the drives were not meeting spec, but running the test myself showed they were operating as expected and matched the vendor specified values.

The recommendation I have is the following.
 * Test IOPS with 4k randread and randwrite
 * Test BW with 4M read and write

You could achieve this with something like this:
Read IOPS: fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=fio_test --bs=4k --iodepth=64 --size=10G --filename=$DRIVEPATH --readwrite=randread
Write IOPS: fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=fio_test --bs=4k --iodepth=64 --size=10G --filename=$DRIVEPATH --readwrite=randwrite
Read BW: fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=fio_test --bs=4M --iodepth=64 --size=100G --filename=$DRIVEPATH --readwrite=read
Write BW: fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=fio_test --bs=4M --iodepth=64 --size=100G --filename=$DRIVEPATH --readwrite=write

With this you would get 4k sized random read/write IOPS and 4M sequential maximum read/write Bandwidth. These align with the maximum specifications and provide a much better view if a drive is operating within spec or not.

Related branches

description: updated
Chris Sanders (chris.sanders) wrote :

Subscribing field medium

Alberto Donato (ack) on 2020-04-07
Changed in maas:
importance: Undecided → Medium
status: New → Triaged
Zachary Zehring (zzehring) wrote :

I've attached an edited version of the IOPs fio test to test more bandwidth.

Lee Trager (ltrager) on 2020-05-11
Changed in maas:
assignee: nobody → Lee Trager (ltrager)
status: Triaged → In Progress
Changed in maas:
milestone: none → 2.8.0rc1
status: In Progress → Fix Committed
Lee Trager (ltrager) on 2020-05-16
no longer affects: maas/2.3
Alberto Donato (ack) on 2020-06-04
Changed in maas:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers