| 2019-02-13 09:30:59 |
Aleksei |
bug |
|
|
added bug |
| 2019-02-13 09:45:05 |
tranadols |
description |
Hi all,
In linux-4.15, which is used as hwe-edge in xenial, there's an unexpected behavior. It results in latency spikes of generic_make_request() function call.
How to reproduce:
1. Run fio:
fio --name=test --rw=randwrite --direct=1 --filename=/dev/sdt --iodepth=1 --fsync=1 --ioengine=libaio --write_lat_log loglat
2. While fio is writing to the device, change nr_requests setting to the same value as current:
echo 256 > /sys/block/sdt/queue/nr_requests
or: if scheduler is already none:
echo none > /sys/block/sdt/queue/scheduler
3. Stop fio and check slat log:
...
3980, 16, 1, 4096
3999, 17, 1, 4096
4010, 19, 1, 4096
4020, 18, 1, 4096
4045, 10821, 1, 4096
4054, 25, 1, 4096
4070, 12, 1, 4096
4083, 17, 1, 4096
4094, 16, 1, 4096
4111, 17, 1, 4096
...
There're a few patches to fix this behavior:
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/commit/?h=for-5.1/block&id=e5fa81408fb43ebabde65938ef8b20ae879017e7
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/commit/?h=for-5.1/block&id=fbd72127c975dc8e532ecc73d52f3b1b00935bec
This behavior affects production systems, because every change uevent triggers udev to change settings prior to udev rules. Some software, for example arcconf (tool for managing adaptec HBA adapters) triggers uevents for every block device. So, every call to arcconf utility results in latency spike.
Could this be backported to 4.15 kernel for xenial/bionic? |
Hi all,
In linux-4.15, which is used as hwe-edge in xenial, there's an unexpected behavior. It results in latency spikes of generic_make_request() function call.
How to reproduce:
1. Run fio:
fio --name=test --rw=randwrite --direct=1 --filename=/dev/sdt --iodepth=1 --fsync=1 --ioengine=libaio --write_lat_log loglat
2. While fio is writing to the device, change nr_requests setting to the same value as current:
echo 256 > /sys/block/sdt/queue/nr_requests
or: if scheduler is already none:
echo none > /sys/block/sdt/queue/scheduler
3. Stop fio and check slat log:
https://bit.ly/2BtN52W
...
3980, 16, 1, 4096
3999, 17, 1, 4096
4010, 19, 1, 4096
4020, 18, 1, 4096
4045, 10821, 1, 4096
4054, 25, 1, 4096
4070, 12, 1, 4096
4083, 17, 1, 4096
4094, 16, 1, 4096
4111, 17, 1, 4096
...
There're a few patches to fix this behavior:
https://bit.ly/2BtN52W
https://bit.ly/2BtN52W
This behavior affects production systems, because every change uevent triggers udev to change settings prior to udev rules. Some software, for example arcconf (tool for managing adaptec HBA adapters) triggers uevents for every block device. So, every call to arcconf utility results in latency spike.
Could this be backported to 4.15 kernel for xenial/bionic? |
|
| 2019-02-13 10:44:32 |
Aleksei |
description |
Hi all,
In linux-4.15, which is used as hwe-edge in xenial, there's an unexpected behavior. It results in latency spikes of generic_make_request() function call.
How to reproduce:
1. Run fio:
fio --name=test --rw=randwrite --direct=1 --filename=/dev/sdt --iodepth=1 --fsync=1 --ioengine=libaio --write_lat_log loglat
2. While fio is writing to the device, change nr_requests setting to the same value as current:
echo 256 > /sys/block/sdt/queue/nr_requests
or: if scheduler is already none:
echo none > /sys/block/sdt/queue/scheduler
3. Stop fio and check slat log:
https://bit.ly/2BtN52W
...
3980, 16, 1, 4096
3999, 17, 1, 4096
4010, 19, 1, 4096
4020, 18, 1, 4096
4045, 10821, 1, 4096
4054, 25, 1, 4096
4070, 12, 1, 4096
4083, 17, 1, 4096
4094, 16, 1, 4096
4111, 17, 1, 4096
...
There're a few patches to fix this behavior:
https://bit.ly/2BtN52W
https://bit.ly/2BtN52W
This behavior affects production systems, because every change uevent triggers udev to change settings prior to udev rules. Some software, for example arcconf (tool for managing adaptec HBA adapters) triggers uevents for every block device. So, every call to arcconf utility results in latency spike.
Could this be backported to 4.15 kernel for xenial/bionic? |
Hi all,
In linux-4.15, which is used as hwe-edge in xenial, there's an unexpected behavior. It results in latency spikes of generic_make_request() function call.
How to reproduce:
1. Run fio:
fio --name=test --rw=randwrite --direct=1 --filename=/dev/sdt --iodepth=1 --fsync=1 --ioengine=libaio --write_lat_log loglat
2. While fio is writing to the device, change nr_requests setting to the same value as current:
echo 256 > /sys/block/sdt/queue/nr_requests
or: if scheduler is already none:
echo none > /sys/block/sdt/queue/scheduler
3. Stop fio and check slat log:
...
3980, 16, 1, 4096
3999, 17, 1, 4096
4010, 19, 1, 4096
4020, 18, 1, 4096
4045, 10821, 1, 4096
4054, 25, 1, 4096
4070, 12, 1, 4096
4083, 17, 1, 4096
4094, 16, 1, 4096
4111, 17, 1, 4096
...
There're a few patches to fix this behavior:
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/commit/?h=for-5.1/block&id=e5fa81408fb43ebabde65938ef8b20ae879017e7
https://git.kernel.org/pub/scm/linux/kernel/git/axboe/linux-block.git/commit/?h=for-5.1/block&id=fbd72127c975dc8e532ecc73d52f3b1b00935bec
This behavior affects production systems, because every change uevent triggers udev to change settings prior to udev rules. Some software, for example arcconf (tool for managing adaptec HBA adapters) triggers uevents for every block device. So, every call to arcconf utility results in latency spike.
Could this be backported to 4.15 kernel for xenial/bionic? |
|
| 2019-02-13 12:24:45 |
Ubuntu Foundations Team Bug Bot |
tags |
|
bot-comment |
|
| 2019-02-13 15:15:58 |
Brian Murray |
affects |
ubuntu |
linux (Ubuntu) |
|
| 2019-02-13 15:30:05 |
Ubuntu Kernel Bot |
linux (Ubuntu): status |
New |
Incomplete |
|
| 2019-02-14 00:17:31 |
Terry Rudd |
bug |
|
|
added subscriber Terry Rudd |
| 2019-02-21 14:58:57 |
Kai-Heng Feng |
attachment added |
|
0001-block-avoid-setting-wbt_lat_usec-to-current-value.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1815733/+attachment/5240679/+files/0001-block-avoid-setting-wbt_lat_usec-to-current-value.patch |
|
| 2019-02-21 14:59:13 |
Kai-Heng Feng |
attachment added |
|
0002-block-avoid-setting-none-scheduler-if-it-s-already-n.patch https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1815733/+attachment/5240680/+files/0002-block-avoid-setting-none-scheduler-if-it-s-already-n.patch |
|
| 2019-02-21 16:21:15 |
Ubuntu Foundations Team Bug Bot |
tags |
bot-comment |
bot-comment patch |
|
| 2019-02-22 11:19:44 |
Stefan Bader |
nominated for series |
|
Ubuntu Cosmic |
|
| 2019-02-22 11:19:44 |
Stefan Bader |
bug task added |
|
linux (Ubuntu Cosmic) |
|
| 2019-02-22 11:19:44 |
Stefan Bader |
nominated for series |
|
Ubuntu Bionic |
|
| 2019-02-22 11:19:44 |
Stefan Bader |
bug task added |
|
linux (Ubuntu Bionic) |
|
| 2019-02-22 11:35:22 |
Kai-Heng Feng |
nominated for series |
|
Ubuntu Disco |
|
| 2019-02-22 11:35:22 |
Kai-Heng Feng |
bug task added |
|
linux (Ubuntu Disco) |
|
| 2019-02-25 12:49:27 |
Kleber Sacilotto de Souza |
linux (Ubuntu Disco): status |
Incomplete |
In Progress |
|
| 2019-02-27 13:37:42 |
Seth Forshee |
linux (Ubuntu Disco): status |
In Progress |
Fix Committed |
|
| 2019-03-11 16:44:48 |
Launchpad Janitor |
linux (Ubuntu Disco): status |
Fix Committed |
Fix Released |
|
| 2019-04-27 03:14:58 |
AceLan Kao |
bug task added |
|
linux-oem (Ubuntu) |
|
| 2019-04-27 03:15:57 |
AceLan Kao |
linux-oem (Ubuntu Bionic): status |
New |
Fix Committed |
|
| 2019-04-27 03:16:18 |
AceLan Kao |
linux-oem (Ubuntu Cosmic): status |
New |
Invalid |
|
| 2019-04-27 03:16:22 |
AceLan Kao |
linux-oem (Ubuntu Disco): status |
New |
Invalid |
|
| 2019-05-02 07:44:35 |
AceLan Kao |
tags |
bot-comment patch |
bot-comment patch verification-needed-bionic |
|
| 2019-05-03 12:19:37 |
Aleksei |
tags |
bot-comment patch verification-needed-bionic |
bot-comment patch verification-failed-bionic |
|
| 2019-05-03 12:25:17 |
Aleksei |
tags |
bot-comment patch verification-failed-bionic |
bot-comment patch verification-needed-bionic |
|
| 2019-05-03 12:46:08 |
Aleksei |
tags |
bot-comment patch verification-needed-bionic |
bot-comment patch verification-done-bionic |
|
| 2019-05-03 15:45:53 |
Kleber Sacilotto de Souza |
linux (Ubuntu Bionic): status |
New |
In Progress |
|
| 2019-05-03 15:45:55 |
Kleber Sacilotto de Souza |
linux (Ubuntu Cosmic): status |
New |
In Progress |
|
| 2019-05-07 04:21:01 |
Khaled El Mously |
linux (Ubuntu Bionic): status |
In Progress |
Fix Committed |
|
| 2019-05-07 04:21:05 |
Khaled El Mously |
linux (Ubuntu Cosmic): status |
In Progress |
Fix Committed |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
linux-oem (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2017-5715 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2017-5753 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2017-5754 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2018-12126 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2018-12127 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2018-12130 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2018-16884 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2018-3620 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2018-3639 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2018-3646 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2019-3874 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2019-3882 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2019-9500 |
|
| 2019-05-14 19:03:14 |
Launchpad Janitor |
cve linked |
|
2019-9503 |
|
| 2019-05-17 05:35:57 |
Launchpad Janitor |
linux-oem (Ubuntu Cosmic): status |
Invalid |
Fix Released |
|
| 2019-05-17 05:35:58 |
Launchpad Janitor |
linux-oem (Ubuntu Cosmic): status |
Invalid |
Fix Released |
|
| 2019-05-17 05:36:23 |
Launchpad Janitor |
linux-oem (Ubuntu Disco): status |
Invalid |
Fix Released |
|
| 2019-05-17 05:36:23 |
Launchpad Janitor |
linux-oem (Ubuntu Disco): status |
Invalid |
Fix Released |
|
| 2019-05-20 10:01:57 |
Launchpad Janitor |
linux-oem (Ubuntu): status |
New |
Fix Released |
|
| 2019-05-20 14:01:57 |
Ubuntu Kernel Bot |
tags |
bot-comment patch verification-done-bionic |
bot-comment patch verification-done-bionic verification-needed-cosmic |
|
| 2019-05-21 10:08:13 |
Aleksei |
tags |
bot-comment patch verification-done-bionic verification-needed-cosmic |
bot-comment patch verification-done-bionic verification-done-cosmic |
|
| 2019-06-04 09:18:16 |
Launchpad Janitor |
linux (Ubuntu Bionic): status |
Fix Committed |
Fix Released |
|
| 2019-06-04 09:18:39 |
Launchpad Janitor |
linux (Ubuntu Cosmic): status |
Fix Committed |
Fix Released |
|
