mdmonitor doesn't start recovery immediately

[Mariusz Tkaczyk]

mdmonitor reacts on md events, it pools on /proc/mdstat file. Those events are generated if a change on any mddevice is observed in kernel. This is done asynchronously and can be caused by user space process (mdadm called by udev or user), or by kernel itself (drive is removed because it has to many errors).

The problem here is that mdmonitor isn't dealing with user space or udev. When drive with metadata is inserted, mdadm adds it to mddevice (it is done by udev). Md Event is generated then and mdmonitor may try to move drive to other mddevice if needed. It relies on by-path links, but this link to newly appeared device is not created yet, udev is still working on. As a result recovery doesn't start immediately.

Observed on Ubuntu 20.04.

Steps to reproduce:

1. Create RAID volume:
# mdadm --create /dev/md/imsm0 --metadata=imsm --raid-devices=4 /dev/nvme6n1 /dev/nvme1n1 /dev/nvme7n1 /dev/nvme3n1 --run
# mdadm --create /dev/md/r10d4s64-20_A --level=10 --chunk 64 --raid-devices=4 /dev/nvme6n1 /dev/nvme1n1 /dev/nvme7n1 /dev/nvme3n1 --run

2. Add spare to container:
# mdadm --add /dev/md/imsm0 /dev/nvme0n1

3. Create appropriate policy line in /etc/mdadm/mdadm.conf.

POLICY domain=RAID_DOMAIN_1 path=* action=spare-same-slot

4. Disconnect spare from container.

5. Start mdadm monitor with big delay (ex. 10 minutes):
# mdadm --monitor --delay 6000 --scan --mail=root@localhost --daemonize --syslog

6. Hot remove disk from array (physical disconnect).

7. Connect previously prepared spare.

Expected results:
Rebuild should start.

Actual results:
Rebuild does not start, added spare is in separate container.

[Mariusz Tkaczyk (mtkaczyk)]

Hi,
mdmonitor needs to deal with other tasks, issue still in development.

Thanks, Mariusz

[Mariusz Tkaczyk (mtkaczyk)]

Hi,
Patch has been accepted upstream:
https://git.kernel.org/pub/scm/utils/mdadm/mdadm.git/commit/?id=cab9c67d461c65a1138359f9f6d39636466b90e4

[Launchpad Janitor (janitor)]

Status changed to 'Confirmed' because the bug affects multiple users.

[Jeff Lane  (bladernr)]

Can we please target this for 21.10?

[Jeff Lane  (bladernr)]

Is this resolved in 4.2-rc2?

[Mariusz Tkaczyk (mtkaczyk)]

yes, it is resolved.