{upstream} --incremental --scan --run does not start anything

[ceg]

Binary package hint: mdadm

(ubuntu 8.04)

The man page suggests that "mdadm -IRs" will start the arrays that have been assembled incrementally, but which are still missing some redundant members (degraded mode).

But this does not seem to have any effect on inactive arrays here.
("mdadm --incremental --scan --run" does not start the (multiple) arrays that have been partially assembled by prior "mdadm --incremental <devices>" commands.)

[ceg (ceg)]

Current state of ubuntu systems with md raid: https://wiki.ubuntu.com/ReliableRaid

[ceg (ceg)]

Note that starting all incomplete array indiscriminately is not a good idea in general.

i.e. on boot, start only selected arrays degraded that are known to be needed.

[Clint Byrum (clint-fewbar)]

ceg, I know these have been around a long time without a response. I'd like to know what we can do in Ubuntu to help resolve this issue for you. It would be helpful if you could confirm that this is still a problem for you in the latest release (Ubuntu 11.04 is just about to release on 4/28). Also if you could state clearly:

- What you expect the system to do (assemble all inactive arrays?)
- What it actually does (??)
- A clear way we can attempt to confirm this behavior (i.e. break an array and then attempt to activate it?)

I'm going to mark this one as Incomplete pending your response.

[ceg (ceg)]

I'm sorry I won't be able to test 11.04 soon.

> - A clear way we can attempt to confirm this behavior (i.e. break an array and then attempt to activate it?)

Most of the issues should be covered by the overview in ReliableRaid and the test cases.
https://wiki.ubuntu.com/ReliableRaid and
http://testcases.qa.ubuntu.com/Install/AlternateRaidMirror with all three partitoning schemes.

ReliableRaid says this about the bug at hand:
Using the legacy method to start degraded raids selectively (mdadm --assemble --run -uuid) will break later --incremental (re)additions by udev/hotplugging. (The initramfs currently uses "mdadm --assemble --scan --run" and starts *all* arrays available degraded! 497186. The corresponding command "mdadm --incremental --scan --run" to start *all remaining* hotplugable raids degraded (something still to execute only manually if at all!) does not start anything. 244808)

> - What you expect the system to do (assemble all inactive arrays?)
By default, it should assemble all raid partitions that are connected with --incremental and run them only when they are complete (as defined by the state of their superblocks) to ensure no arrays are never started partially.

> - What it actually does (??)
On boot, ubuntu starts ALL arrays in degraded mode, if the rootfs is still incomplete after a timeout. It should only start the raids necessary for the rootfs in degraded mode. (how this could be done: see ReliableRaid Wiki)

[ceg (ceg)]

Please don't make the issues dissappear by the incomplete->expiraition mechanism (assume that it has gone away), if they are not confirmed to be fixed.

These things are fundamental to know not to use ubuntu for software raid.

[Clint Byrum (clint-fewbar)]

Excerpts from ceg's message of Thu Apr 28 10:01:29 UTC 2011:
> I'm sorry I won't be able to test 11.04 soon.
>
> > - A clear way we can attempt to confirm this behavior (i.e. break an
> array and then attempt to activate it?)
>
> Most of the issues should be covered by the overview in ReliableRaid and the test cases.
> https://wiki.ubuntu.com/ReliableRaid and
> http://testcases.qa.ubuntu.com/Install/AlternateRaidMirror with all three partitoning schemes.
>
> ReliableRaid says this about the bug at hand:
> Using the legacy method to start degraded raids selectively (mdadm --assemble --run -uuid) will break later --incremental (re)additions by udev/hotplugging. (The initramfs currently uses "mdadm --assemble --scan --run" and starts *all* arrays available degraded! 497186. The corresponding command "mdadm --incremental --scan --run" to start *all remaining* hotplugable raids degraded (something still to execute only manually if at all!) does not start anything. 244808)
>
> > - What you expect the system to do (assemble all inactive arrays?)
> By default, it should assemble all raid partitions that are connected with --incremental and run them only when they are complete (as defined by the state of their superblocks) to ensure no arrays are never started partially.
>
> > - What it actually does (??)
> On boot, ubuntu starts ALL arrays in degraded mode, if the rootfs is still incomplete after a timeout. It should only start the raids necessary for the rootfs in degraded mode. (how this could be done: see ReliableRaid Wiki)

Alright, I haven't seen this behavior with BOOT_DEGRADED=false. Quite
the opposite, the system stops and asks you what to do, then gives you
a chance to re-assemble the arrays. But maybe I'm missing something in
your description.

I've proposed a blueprint to suggest these fixes at UDS-O in Budapest.

https://blueprints.launchpad.net/ubuntu/+spec/server-o-software-raid

It would be helpful if you could register for uds-o as a virtual attendee
(or even better, a physical attendee!), subscribe as participation
essential (I went ahead and subscribed you but making you participation
essential just helps the scheduler if you are in fact registered),
and join us via IRC/Streaming for the discussion if the blueprint is
approved for a session at the summit.

I've linked this bug report to the blueprint. If you could link all of
the others that you feel should be grouped around this, that would be
quite helpful.

[ceg (ceg)]

> I haven't seen this behavior with BOOT_DEGRADED=false.

Right, however software raid has been created to allow your box running and rebooting fine even with partial hardware failures. That BOOT_DEGRADED=false option allows admins to safeguard against the bugs and do things manually, but shouldn't have to be there.

On the good side, mdadm now officially supports and ships udev --incremental scripts.
I'd suggest to support the debian maintainer in the swich to the --incremental method.

[vak (khamenya)]

not sure if this bug is directly related: https://bugs.launchpad.net/ubuntu/+source/mdadm/+bug/1335642