cannot stop RAID1

[psl]

Binary package hint: mdadm

Ubuntu 8.04.2, i386, server

Cannot stop RAID1 device. I do a disk cleanup and I want to replace RAID1 device with RAID10 device. I cannot stop RAID1 device. /dev/md1 was used as a swap, but swap is "off" now.

# cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md2 : active raid5 sda5[0] sdd5[3] sdc5[2] sdb5[1]
      879100416 blocks level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
      [==>..................] resync = 11.6% (34281344/293033472) finish=251.8min speed=17120K/sec

md1 : active raid1 sdb3[1]
      979840 blocks [2/1] [_U]

md0 : active raid1 sda1[0] sdb1[1]
      8787456 blocks [2/2] [UU]

unused devices: <none>

# free
             total used free shared buffers cached
Mem: 515436 131576 383860 0 8632 69280
-/+ buffers/cache: 53664 461772
Swap: 0 0 0

# mdadm --stop /dev/md1
mdadm: fail to stop array /dev/md1: Device or resource busy

# mdadm /dev/md1 --fail /dev/sdb3
mdadm: set /dev/sdb3 faulty in /dev/md1

# mdadm /dev/md1 --remove /dev/sdb3
mdadm: hot remove failed for /dev/sdb3: Device or resource busy

# mdadm --stop -v /dev/md1
mdadm: fail to stop array /dev/md1: Device or resource busy

My observation is that most documentation focus on creation of new RAID devices but there is only little information about destruction of RAID devices (that is needed sometimes...). I want to delete /dev/md1 and create new /dev/md1 that will be RAID10 type. Should be an easy task but I met an issue I cannot overcome...

[psl (slansky)]

# uname -srvmo
Linux 2.6.24-23-server #1 SMP Wed Apr 1 22:22:14 UTC 2009 i686 GNU/Linux

[psl (slansky)]

# mdadm --version
mdadm - v2.6.3 - 20th August 2007

[Sander Smeenk (ubuntu-freshdot)]

This issue seems related to the device once being used for swap.

I'm experiencing /exactly/ the same problem as you describe above.
I've swapoff'd the md-device, but still i'm unable to stop the array alltogether or remove a member from it.
Everything results in EBUSY.

Ubuntu 6.06.2 LTS
Linux 2.6.15-54-amd64-k8 #1 SMP PREEMPT Thu Apr 2 20:27:18 UTC 2009 x86_64 GNU/Linux
mdadm - v1.12.0 - 14 June 2005

I also tried compiling 2.6.7.1 from Jaunty. The resulting binary works for other operations but also fails removing or stopping the raidset with EBUSY. This makes me wonder if this is mdadm related.

[Andrew Cranwell (andrew-cranwell)]

I've got a similar problem trying to break down a RAID1 array to use the individual partitions for swap. From what I understand the array was created during installation and marked as swap, then had a single partition removed from it (currently activated as swap).

$ uname -a
Linux adlai 2.6.24-23-server #1 SMP Mon Jan 26 00:55:21 UTC 2009 i686 GNU/Linux

$ cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=8.04
DISTRIB_CODENAME=hardy
DISTRIB_DESCRIPTION="Ubuntu 8.04.2"

$ mdadm --version
mdadm - v2.6.3 - 20th August 2007

$ sudo fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x9e169e16

   Device Boot Start End Blocks Id System
/dev/sda1 * 1 973 7815591 fd Linux raid autodetect
/dev/sda2 974 31367 244139805 fd Linux raid autodetect
/dev/sda3 60668 60801 1076355 82 Linux swap / Solaris
/dev/sda4 31368 60667 235352250 fd Linux raid autodetect

Partition table entries are not in disk order

Disk /dev/sdb: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x07a6b49b

   Device Boot Start End Blocks Id System
/dev/sdb1 * 1 973 7815591 fd Linux raid autodetect
/dev/sdb2 974 31367 244139805 fd Linux raid autodetect
/dev/sdb3 60668 60801 1076355 82 Linux swap / Solaris
/dev/sdb4 31368 60667 235352250 fd Linux raid autodetect

Partition table entries are not in disk order

(snip)

$ cat /proc/mdstat
Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10]
md3 : active raid1 sda2[0] sdb2[1]
      244139712 blocks [2/2] [UU]

md4 : active raid1 sda4[0] sdb4[1]
      235352128 blocks [2/2] [UU]

md2 : active raid1 sdb3[1]
      1076288 blocks [2/1] [_U]

md0 : active raid1 sda1[0] sdb1[1]
      7815488 blocks [2/2] [UU]

unused devices: <none>

$ swapon -s
Filename Type Size Used Priority
/dev/sda3 partition 1076344 112 -1

$ sudo mdadm --stop /dev/md2
mdadm: fail to stop array /dev/md2: Device or resource busy

$ sudo mdadm /dev/md2 --fail --verbose /dev/sdb3
mdadm: set /dev/sdb3 faulty in /dev/md2

$ sudo mdadm /dev/md2 --remove --verbose /dev/sdb3
mdadm: hot remove failed for /dev/sdb3: Device or resource busy

The only other thing I can think of is booting from a livecd and zeroing the superblock / running mkswap, but that will probably take a while to schedule downtime for.

[Oleg Blednov (blake-r)]

I have same problem, but on far away server.

Can anybody test this solution?: http://en.wikipedia.org/wiki/Mdadm#Known_problems

[psl (slansky)]

The latest Ubuntu 9.10, i386 has still the same issue; not fixed yet.
Kernel is 2.6.31-16-generic-pae
mdadm is 2.6.7.1-1ubuntu13

[Andrew Cranwell (andrew-cranwell)]

This has resolved itself on the server I had a problem with - after commenting out the array definition from mdadm.conf, rebooting, and zeroing the superblock.

[Andrew Cranwell (andrew-cranwell)]

Is this still affecting 10.04? I managed to fix my 8.04 system with a reboot.