Rebuilding my RAID for the 5th time now. Here's the complete shell session, so in case I'm doing something wrong, PLEASE point it out! Let's hope it works this time! .... And here we go..... trinitronx@saturn:/media$ sudo fdisk -l /dev/sde Disk /dev/sde: 750.2 GB, 750156374016 bytes 255 heads, 63 sectors/track, 91201 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sde doesn't contain a valid partition table trinitronx@saturn:/media$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md_d0 : inactive sde1[3](S) 732571904 blocks md0 : inactive sdb1[0](S) sdc1[1](S) sdd1[2](S) 2197715712 blocks unused devices: trinitronx@saturn:/media$ sudo mdadm --stop /dev/md_d0 [sudo] password for trinitronx: mdadm: stopped /dev/md_d0 trinitronx@saturn:/media$ sudo mdadm --examine /dev/md_d0 mdadm: cannot open /dev/md_d0: No such file or directory trinitronx@saturn:/media$ sudo mdadm --examine /dev/sde mdadm: No md superblock detected on /dev/sde. trinitronx@saturn:/media$ sudo mdadm --examine /dev/sde1 /dev/sde1: Magic : a92b4efc Version : 00.90.00 UUID : 7a0e5ba8:669595fc:852484a6:e390a598 (local to host saturn) Creation Time : Fri Jan 9 03:40:39 2009 Raid Level : raid5 Used Dev Size : 732571904 (698.64 GiB 750.15 GB) Array Size : 2197715712 (2095.91 GiB 2250.46 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Update Time : Sun May 16 22:13:13 2010 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Checksum : b38b3b34 - correct Events : 122066 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 3 8 65 3 active sync /dev/sde1 0 0 8 17 0 active sync /dev/sdb1 1 1 8 33 1 active sync /dev/sdc1 2 2 8 49 2 active sync /dev/sdd1 3 3 8 65 3 active sync /dev/sde1 trinitronx@saturn:/media$ sudo mdadm --zero-superblock /dev/sde1 trinitronx@saturn:/media$ sudo mdadm --examine /dev/sde1 mdadm: No md superblock detected on /dev/sde1. trinitronx@saturn:/media$ sudo mdadm --zero-superblock /dev/sde mdadm: Unrecognised md component device - /dev/sde ------------------------------------------------------------------------------------------------- ####### HERE'S WHERE I DELETED OLD PARTITION, ADDED NEW TABLE, AND REFORMATTED IN GPARTED ####### ------------------------------------------------------------------------------------------------- trinitronx@saturn:/media$ sudo fdisk -l /dev/sde Disk /dev/sde: 750.2 GB, 750156374016 bytes 255 heads, 63 sectors/track, 91201 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0008b84d Device Boot Start End Blocks Id System /dev/sde1 1 91201 732572001 fd Linux raid autodetect trinitronx@saturn:/media$ sudo mdadm --detail /dev/md0 mdadm: md device /dev/md0 does not appear to be active. trinitronx@saturn:/media$ sudo mdadm --examine /dev/sde1 mdadm: No md superblock detected on /dev/sde1. trinitronx@saturn:/media$ sudo mdadm --examine /dev/sde mdadm: No md superblock detected on /dev/sde. trinitronx@saturn:/media$ sudo mdadm --examine /dev/sdb mdadm: No md superblock detected on /dev/sdb. trinitronx@saturn:/media$ sudo mdadm --examine /dev/sdb1 /dev/sdb1: Magic : a92b4efc Version : 00.90.00 UUID : 7a0e5ba8:669595fc:852484a6:e390a598 (local to host saturn) Creation Time : Fri Jan 9 03:40:39 2009 Raid Level : raid5 Used Dev Size : 732571904 (698.64 GiB 750.15 GB) Array Size : 2197715712 (2095.91 GiB 2250.46 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Update Time : Sun May 16 22:13:13 2010 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Checksum : b38b3afe - correct Events : 122066 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 0 8 17 0 active sync /dev/sdb1 0 0 8 17 0 active sync /dev/sdb1 1 1 8 33 1 active sync /dev/sdc1 2 2 8 49 2 active sync /dev/sdd1 3 3 8 65 3 active sync /dev/sde1 trinitronx@saturn:/media$ sudo mdadm --examine /dev/sdc1 /dev/sdc1: Magic : a92b4efc Version : 00.90.00 UUID : 7a0e5ba8:669595fc:852484a6:e390a598 (local to host saturn) Creation Time : Fri Jan 9 03:40:39 2009 Raid Level : raid5 Used Dev Size : 732571904 (698.64 GiB 750.15 GB) Array Size : 2197715712 (2095.91 GiB 2250.46 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Update Time : Sun May 16 22:13:13 2010 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Checksum : b38b3b10 - correct Events : 122066 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 1 8 33 1 active sync /dev/sdc1 0 0 8 17 0 active sync /dev/sdb1 1 1 8 33 1 active sync /dev/sdc1 2 2 8 49 2 active sync /dev/sdd1 3 3 8 65 3 active sync /dev/sde1 trinitronx@saturn:/media$ sudo mdadm --examine /dev/sdd1 /dev/sdd1: Magic : a92b4efc Version : 00.90.00 UUID : 7a0e5ba8:669595fc:852484a6:e390a598 (local to host saturn) Creation Time : Fri Jan 9 03:40:39 2009 Raid Level : raid5 Used Dev Size : 732571904 (698.64 GiB 750.15 GB) Array Size : 2197715712 (2095.91 GiB 2250.46 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Update Time : Sun May 16 22:13:13 2010 State : clean Active Devices : 4 Working Devices : 4 Failed Devices : 0 Spare Devices : 0 Checksum : b38b3b22 - correct Events : 122066 Layout : left-symmetric Chunk Size : 64K Number Major Minor RaidDevice State this 2 8 49 2 active sync /dev/sdd1 0 0 8 17 0 active sync /dev/sdb1 1 1 8 33 1 active sync /dev/sdc1 2 2 8 49 2 active sync /dev/sdd1 3 3 8 65 3 active sync /dev/sde1 trinitronx@saturn:/media$ sudo mdadm --examine /dev/sde1 mdadm: No md superblock detected on /dev/sde1. trinitronx@saturn:/media$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md0 : inactive sdb1[0](S) sdc1[1](S) sdd1[2](S) 2197715712 blocks unused devices: trinitronx@saturn:/media$ sudo mdadm --stop /dev/md0 mdadm: stopped /dev/md0 trinitronx@saturn:/media$ cat /etc/mdadm/mdadm.conf # mdadm.conf # # Please refer to mdadm.conf(5) for information about this file. # # by default, scan all partitions (/proc/partitions) for MD superblocks. # alternatively, specify devices to scan, using wildcards if desired. #DEVICE partitions # auto-create devices with Debian standard permissions CREATE owner=root group=disk mode=0660 auto=yes # automatically tag new arrays as belonging to the local system HOMEHOST # instruct the monitoring daemon where to send mail alerts MAILADDR "" MAILFROM "" # definitions of existing MD arrays ####### OLD line that kept failing #ARRAY /dev/md0 level=raid5 num-devices=4 UUID=7a0e5ba8:669595fc:852484a6:e390a598 ####### NEW line I added (let's hope it works) ARRAY /dev/md0 level=raid5 num-devices=4 devices=/dev/sdb1,/dev/sdc1,/dev/sdd1,/dev/sde1 # This file was auto-generated on Sat, 17 Apr 2010 00:23:26 -0600 # by mkconf $Id$ trinitronx@saturn:/media$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] unused devices: trinitronx@saturn:/media$ sudo mdadm --assemble --scan mdadm: /dev/md0 has been started with 3 drives (out of 4). trinitronx@saturn:/media$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md0 : active raid5 sdb1[0] sdd1[2] sdc1[1] 2197715712 blocks level 5, 64k chunk, algorithm 2 [4/3] [UUU_] unused devices: trinitronx@saturn:/media$ sudo mdadm /dev/md0 --add /dev/sde1 mdadm: added /dev/sde1 trinitronx@saturn:/media$ cat /proc/mdstat Personalities : [linear] [multipath] [raid0] [raid1] [raid6] [raid5] [raid4] [raid10] md0 : active raid5 sde1[4] sdb1[0] sdd1[2] sdc1[1] 2197715712 blocks level 5, 64k chunk, algorithm 2 [4/3] [UUU_] [>....................] recovery = 0.0% (44636/732571904) finish=273.4min speed=44636K/sec unused devices: trinitronx@saturn:/media$ sudo mdadm --detail /dev/md0 /dev/md0: Version : 00.90 Creation Time : Fri Jan 9 03:40:39 2009 Raid Level : raid5 Array Size : 2197715712 (2095.91 GiB 2250.46 GB) Used Dev Size : 732571904 (698.64 GiB 750.15 GB) Raid Devices : 4 Total Devices : 4 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Sun May 16 23:50:14 2010 State : clean, degraded, recovering Active Devices : 3 Working Devices : 4 Failed Devices : 0 Spare Devices : 1 Layout : left-symmetric Chunk Size : 64K Rebuild Status : 0% complete UUID : 7a0e5ba8:669595fc:852484a6:e390a598 (local to host saturn) Events : 0.122067 Number Major Minor RaidDevice State 0 8 17 0 active sync /dev/sdb1 1 8 33 1 active sync /dev/sdc1 2 8 49 2 active sync /dev/sdd1 4 8 65 3 spare rebuilding /dev/sde1 ------------------------------------------------------------------------------------------------- ######################## MEANWHILE, IN: /var/log/messages ... ################################### ------------------------------------------------------------------------------------------------- May 16 23:33:51 saturn kernel: [ 4763.217643] md: bind May 16 23:34:18 saturn kernel: [ 4790.391128] SGI XFS with ACLs, security attributes, realtime, large block/inode numbers, no debug enabled May 16 23:34:18 saturn kernel: [ 4790.396650] SGI XFS Quota Management subsystem May 16 23:34:18 saturn kernel: [ 4790.397410] XFS: SB read failed May 16 23:34:45 saturn kernel: [ 4817.044323] __ratelimit: 6 callbacks suppressed May 16 23:38:12 saturn kernel: [ 5024.099816] md: md_d0 stopped. May 16 23:38:12 saturn kernel: [ 5024.099859] md: unbind May 16 23:38:12 saturn kernel: [ 5024.126414] md: export_rdev(sde1) May 16 23:40:59 saturn kernel: [ 5191.024406] XFS: SB read failed May 16 23:46:00 saturn kernel: [ 5491.878478] md: md0 stopped. May 16 23:46:00 saturn kernel: [ 5491.878512] md: unbind May 16 23:46:00 saturn kernel: [ 5491.906072] md: export_rdev(sdb1) May 16 23:46:00 saturn kernel: [ 5491.906248] md: unbind May 16 23:46:00 saturn kernel: [ 5491.935829] md: export_rdev(sdc1) May 16 23:46:00 saturn kernel: [ 5491.936097] md: unbind May 16 23:46:00 saturn kernel: [ 5491.973645] md: export_rdev(sdd1) May 16 23:49:42 saturn kernel: [ 5712.161450] md: md0 stopped. May 16 23:49:42 saturn kernel: [ 5712.255438] md: bind May 16 23:49:42 saturn kernel: [ 5712.255608] md: bind May 16 23:49:42 saturn kernel: [ 5712.255763] md: bind May 16 23:49:42 saturn kernel: [ 5712.332970] raid5: device sdb1 operational as raid disk 0 May 16 23:49:42 saturn kernel: [ 5712.332974] raid5: device sdd1 operational as raid disk 2 May 16 23:49:42 saturn kernel: [ 5712.332977] raid5: device sdc1 operational as raid disk 1 May 16 23:49:42 saturn kernel: [ 5712.333463] raid5: allocated 4282kB for md0 May 16 23:49:42 saturn kernel: [ 5712.342566] 0: w=1 pa=0 pr=4 m=1 a=2 r=4 op1=0 op2=0 May 16 23:49:42 saturn kernel: [ 5712.344278] 2: w=2 pa=0 pr=4 m=1 a=2 r=4 op1=0 op2=0 May 16 23:49:42 saturn kernel: [ 5712.344281] 1: w=3 pa=0 pr=4 m=1 a=2 r=4 op1=0 op2=0 May 16 23:49:42 saturn kernel: [ 5712.344287] RAID5 conf printout: May 16 23:49:42 saturn kernel: [ 5712.344288] --- rd:4 wd:3 May 16 23:49:42 saturn kernel: [ 5712.344291] disk 0, o:1, dev:sdb1 May 16 23:49:42 saturn kernel: [ 5712.344293] disk 1, o:1, dev:sdc1 May 16 23:49:42 saturn kernel: [ 5712.344295] disk 2, o:1, dev:sdd1 May 16 23:49:42 saturn kernel: [ 5712.344338] md0: detected capacity change from 0 to 2250460889088 May 16 23:49:42 saturn kernel: [ 5712.345082] md0: unknown partition table May 16 23:50:14 saturn kernel: [ 5743.992355] md: bind May 16 23:50:14 saturn kernel: [ 5744.021356] RAID5 conf printout: May 16 23:50:14 saturn kernel: [ 5744.021366] --- rd:4 wd:3 May 16 23:50:14 saturn kernel: [ 5744.021373] disk 0, o:1, dev:sdb1 May 16 23:50:14 saturn kernel: [ 5744.021377] disk 1, o:1, dev:sdc1 May 16 23:50:14 saturn kernel: [ 5744.021381] disk 2, o:1, dev:sdd1 May 16 23:50:14 saturn kernel: [ 5744.021386] disk 3, o:1, dev:sde1 May 16 23:50:14 saturn kernel: [ 5744.023659] md: recovery of RAID array md0 May 16 23:50:14 saturn kernel: [ 5744.023668] md: minimum _guaranteed_ speed: 1000 KB/sec/disk. May 16 23:50:14 saturn kernel: [ 5744.023674] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery. May 16 23:50:14 saturn kernel: [ 5744.023687] md: using 128k window, over a total of 732571904 blocks. ------------------------------------------------------------------------------------------------- ################################## CROSS YOUR FINGERS!!! ######################################## -------------------------------------------------------------------------------------------------