Bug #191119 “Installer corrupts raid drive” : Bugs : parted package : Ubuntu

Revision history for this message

Koen (koen-beek) wrote on 2008-02-11:

#1

Thank you for taking the time to report this bug and helping to make Ubuntu better. This bug did not have a package associated with it, which is important for ensuring that it gets looked at by the proper developers. You can learn more about finding the right package at [WWW] https://wiki.ubuntu.com/Bugs/FindRightPackage . I have classified this bug as a bug in ubiquity.

Revision history for this message

Greg (gregcouch) wrote on 2008-11-02:

#2

I have the same problem. I don't know how I'm going to recover my partitions yet. The bug is apparently in the parted package since it happened while using the gparted program that supposedly calls /sbin/parted.

Revision history for this message

Greg (gregcouch) wrote on 2008-11-02:

#3

Was assigned to ubiquity, but it looks like its /sbin/parted that does the dirty work.

Revision history for this message

Anzenketh (anzenketh) wrote on 2010-02-22:

#4

Thank you for taking the time to report this bug and helping to make Ubuntu better. We are sorry that we do not always have the capacity to look at all reported bugs in a timely manner. There have been many changes in Ubuntu since that time you reported the bug and your problem may have been fixed with some of the updates. If you could test the current Ubuntu development version, this would help us a lot. If you can test it, and it is still an issue, we would appreciate if you could upload updated logs by running apport-collect <bug #>, and any other logs that are relevant for this particular issue.

Please report back if it is fixed or not.

Anzenketh (anzenketh) on 2010-02-22

Changed in parted (Ubuntu):
status:	New → Incomplete

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-04-22:

#5

No, it is NOT fucking fixed. I can't believe it's been OVER TWO YEARS and no one has bothered to even look at this easily reproducible issue? And because of it my (and who knows how many other people's) data is hosed. What issues DO you investigate, if you don't investigate corrupted file systems during the install? I've defended you guys ad nauseum in the past, but I can't avoid it any more: you guys are just plain lazy. I guess you just want a bullet on your resume saying that you're a contributer to Ubuntu, but don't actually want to do any work. I can't just say that you're useless, because useless people just don't do anything and are a drain on society. YOU allowed people's data to be lost because of your buggy garbage that you call code, and you didn't bother to fix it because you're lazy.

Changed in parted (Ubuntu):
status:	Incomplete → Confirmed

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-04-22:

#6

Please see bug #568183 for steps on recreating the issue.

Revision history for this message

Timo Jyrinki (timo-jyrinki) wrote on 2010-04-22:

#7

Name-calling people is unfortunately not helping. Note that there are quite few people capable of fixing code on this kind of low level software. There is also always limited amount of people who are both able to fix the problem, who have the affected hardware and are motivated enough to fix it on their free time. Name-calling is certainly not motivating.

In the more "reality calls" perspective, this sounds like a very important bug, but there are more bugs reported in Launchpad, and generally in all the software on the planet, than anyone will be able to fix. Data corruption bugs are very important of course, and probably something that should be on the radar of the few paid developers as well, but note that there are again thousands of reported bugs per one paid developer. In this specific case, I believe it being not fixed is a result of a combination of not finding this one skilled and motivated free-time contributor and not affecting anyone of the paid developers directly (who would potentially have the motivation to at least try to find the one person that could fix it).

Finally, please note that the bug seems to be in parted software, which is not developed by Ubuntu developers as such at all. The actual project and its bug reporting tool is at http://www.gnu.org/software/parted/index.shtml

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-04-23:

#8

I forgot to mention in my first post that once the problem shows itself, it persists indefinitely across attempts to install Ubuntu. The way I was able to fix the problem at that point was to physically zero out the defunct file system on the disk. This should be obvious, but I'll say it anyway: if zeroing out the defunct file system fixes the problem, then it stands to reason that the cause of the problem is the presence of the defunct file system. In other words, the installer is detecting the defunct file system and screwing up as a result.

Revision history for this message

Dave Howorth (dhoworth) wrote on 2010-04-23:

#9

Reading the original report here, I think it is a possibility that the problem was caused by grub overwriting part of the fakeRAID control area. A contributing factor may be a hardware environment that causes different drive orders to be recognized by BIOS, grub and kernel(s).

I don't know this for a fact but I offer it as a suggestion for an area to investigate. If so, this would mean it is a different problem to bug 568183

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-04-23:

#10

Logic 101:
There is nothing in this report that would even remotely suggest that it's a GRUB problem. By your logic, it could be anything that writes directly to the hard drive, not just GRUB. While it's true that the absence of evidence is not evidence of absence, the absence of evidence *should* tell you to concentrate your effort on the evidence you *do* have. Following your logic, we should also create a separate bug report because it's possible that invaders from Mars, who have cunningly hidden themselves from us thus far, came down and corrupted the original reporter's RAID array right when he was installing Ubuntu. There's no evidence of that (just just there's no evidence it was a GRUB problem), but it *is* technically possible.

Also, there is plenty of evidence to suggest it *isn't* GRUB. As I mentioned in my emails several times:

1. GRUB only writes in the first few kilobytes of the disk, which is safely out of the way of the data. Any other GRUB stuff is written to a file system.
2. I've replicated the problem, and simply wiping the left over file system fixes it. If it were a GRUB problem, the problem would continue.

Additionally, if this issue were a separate GRUB problem:

3. Ubuntu 8.04 used GRUB version 1, which was extremely mature and well vetted by that point.
4. It would affect ALL distributions that use that version of GRUB, and ALL installs where the user has a RAID drive, and EVERY time the user installs GRUB. This is clearly not the case.
5. The installer actually installs GRUB on ALL of your hard drives (since it doesn't know which one will be active; some other OSs change the active drive at boot), so if that were true, ALL drives would have been corrupted. The reporter clearly stated that only one RAID drive was affected, and he just had to start in degraded mode and resync.

Please, stay in school. If you're not in school, go back.

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-04-24:

#11

Download full text (8.8 KiB)

I'm adding this message from the thread to provide as much information for people trying to recreate this bug as possible;

Top-posting for (my) convenience on this one...

It's nice to see someone actually trying to recreate the bug instead of
just flapping their gums trying to sound smart. If you have the time,
could you try recreating it again? I have some suggestions to make it
more like my scenario (in fact, number 3 below is required for the
problem to occur, and number 4 below is likely to be required). I know
these suggestions are long, but it would be appreciated. In for a
penny, in for a pound, eh? You'll have to do the first install again,
but you won't have to actually go through with the second install. You
can cancel after getting past the partitioning screen. I've noticed
that when things go awry there are two tell-tale signs:

1. On the partitioning screen, the original (now defunct) file system(s)
will be detected and show up.

2. Once you select "finish partitioning", the installer will show a list
of partition tables that will be modified. One or more RAID partitions
will show up on the list, irregardless of that fact that you didn't
select them for anything.

If those signs are present, the RAID array will be hosed. If the signs
are not there, the install will go fine and there's no need to continue.
Additionally, you don't have to worry about test data or even mounting
the RAID array.

When doing the second install, if you post exactly which file systems
were detected on the manual partitioning screen and which partitions
were shown on the "to be modified" list once you hit "finish
partitioning", I'd appreciate it. Now on to the suggestions:

1. It sounds like in your setup you installed for the second time after
setting up the RAID array, but before the array finished resyncing for
the first time. In my setup, the array had been around for a while and
was fully resynced. In fact, I (likely) waited for the array to be
fully resynced before even installing XFS on it. If you *did* wait for
the drive to finish resyncing before the second install, please RSVP
because your array was indeed corrupted, but since it was only one drive
the array was somehow able to resync and recover.

2. I was using the ubuntu 10.4 beta 2 server 64-bit disk for the
(second) install when things went south. Could you try that one?

3. REQUIRED. It sounds like when doing the second install, you just
installed to an existing partition. In order for the problem to occur,
you have to remove/create a partition (even though you're leaving the
RAID partitions alone). If you recreate the partitions I used (6,
below), this will be taken care of.

4. POSSIBLY REQUIRED. When creating the RAID array with the default
options as you did in your first test, by default the array is created
in degraded mode, with a drive added later and resynced. This makes the
initial sync faster. Since I'm a neurotic perfectionist, I always
create my arrays with the much more manly and macho "--force" option to
create them "properly". Its very possible that doing the initial resync
with a degraded array will overwrite t...

I'm adding this message from the thread to provide as much information for people trying to recreate this bug as possible;

Top-posting for (my) convenience on this one...

It's nice to see someone actually trying to recreate the bug instead of 
just flapping their gums trying to sound smart.  If you have the time, 
could you try recreating it again?   I have some suggestions to make it 
more like my scenario (in fact, number 3 below is required for the 
problem to occur, and number 4 below is likely to be required).  I know 
these suggestions are long, but it would be appreciated.  In for a 
penny, in for a pound, eh?  You'll have to do the first install again, 
but you won't have to actually go through with the second install.  You 
can cancel after getting past the partitioning screen.  I've noticed 
that when things go awry there are two tell-tale signs:

1. On the partitioning screen, the original (now defunct) file system(s) 
will be detected and show up.

2. Once you select "finish partitioning", the installer will show a list 
of partition tables that will be modified.  One or more RAID partitions 
will show up on the list, irregardless of that fact that you didn't 
select them for anything.

If those signs are present, the RAID array will be hosed.  If the signs 
are not there, the install will go fine and there's no need to continue. 
  Additionally, you don't have to worry about test data or even mounting 
the RAID array.

When doing the second install, if you post exactly which file systems 
were detected on the manual partitioning screen and which partitions 
were shown on the "to be modified" list once you hit "finish 
partitioning", I'd appreciate it.  Now on to the suggestions:

1. It sounds like in your setup you installed for the second time after 
setting up the RAID array, but before the array finished resyncing for 
the first time.  In my setup, the array had been around for a while and 
was fully resynced.  In fact, I (likely) waited for the array to be 
fully resynced before even installing XFS on it.  If you *did* wait for 
the drive to finish resyncing before the second install,  please RSVP 
because your array was indeed corrupted, but since it was only one drive 
the array was somehow able to resync and recover.

2. I was using the ubuntu 10.4 beta 2 server 64-bit disk for the 
(second) install when things went south.  Could you try that one?

3. REQUIRED.  It sounds like when doing the second install, you just 
installed to an existing partition.  In order for the problem to occur, 
you have to remove/create a partition (even though you're leaving the 
RAID partitions alone).  If you recreate the partitions I used (6, 
below), this will be taken care of.

4. POSSIBLY REQUIRED.  When creating the RAID array with the default 
options as you did in your first test, by default the array is created 
in degraded mode, with a drive added later and resynced. This makes the 
initial sync faster.  Since I'm a neurotic perfectionist, I always 
create my arrays with the much more manly and macho "--force" option to 
create them "properly".  Its very possible that doing the initial resync 
with a degraded array will overwrite the defunct file system, while 
doing the initial sync in the more "proper" way will not.  Please use 
the "--force" option when creating the array to take this possibility 
into account.

5. My array has 4 drives.  If possible, could you scare up a fourth 
drive?  If not, don't worry about it.  Especially if you do number 7.

6. Prior to/during the infamous install, my partitions were as follows. 
  If feasible, recreating as much of this as possible would be appreciated:

sd[abcd]1 25GB
    sd[abcd]2 475GB

My RAID5 array was sd[abcd]2 set up as md1, and my file systems were:

sda1:     ext4    /
    md1:      xfs     /data
    sd[bcd]1: (partitioned, but not used)

Note I had no swap partition originally.

On the manual partition screen of the ill-fated install, I left the 
sd[abcd]2 partitions alone (RAID array), deleted all the sd[abcd]1 
partitions, and created the following partitions:

sd[abcd]1:  22GB   RAID5   md2   /
    sd[abcd]3*:  3GB   RAID5   md3   (swap)

* POSSIBLY REQUIRED:  Note that partition 2 of the drives' partition 
tables was already taken by the RAID, so I created the 3GB partitions as 
partition 3, even though the sectors in partition 3 physically resided 
before the sectors in partition 2.  This is perfectly "legal", if not 
"normal" (fdisk will warn you about this).  Please try to recreate this 
condition if you can, because it's very possible that was the source of 
the problems.

BTW, all partitions were primary partitions.

If you don't have that much space, you can likely get away with 
making sd[abcd]2 as small as needed.

7. I simplified when recreating this bug, but in my original scenario I 
had 2 defunct file systems detected by the installer: one on sda2 and 
one on sdd2 (both ext4).  That's why I couldn't just fail and remove the 
corrupted drive even if I had known to do so at that point.  I figure 
the more defunct file systems there are, the more chances you have of 
recreating the bug.  So how about creating file systems on all four 
partitions (sd[abcd]2) before creating the RAID array?

8. My original setup left the RAID partitions' type as "linux" instead 
of "RAID autodetect".  It's no longer necessary to set the partition 
type for RAID members, as the presence of the RAID superblock is enough. 
  When recreating the problem I did set the type to "RAID autodetect", 
but to be thorough, try leaving the type as "linux".

9. If you *really* have too much time on your hands, my original ubuntu 
install, used for creating the original file systems, was 8.10 desktop 
64 bit.  I created the non-RAID during the install and the RAID array 
after the install, after apt-getting mdadm.  I seriously doubt this 
makes a difference though.

10. I was using an external USB DVD-ROM drive to due the install.  It's 
very remotely possible since the drive has to be re-detected during the 
install process, it could wind up reshuffling the device letters.  If 
you have an external CD or DVD drive, could you try installing with it?

If you (or anybody) can try recreating the problem with this new 
information I'd very much appreciate it.

Thanks,
Alvin

On 04/23/2010 02:22 PM, J wrote:
> FWIW, this is what I just went through, step by step to try to
> recreate a loss of data on an existing sofware raid array:
> 
> 1: Installed a fresh Karmic system on a single disk with three partitions:
> /dev/sda1 = /
> /dev/sda2 = /data
> /dev/sda3 = swap
> 
> all were primary partitions.
> 
> 2: After installing 9.10, I created some test "important data" by
> copying the contents of /etc into /data.
> 3: For science, rebooted and verified that /data automounted and the
> "important data" was still there.
> 4: Shut the system down and added two disks.  Rebooted the system.
> 5: Moved the contents of /data to /home/myuser/holding/
> 6: created partitions on /dev/sdb and /dev/sdc (the two new disks, one
> partiton each)
> 7: installed mdadm and xfsprogs, xfsdump
> 8: created /dev/md0 with mdadm using /dev/sda2, /dev/sdb1 and
> /dev/sdc1 in a RAID5 array
> 9: formatted the new raid device as xfs
> 10: configured mdadm.conf and fstab to start and automount the new
> array to /data at boot time.
> 11: mounted /data (my new RAID5 array) and moved the contents of
> /home/myuser/holding to /data (essentially moving the "important data"
> that used to reside on /dev/sda2 to the new R5 ARRAY).
> 12: rebooted the system and verified that A: RAID started, B: /data
> (md0) mounted, and C: my data was there.
> 13: rebooted the system using Lucid
> 14: installed Lucid, choosing manual partitioning as you described.
> **Note: the partitioner showed all partitions, but did NOT show the
> RAID partitions as ext4
> 15: configured the partitioner so that / was installed to /dev/sda1
> and the original swap partition was used. DID NOT DO ANYTHING with the
> RAID partitions.
> 16: installed.  Installer only showed formatting /dev/sda1 as ext4,
> just as I'd specified.
> 17: booted newly installed Lucid system.
> 18: checked with fdisk -l and saw that all RAID partitions showed as
> "Linux raid autodetect"
> 19: mdadm.conf was autoconfigured and showed md0 present.
> 20: edited fstab to add the md0 entry again so it would mount to /data
> 21: did an mdadm --assemble --scan and waited for the array to rebuild
> 22: after rebuild/re-assembly was complete, mounted /data (md0)
> 23, verified that all the "important data" was still there, in my
> array, on my newly installed Lucid system.
> 
> The only thing I noticed was that when I did the assembly, it started
> degraded with sda2 and sdb1 as active and sdc1 marked as a spare with
> rebuilding in progress.
> 
> Once the rebuild was done was when I mounted the array and verified my
> data was still present.
> 
> So... what did I miss in recreating this failure?

Revision history for this message

Jeff Lane  (bladernr) wrote on 2010-04-27:

#12

Download full text (11.8 KiB)

> Top-posting for (my) convenience on this one...
>
> It's nice to see someone actually trying to recreate the bug instead of
> just flapping their gums trying to sound smart. If you have the time,
> could you try recreating it again? I have some suggestions to make it
> more like my scenario (in fact, number 3 below is required for the
> problem to occur, and number 4 below is likely to be required). I know
> these suggestions are long, but it would be appreciated. In for a
> penny, in for a pound, eh? You'll have to do the first install again,
> but you won't have to actually go through with the second install. You
> can cancel after getting past the partitioning screen. I've noticed
> that when things go awry there are two tell-tale signs:

Be warned, I have to do this in a virtualized environment as I don't
have enough physical hardware to recreate this. My only multi-disk
system is my home fileserver and I don't really fancy tearing that
apart and rebuilding it... so keep in mind that I'm doing this in VM
space... but given that this is software RAID that shouldn't matter as
the OS doesn't know it's a VM.

Rest of my reply is inline:

> 1. On the partitioning screen, the original (now defunct) file system(s)
> will be detected and show up.
>
> 2. Once you select "finish partitioning", the installer will show a list
> of partition tables that will be modified. One or more RAID partitions
> will show up on the list, irregardless of that fact that you didn't
> select them for anything.
>
> If those signs are present, the RAID array will be hosed. If the signs
> are not there, the install will go fine and there's no need to continue.
> Additionally, you don't have to worry about test data or even mounting
> the RAID array.
>
> When doing the second install, if you post exactly which file systems
> were detected on the manual partitioning screen and which partitions
> were shown on the "to be modified" list once you hit "finish
> partitioning", I'd appreciate it. Now on to the suggestions:
>
> 1. It sounds like in your setup you installed for the second time after
> setting up the RAID array, but before the array finished resyncing for
> the first time. In my setup, the array had been around for a while and
> was fully resynced. In fact, I (likely) waited for the array to be
> fully resynced before even installing XFS on it. If you *did* wait for
> the drive to finish resyncing before the second install, please RSVP
> because your array was indeed corrupted, but since it was only one drive
> the array was somehow able to resync and recover.

Yes... I rebuild the entire thing from the ground up, this time when I
added three disks (instead of two) I used --force and waited for the
entire thing to sync and become active before putting an XFS
filesystem on it. After that point, I mounted /dev/md0 and copied
about 2GB of data to the new array.

> 2. I was using the ubuntu 10.4 beta 2 server 64-bit disk for the
> (second) install when things went south. Could you try that one?

Sadly, I can not. I no longer have access to the beta 2 ISOs (not
even sure where they are online at this point), however, that's not
necessarily a bad ...

> Top-posting for (my) convenience on this one...
>
> It's nice to see someone actually trying to recreate the bug instead of
> just flapping their gums trying to sound smart.  If you have the time,
> could you try recreating it again?   I have some suggestions to make it
> more like my scenario (in fact, number 3 below is required for the
> problem to occur, and number 4 below is likely to be required).  I know
> these suggestions are long, but it would be appreciated.  In for a
> penny, in for a pound, eh?  You'll have to do the first install again,
> but you won't have to actually go through with the second install.  You
> can cancel after getting past the partitioning screen.  I've noticed
> that when things go awry there are two tell-tale signs:

Be warned, I have to do this in a virtualized environment as I don't
have enough physical hardware to recreate this.  My only multi-disk
system is my home fileserver and I don't really fancy tearing that
apart and rebuilding it...  so keep in mind that I'm doing this in VM
space... but given that this is software RAID that shouldn't matter as
the OS doesn't know it's a VM.

Rest of my reply is inline:

> 1. On the partitioning screen, the original (now defunct) file system(s)
> will be detected and show up.
>
> 2. Once you select "finish partitioning", the installer will show a list
> of partition tables that will be modified.  One or more RAID partitions
> will show up on the list, irregardless of that fact that you didn't
> select them for anything.
>
> If those signs are present, the RAID array will be hosed.  If the signs
> are not there, the install will go fine and there's no need to continue.
>  Additionally, you don't have to worry about test data or even mounting
> the RAID array.
>
> When doing the second install, if you post exactly which file systems
> were detected on the manual partitioning screen and which partitions
> were shown on the "to be modified" list once you hit "finish
> partitioning", I'd appreciate it.  Now on to the suggestions:
>
> 1. It sounds like in your setup you installed for the second time after
> setting up the RAID array, but before the array finished resyncing for
> the first time.  In my setup, the array had been around for a while and
> was fully resynced.  In fact, I (likely) waited for the array to be
> fully resynced before even installing XFS on it.  If you *did* wait for
> the drive to finish resyncing before the second install,  please RSVP
> because your array was indeed corrupted, but since it was only one drive
> the array was somehow able to resync and recover.

Yes... I rebuild the entire thing from the ground up, this time when I
added three disks (instead of two) I used --force and waited for the
entire thing to sync and become active before putting an XFS
filesystem on it.  After that point, I mounted /dev/md0 and copied
about 2GB of data to the new array.

> 2. I was using the ubuntu 10.4 beta 2 server 64-bit disk for the
> (second) install when things went south.  Could you try that one?

Sadly, I can not.  I no longer have access to the beta 2 ISOs (not
even sure where they are online at this point), however, that's not
necessarily a bad thing in and of itself.  If this issue was resolved
by RC, (though the first time I tried this I DID use a Beta 2 ISO)
then it may be ok... BUT, I do understand the need to recreate the
original failure as well, because using a later version of the code
introduces a new variable that was not present originally.

But, it is what it is.

> 3. REQUIRED.  It sounds like when doing the second install, you just
> installed to an existing partition.  In order for the problem to occur,
> you have to remove/create a partition (even though you're leaving the
> RAID partitions alone).  If you recreate the partitions I used (6,
> below), this will be taken care of.

Aye, that I did.  This time around, I followed what you outlined and
deleted the original sda1 and created a couple new arrays for the new
10.04 install.

> 4. POSSIBLY REQUIRED.  When creating the RAID array with the default
> options as you did in your first test, by default the array is created
> in degraded mode, with a drive added later and resynced. This makes the
> initial sync faster.  Since I'm a neurotic perfectionist, I always
> create my arrays with the much more manly and macho "--force" option to
> create them "properly".  Its very possible that doing the initial resync
> with a degraded array will overwrite the defunct file system, while
> doing the initial sync in the more "proper" way will not.  Please use
> the "--force" option when creating the array to take this possibility
> into account.

This time around, I created the first array from sd[abcd]2 11.5GB each
using --force.

I also waited until the array had fully built before putting an XFS
filesystem on it, mounting and loading with about 2GB of data.

> 5. My array has 4 drives.  If possible, could you scare up a fourth
> drive?  If not, don't worry about it.  Especially if you do number 7.

Did that... 1 20GB disk as sda, 3 20GB disks as sd[bcd]

> 6. Prior to/during the infamous install, my partitions were as follows.
>  If feasible, recreating as much of this as possible would be appreciated:
>
>    sd[abcd]1 25GB
>    sd[abcd]2 475GB

This second attempt I started with:

sda1 10GB
sda2 10GB

>    My RAID5 array was sd[abcd]2 set up as md1, and my file systems were:
>
>    sda1:     ext4    /
>    md1:      xfs     /data
>    sd[bcd]1: (partitioned, but not used)

I've matched that with 9.10:

sda1:  ext4  /
md0:  xfs    /data
sd[bcd]1: partitioned but not used

>    On the manual partition screen of the ill-fated install, I left the
> sd[abcd]2 partitions alone (RAID array), deleted all the sd[abcd]1
> partitions, and created the following partitions:
>
>    sd[abcd]1:  22GB   RAID5   md2   /
>    sd[abcd]3*:  3GB   RAID5   md3   (swap)

I should add that I also set sd[abcd]2 flags to raid

>    * POSSIBLY REQUIRED:  Note that partition 2 of the drives' partition
> tables was already taken by the RAID, so I created the 3GB partitions as
> partition 3, even though the sectors in partition 3 physically resided
> before the sectors in partition 2.  This is perfectly "legal", if not
> "normal" (fdisk will warn you about this).  Please try to recreate this
> condition if you can, because it's very possible that was the source of
> the problems.2
>
>    BTW, all partitions were primary partitions.
>
>    If you don't have that much space, you can likely get away with
> making sd[abcd]2 as small as needed.

So then I set about installing 10.04 RC and following what you posted above,

Once in the partitioner, This is what was listed:

RAID 5 device #0 - 34.4 GB Software RAID device
   #1                  34.4 GB     xfs
                         196.6 kB     unusable
SCSI3 (0,0,0) (sda) - 21.5 GB ATA VBOX HARDDISK
   #1    primary    10.0 GB   B    ext4
   #2    primary    11.5 GB        K    raid
SCSI4 (0,0,0) (sdb) - 21.5 GB ATA VBOX HARDDISK
   #1    primary    10.0 GB
   #2    primary    11.5 GB        K    raid
SCSI5 (0,0,0) (sdc) - 21.5 GB ATA VBOX HARDDISK
   #1    primary    10.0 GB
   #2    primary    11.5 GB        K    raid
SCSI6 (0,0,0) (sdd) - 21.5 GB ATA VBOX HARDDISK
   #1    primary    10.0 GB
   #2    primary    11.5 GB        K    raid

I deleted sda1 and created 2 new partitons from the 10GB of newly free space.

sda1 9GB
sd

and set each to "physical volume for RAID" and set each partiton to be
a primary so no logicals involved here.

Now, my partitioning looked like this:

RAID 5 device #0 - 34.4 GB Software RAID device
   #1                  34.4 GB     xfs
                         196.6 kB     unusable
SCSI3 (0,0,0) (sda) - 21.5 GB ATA VBOX HARDDISK
   #1    primary      9.0 GB        K    raid
   #3    primary      1.0 GB        K    raid
   #2    primary    11.5 GB        K    raid
SCSI4 (0,0,0) (sdb) - 21.5 GB ATA VBOX HARDDISK
   #1    primary      9.0 GB        K    raid
   #3    primary      1.0 GB        K    raid
   #2    primary    11.5 GB        K    raid
SCSI5 (0,0,0) (sdc) - 21.5 GB ATA VBOX HARDDISK
   #1    primary      9.0 GB        K    raid
   #3    primary      1.0 GB        K    raid
   #2    primary    11.5 GB        K    raid
SCSI6 (0,0,0) (sdd) - 21.5 GB ATA VBOX HARDDISK
   #1    primary      9.0 GB        K    raid
   #3    primary      1.0 GB        K    raid
   #2    primary    11.5 GB        K    raid

I then set about creating md1 and md2:

md1: sd[abcd]1 (note that during RAID configuration, NONE of the
existing partitions for md0 were listed as available.  The installer
picked up the metadata and maintained the array (at least it appeared
to be so at this point).
md2: sd[abcd]3

md1 I set up as / using ext4 and (also note that unless I specifically
state a setting, everything else was left as defaults)
md2 set up as swap.

so now I had this:

RAID 5 device #0 - 34.4 GB Software RAID device
   #1                  34.4 GB     xfs
                       196.6 kB      unusable
RAID 5 device #1 - 27.0 GB Software RAID device
   #1                  27.0 GB   f ext4        /
                       196.6 kB      unusable
RAID 5 device #2 - 3.0 GB Software RAID device
   #1                    3.0 GB   f swap      swap
                       196.6 kB      unusable

At that, I selected "write changes to disk" and the next screen shows:

The partition tables of the following devices are changed:
   RAID device #1
   RAID device #2

The following partitions are going to be formatted:
   partition #1 of RAID 5 device #1 as ext4
   partition #1 of RAID 5 device #2 as swap

I hit yes and off she went.

Not once did I see the two fail condition indicators you mentioned
above, but I let the install continue to see what would happen after
installation and reboot.

On reboot, after logging in, mdadm --detail /dev/md0 showed my
original RAID 5 array active and in good health.  Mounting the device
showed that the data copied into it prior to the 10.04 install was
still there and likewise in good health.

> 7. I simplified when recreating this bug, but in my original scenario I
> had 2 defunct file systems detected by the installer: one on sda2 and
> one on sdd2 (both ext4).  That's why I couldn't just fail and remove the
> corrupted drive even if I had known to do so at that point.  I figure
> the more defunct file systems there are, the more chances you have of
> recreating the bug.  So how about creating file systems on all four
> partitions (sd[abcd]2) before creating the RAID array?

I may be willing to do this again, but it'll be a while as I've
neglected a lot of work today to try to recreate this (I'm writing
this after my second attempt today).

> 8. My original setup left the RAID partitions' type as "linux" instead
> of "RAID autodetect".  It's no longer necessary to set the partition
> type for RAID members, as the presence of the RAID superblock is enough.
>  When recreating the problem I did set the type to "RAID autodetect",
> but to be thorough, try leaving the type as "linux".

on my first attempt, I left them as linux and did not see the issue.
on this attempt I set them all to RAID as I created them, and again,
no issue.

> 9. If you *really* have too much time on your hands, my original ubuntu
> install, used for creating the original file systems, was 8.10 desktop
> 64 bit.  I created the non-RAID during the install and the RAID array
> after the install, after apt-getting mdadm.  I seriously doubt this
> makes a difference though.

> 10. I was using an external USB DVD-ROM drive to due the install.  It's
> very remotely possible since the drive has to be re-detected during the
> install process, it could wind up reshuffling the device letters.  If
> you have an external CD or DVD drive, could you try installing with it?

Via the power of VMs, everything has been installed via a CDROM,
however, I could try by using an external DVDROM pushed through...
again, see previous comments regarding time.

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-04-27:

#13

It probably won't work in a VM, because of the way a VM's virtual storage devices work. They discard everything that's not being used at that moment, otherwise you'd have to allocate the full size of the drive. Thus, you probably couldn't have any defunct file systems sitting around.

Revision history for this message

Jeff Lane  (bladernr) wrote on 2010-04-27:

#14

FWIW, I do not use sparse files on my VMs... I allocate the full size prior and build each drive prior to use, so in my case, a 20GB drive is actually 20GB, not 1GB with a 20GB limit... but it was worth a shot at least... I tried.

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-04-27:

#15

Yup, and I thank you for it. And if this bug does get fixed, I imagine those people whose data won't be hosed as a result would thank you too.

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-04-27:

#16

Note that in bug 560152, NickJ's problem is actually this bug.

Revision history for this message

Linux000 (michael-yoyo) wrote on 2010-04-30:

#17

First off, I don't have the hardware to try this, but there is a chance of grub wiping over information in the RAID controller(like a catalog to retrieve information form multiple disks)and rendering the data unreadable, which would only affect raid arrays, and creating a new file system would work with the new addresses. I will try to replicate it in a VM with fixed size disks. Interested to see if this happens to a raid 5 array... and if it does(and if you can) does taking a disk out and reading it as a single disk recover the information? Sorry if any of this sounds crazy, I don't have much experience with raid systems.

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-05-01:

#18

Linux000, don't bother; I have practically written a dissertation on why it isn't likely to be a grub problem in one of these bugs.

Revision history for this message

Alvin Thompson (alvint-deactivatedaccount) wrote on 2010-05-17:

#19

(copying comment from bug 568183)

Thanks for the answer, but now that I think about it, I'm not sure NEW_LABEL alone would cause wholesale RAID array corruption. Correct me if I'm wrong because I'm not an expert on how parted works, but that bug with NEW_LABEL would only overwrite the RAID superblock and not the data? In which case, the RAID array would just start up in degraded mode and have to be re-synced. In my case, and in the original bug report, and in NickJ's case, the array was not degraded, but rather the data on the array was corrupted.

Even supposing the NEW_LABEL command did cause the corruption, while it may have been an upstream bug that physically wrote to the disk and caused that problem, the root of all evil is still the installer that incorrectly told parted there was a file system present on a device that's a component of the RAID array. Are you saying that parted is also responsible for detecting which file systems are present as well, and the installer only reports it? In any case it needs to be fixed, because only bad things can happen if the installer doesn't know what file systems are actually present.

In any case, there needs to be basic safeguards in place which prevent the installer from writing directly to partitions with a RAID superblock, has a "RAID autodetect" partition type, or are in a logical volume. I guess you could argue that this would rather be an enhancement, in the way you could argue that a car without brakes is not a defect, but rather brakes would be a nice-to-have enhancement.

Revision history for this message

Jeff Lane  (bladernr) wrote on 2011-10-08:

#20

Could someone take a look at this? This was a pretty serious issue that seems to have just fizzled out as far as response goes.

Revision history for this message

Phillip Susi (psusi) wrote on 2011-10-10:

#21

Correct me of I'm wrong, but it sounds like this problem begins with choosing to install Ubuntu to one of the individual disks comprising the raid array, instead of to the array as a whole. If that is the case, that is user error, not a bug.

If that is not the case, please reproduce this, and then attach the generated data from running the boot info script here:

http://sourceforge.net/projects/bootinfoscript/

Changed in parted (Ubuntu):
status:	Confirmed → Incomplete

Revision history for this message

Launchpad Janitor (janitor) wrote on 2011-12-10:

#22

[Expired for parted (Ubuntu) because there has been no activity for 60 days.]

Changed in parted (Ubuntu):
status:	Incomplete → Expired

Revision history for this message

Greg (gregcouch) wrote on 2011-12-10: Re: [Bug 191119] Re: Installer corrupts raid drive

#23

Is that an option now? The installer didn't let me install to the raid
array before and I have been reluctant to try again since it screwed things
up so badly before.

-- Greg

On Mon, Oct 10, 2011 at 10:53 AM, Phillip Susi <email address hidden> wrote:

> Correct me of I'm wrong, but it sounds like this problem begins with
> choosing to install Ubuntu to one of the individual disks comprising the
> raid array, instead of to the array as a whole. If that is the case,
> that is user error, not a bug.
>
> If that is not the case, please reproduce this, and then attach the
> generated data from running the boot info script here:
>
> http://sourceforge.net/projects/bootinfoscript/
>
>
> ** Changed in: parted (Ubuntu)
> Status: Confirmed => Incomplete
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/191119
>
> Title:
> Installer corrupts raid drive
>
> Status in “parted” package in Ubuntu:
> Incomplete
>
> Bug description:
> Boot from standard 8.04 Desktop CD.
> Run installer, select default language, keyboard to get to Partitioner.
> Select Manual install
> Note that raid drives from booting windows are listed separately.
> Select partition on alternate, non raid, drive to install Ubuntu on.
> Edit partition and set mount point to "/"
> Set login, computer name, etc. and install.
> Reboot computer.
> Note that BIOS now shows first drive as not a raid member.
> Run Windows.
> Note that software needs to rebuild RAID 1 mirror.
> Reboot computer.
> Note that BIOS shows first drive as valid raid member again.
>
> The problem I have is that because Ubuntu doesn't recognize the RAID
> drives, it corrupts one of them when it installs the default boot
> loader. To even figure out that the installer is doing this I need to
> look in the advanced settings.
>
> It would help to have a complete list of all disk operations that will
> be performed (and the affected drives or partitions) clearly shown on
> the the final Ready to Install screen.
>
> Another solution would be for Ubuntu to better support the Intel ICH9R
> controller that I happen to be using, but I think that is only a
> partial solution because the next software raid system that Intel
> comes up with might not be supported (and I think that adding better
> support for Intel raid is already on the wishlist).
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/parted/+bug/191119/+subscriptions
>

Is that an option now?  The installer didn't let me install to the raid
array before and I have been reluctant to try again since it screwed things
up so badly before.

-- Greg

On Mon, Oct 10, 2011 at 10:53 AM, Phillip Susi <psusi@cfl.rr.com> wrote:

> Correct me of I'm wrong, but it sounds like this problem begins with
> choosing to install Ubuntu to one of the individual disks comprising the
> raid array, instead of to the array as a whole.  If that is the case,
> that is user error, not a bug.
>
> If that is not the case, please reproduce this, and then attach the
> generated data from running the boot info script here:
>
> http://sourceforge.net/projects/bootinfoscript/
>
>
> ** Changed in: parted (Ubuntu)
>       Status: Confirmed => Incomplete
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/191119
>
> Title:
>  Installer corrupts raid drive
>
> Status in “parted” package in Ubuntu:
>   Incomplete
>
> Bug description:
>  Boot from standard 8.04 Desktop CD.
>  Run installer, select default language, keyboard to get to Partitioner.
>  Select Manual install
>    Note that raid drives from booting windows are listed separately.
>    Select partition on alternate, non raid, drive to install Ubuntu on.
>  Edit partition and set mount point to "/"
>  Set login, computer name, etc. and install.
>  Reboot computer.
>    Note that BIOS now shows first drive as not a raid member.
>  Run Windows.
>    Note that software needs to rebuild RAID 1 mirror.
>  Reboot computer.
>    Note that BIOS shows first drive as valid raid member again.
>
>  The problem I have is that because Ubuntu doesn't recognize the RAID
>  drives, it corrupts one of them when it installs the default boot
>  loader. To even figure out that the installer is doing this I need to
>  look in the advanced settings.
>
>  It would help to have a complete list of all disk operations that will
>  be performed (and the affected drives or partitions) clearly shown on
>  the the final Ready to Install screen.
>
>  Another solution would be for Ubuntu to better support the Intel ICH9R
>  controller that I happen to be using, but I think that is only a
>  partial solution because the next software raid system that Intel
>  comes up with might not be supported (and I think that adding better
>  support for Intel raid is already on the wishlist).
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/parted/+bug/191119/+subscriptions
>

Revision history for this message

Phillip Susi (psusi) wrote on 2011-12-10:

#24

Yes, it has been at least as far back as 9.04, I thought all the way back to 8.04, but I could be mistaken.

Revision history for this message

Greg (gregcouch) wrote on 2011-12-10:

#25

Thank you, I will try it again.

-- Greg

On Sat, Dec 10, 2011 at 9:06 AM, Phillip Susi <email address hidden> wrote:

> Yes, it has been at least as far back as 9.04, I thought all the way
> back to 8.04, but I could be mistaken.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/191119
>
> Title:
> Installer corrupts raid drive
>
> Status in “parted” package in Ubuntu:
> Expired
>
> Bug description:
> Boot from standard 8.04 Desktop CD.
> Run installer, select default language, keyboard to get to Partitioner.
> Select Manual install
> Note that raid drives from booting windows are listed separately.
> Select partition on alternate, non raid, drive to install Ubuntu on.
> Edit partition and set mount point to "/"
> Set login, computer name, etc. and install.
> Reboot computer.
> Note that BIOS now shows first drive as not a raid member.
> Run Windows.
> Note that software needs to rebuild RAID 1 mirror.
> Reboot computer.
> Note that BIOS shows first drive as valid raid member again.
>
> The problem I have is that because Ubuntu doesn't recognize the RAID
> drives, it corrupts one of them when it installs the default boot
> loader. To even figure out that the installer is doing this I need to
> look in the advanced settings.
>
> It would help to have a complete list of all disk operations that will
> be performed (and the affected drives or partitions) clearly shown on
> the the final Ready to Install screen.
>
> Another solution would be for Ubuntu to better support the Intel ICH9R
> controller that I happen to be using, but I think that is only a
> partial solution because the next software raid system that Intel
> comes up with might not be supported (and I think that adding better
> support for Intel raid is already on the wishlist).
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/parted/+bug/191119/+subscriptions
>

Thank you, I will try it again.

-- Greg

On Sat, Dec 10, 2011 at 9:06 AM, Phillip Susi <psusi@cfl.rr.com> wrote:

> Yes, it has been at least as far back as 9.04, I thought all the way
> back to 8.04, but I could be mistaken.
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/191119
>
> Title:
>  Installer corrupts raid drive
>
> Status in “parted” package in Ubuntu:
>   Expired
>
> Bug description:
>  Boot from standard 8.04 Desktop CD.
>  Run installer, select default language, keyboard to get to Partitioner.
>  Select Manual install
>    Note that raid drives from booting windows are listed separately.
>    Select partition on alternate, non raid, drive to install Ubuntu on.
>  Edit partition and set mount point to "/"
>  Set login, computer name, etc. and install.
>  Reboot computer.
>    Note that BIOS now shows first drive as not a raid member.
>  Run Windows.
>    Note that software needs to rebuild RAID 1 mirror.
>  Reboot computer.
>    Note that BIOS shows first drive as valid raid member again.
>
>  The problem I have is that because Ubuntu doesn't recognize the RAID
>  drives, it corrupts one of them when it installs the default boot
>  loader. To even figure out that the installer is doing this I need to
>  look in the advanced settings.
>
>  It would help to have a complete list of all disk operations that will
>  be performed (and the affected drives or partitions) clearly shown on
>  the the final Ready to Install screen.
>
>  Another solution would be for Ubuntu to better support the Intel ICH9R
>  controller that I happen to be using, but I think that is only a
>  partial solution because the next software raid system that Intel
>  comes up with might not be supported (and I think that adding better
>  support for Intel raid is already on the wishlist).
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/ubuntu/+source/parted/+bug/191119/+subscriptions
>

Ubuntu
parted package

Installer corrupts raid drive

Bug Description

Duplicates of this bug

Other bug subscribers

Remote bug watches

Ubuntuparted package

Installer corrupts raid drive

Bug Description

Duplicates of this bug

Other bug subscribers

Remote bug watches

Ubuntu
parted package