Configure partitions for RAID1, received: The installer needs to remove operating system files from the install target, but was unable to do so.

[Jeff Balderson]

Binary package hint: ubiquity

1) Netbooted my Netra X1 from /ubuntu/hardy/main/installer-sparc/20070308ubuntu31/images/sparc64/netboot/2.6/boot.img
2) configured my two internal drives as follows:

IDE1 master (hda) - 20.0 GB ST320413A
> #1 8.2 MB
> #2 641.6 MB K raid
> #4 4.1 GB K raid
> #5 510.0 MB K raid
> #6 3.1 GB K raid
> #7 5.1 GB K raid
> #8 6.6 GB K raid
IDE2 master (hdc) - 20.0 GB ST320413A
> #1 8.2 MB
> #2 641.6 MB K raid
> #4 4.1 GB K raid
> #5 510.0 MB K raid
> #6 3.1 GB K raid
> #7 5.1 GB K raid
> #8 6.6 GB K raid

3) selected "Configure software RAID"
4) Confirmed "Keep current partition layout and configure RAID?" with "Yes"
5) There is a pause, and then I receive:

"[!!] Partition disks
Failed to remove conflicting files
The installer needs to remove operating system files from the install target, but was unable to do so. The install cannot continue."

6) Click continue, and I receive:
"[!!] Partition disks
RAID configuration failure
An error occurred while writing the changes to the storage devices
RAID configuration has been aborted."

7) Click continue and it sends me back to the partitioner.

I know for a fact that I've used this exact same configuration with Gutsy, and possibly as far back as Dapper.

This is on a headless system, so I have to install via text mode and can't switch between consoles.

[Jeff Balderson (jbalders)]

I did a "save debug logs" from the main menu. I found the following interesting:

Mar 3 18:51:03 main-menu[27682]: (process:4769): mount:
Mar 3 18:51:03 main-menu[27682]: (process:4769): Mounting /dev/hda1 on /mnt/tmpmount failed
Mar 3 18:51:03 main-menu[27682]: (process:4769): : Invalid argument
Mar 3 18:51:03 main-menu[27682]: (process:4769): mount:
Mar 3 18:51:03 main-menu[27682]: (process:4769): Mounting /dev/hda1 on /mnt/tmpmount failed
Mar 3 18:51:03 main-menu[27682]: (process:4769): : Invalid argument
Mar 3 18:51:03 main-menu[27682]: INFO: Menu item 'partman-base' succeeded but requested to be left unconfigured.
Mar 3 18:51:03 main-menu[27682]: DEBUG: resolver (libnewt0.52): package doesn't exist (ignored)
Mar 3 18:51:03 main-menu[27682]: DEBUG: resolver (rdate-udeb): package doesn't exist (ignored)
Mar 3 18:51:03 main-menu[27682]: INFO: Falling back to the package description for console-setup-udeb
Mar 3 18:51:10 main-menu[27682]: INFO: Falling back to the package description for console-setup-udeb
Mar 3 18:51:10 main-menu[27682]: INFO: Menu item 'save-logs' selected

Note that /dev/hda1 is marked as "Do Not Use". Why is it trying to mount it?

[Jeff Balderson (jbalders)]

I think this may actually be a debian-installer problem, but I'll leave it up to the experts to decide.

[Jeff Balderson (jbalders)]

This (valid) partition layout also has problems:

IDE1 master (hda) - 20.0 GB ST320413A
> #1 8.2 MB
> #2 641.6 MB f ext3 /
> #4 4.1 GB f ext3 /usr
> #5 510.0 MB f ext3 /tmp
> #6 3.1 GB f swap swap
> #7 5.1 GB f ext3 /var
> #8 6.6 GB f ext3 /export
IDE2 master (hdc) - 20.0 GB ST320413A
> 20.0 GB FREE SPACE

I get a warning about SILO being installed on an "unknown" partition, if I continue through that, I get the same error as above.

Even this fails:

IDE1 master (hda) - 20.0 GB ST320413A
> #1 8.2 MB f ext3 /mnt/t1
> #2 641.6 MB K raid
> #4 4.1 GB K raid
> #5 510.0 MB K raid
> #6 3.1 GB K raid
> #7 5.1 GB K raid
> #8 6.6 GB K raid
IDE2 master (hdc) - 20.0 GB ST320413A
> #1 8.2 MB f ext3 /mnt/t2
> #2 641.6 MB K raid
> #4 4.1 GB K raid
> #5 510.0 MB K raid
> #6 3.1 GB K raid
> #7 5.1 GB K raid
> #8 6.6 GB K raid

Although, this time it's trying to mount /dev/hda6. (syslog attached)

[Jeff Balderson (jbalders)]

And in case you're wondering, the reason I leave /dev/hda1 and /dev/hdc1 unconfigured is due to the fact that RAID volumes will overwrite the Sun disklabel.

This configuration appears to work:
IDE1 master (hda) - 20.0 GB ST320413A
> #1 131.6 MB f ext3 /boot
> #2 510.0 MB f ext3 /
> #4 4.1 GB f ext3 /usr
> #5 510.0 MB f ext3 /tmp
> #6 3.1 GB f swap swap
> #7 5.1 GB f ext3 /var
> #8 6.6 GB f ext3 /export
IDE2 master (hdc) - 20.0 GB ST320413A
> 20.0 GB FREE SPACE

Unfortunately, the install bombs out with an invalid release signature. With as unstable as the repositories have been lately in that regard, I'm not sure whether this is a d-i issue or just the flakiness of the repositories. Either way, that's a separate issue from what I reported above.

[Colin Watson (cjwatson)]

I don't think this can possibly be a ubiquity bug, so rejecting that task. debian-installer is a good holding area until we figure out a more accurate component assignment.

[Colin Watson (cjwatson)]

Confirmed - we're definitely missing a few checks here. Ouch.

[Colin Watson (cjwatson)]

I've committed a fix for this and am having it reviewed. Once it's in the archive, if it would be possible for you to try to recreate this situation and verify that the bug is fixed, I'd much appreciate it; I suspect that this had other much more serious consequences that you were lucky to escape.

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-target - 54ubuntu1

---------------
partman-target (54ubuntu1) hardy; urgency=low

  [ Colin Watson ]
  * Resynchronise with Debian, largely to pick up checks for directories
    that must be on the root file system (LP: #148331). Remaining changes:
    - Use UUID= fstab syntax for all partitions if a UUID is available, and
      add a comment above each UUID to indicate the corresponding device at
      install time.
    - Disable automatic mounting of USB removable devices.
    - Don't use UUIDs in fstab for those device types that volumeid.postinst
      refuses to convert.
    - Mount CD-ROMs and floppies with 'exec'.
    - Use the path of the file associated with a loop device in /etc/fstab,
      rather than the filesystem's UUID or the loop device path.
    - Always set the loop option for loop devices.
    - Remove critical system files from the existing filesystem before
      installing.
    - Preserve the UID and GID of the initial user, if possible. Requires a
      patch to user-setup.
    - Mount CD-ROMs and floppies with the "utf8" option by default.
  * Don't clear partitions marked "do not use" (LP: #198106).

  [ Evan Dandrea ]
  * Clean up mountpoint check for bug fixing and readability. Thanks
    Colin Watson.

partman-target (54) unstable; urgency=low

  [ Colin Watson ]
  * Ensure that /bin, /etc, /lib*, and /sbin are on the root file system.

  [ Frans Pop ]
  * Add default value for mount_failed template (see #457901).

  [ Updated translations ]
  * Amharic (am.po) by tegegne tefera
  * Bulgarian (bg.po) by Damyan Ivanov
  * Catalan (ca.po) by Jordi Mallach
  * Czech (cs.po) by Miroslav Kure
  * German (de.po) by Jens Seidel
  * Dzongkha (dz.po) by Jurmey Rabgay
  * Esperanto (eo.po) by Serge Leblanc
  * Spanish (es.po) by Javier Fernández-Sanguino Peña
  * Basque (eu.po) by Piarres Beobide
  * Finnish (fi.po) by Esko Arajärvi
  * French (fr.po) by Christian Perrier
  * Galician (gl.po) by Jacobo Tarrio
  * Hebrew (he.po) by Lior Kaplan
  * Hindi (hi.po) by Kumar Appaiah
  * Hungarian (hu.po) by SZERVÁC Attila
  * Indonesian (id.po) by Arief S Fitrianto
  * Italian (it.po) by Stefano Canepa
  * Japanese (ja.po) by Kenshi Muto
  * Central Khmer (km.po) by Khoem Sokhem
  * Korean (ko.po) by Changwoo Ryu
  * Kurdish (ku.po) by Erdal Ronahi
  * Lithuanian (lt.po) by Kęstutis Biliūnas
  * Latvian (lv.po) by Viesturs Zarins
  * Malayalam (ml.po) by Praveen|പ്രവീണ്‍ A|എ
  * Norwegian Bokmål (nb.po) by Hans Fredrik Nordhaug
  * Nepali (ne.po) by Shyam Krishna Bal
  * Dutch (nl.po) by Frans Pop
  * Norwegian Nynorsk (nn.po) by Håvard Korsvoll
  * Panjabi (pa.po) by Amanpreet Singh Alam
  * Polish (pl.po) by Bartosz Fenski
  * Portuguese (Brazil) (pt_BR.po) by Felipe Augusto van de Wiel (faw)
  * Portuguese (pt.po) by Miguel Figueiredo
  * Romanian (ro.po) by Eddy Petrișor
  * Russian (ru.po) by Yuri Kozlov
  * Slovak (sk.po) by Peter Mann
  * Slovenian (sl.po) by Matej Kovacic
  * Swedish (sv.po) by Daniel Nylander
  * Tamil (ta.po) by Dr.T.Vasudevan
  * Thai (th.po) by Theppitak Karoonboonyanan
  * Turkish (tr.po) by Recai Oktaş
  * Ukrainian (uk.po)
  * Vietnamese (vi.po) by Clyti...

[Jeff Balderson (jbalders)]

Unfortunately, no improvement.

The only difference I can identify is that I can no longer mark partition 1 as "Do Not Use". I'm using this partition layout, which is similar to one I've tried above.

IDE1 master (hda) - 20.0 GB ST320413A
> #1 8.2 MB F ext3
> #2 641.6 MB K raid
> #4 4.1 GB K raid
> #5 510.0 MB K raid
> #6 3.1 GB K raid
> #7 5.1 GB K raid
> #8 6.6 GB K raid
IDE2 master (hdc) - 20.0 GB ST320413A
> #1 8.2 MB F ext3
> #2 641.6 MB K raid
> #4 4.1 GB K raid
> #5 510.0 MB K raid
> #6 3.1 GB K raid
> #7 5.1 GB K raid
> #8 6.6 GB K raid

Note that since I can't mark partition 1 as "Do Not Use", it's now an EXT3 and marked as "Do Not Mount". When I select "Configure software RAID", it gives me the usual "Are you sure you want to do this?" page, two progress bars quickly scroll across the page, and I receive the original "The installer needs to remove ..." message.

Syslog is attached, but I can't identify anything seemingly related. None of the other debug files seem to contain anything different from before and/or pertaining to this.

[Jeff Balderson (jbalders)]

Somehow I over looked this in the syslog:

Mar 4 15:21:39 main-menu[5516]: (process:5580): /lib/partman/choose_method/70dont_use/choices: Permission denied
Mar 4 15:21:39 main-menu[5516]: (process:5580):
Mar 4 15:21:39 main-menu[5516]: (process:5580): /lib/partman/active_partition/15method/do_option:
Mar 4 15:21:39 main-menu[5516]: (process:5580): /lib/partman/active_partition/15method/do_option: 10:
Mar 4 15:21:39 main-menu[5516]: (process:5580): /lib/partman/choose_method/70dont_use/choices: Permission denied
Mar 4 15:21:39 main-menu[5516]: (process:5580):
Mar 4 15:21:39 main-menu[5516]: (process:5580): mount:
Mar 4 15:21:39 main-menu[5516]: (process:5580): Mounting /dev/hda4 on /mnt/tmpmount failed
Mar 4 15:21:39 main-menu[5516]: (process:5580): : Invalid argument
Mar 4 15:21:39 main-menu[5516]: INFO: Menu item 'partman-base' succeeded but requested to be left unconfigured.

The "Mounting /dev/hda4 on /mnt/tmpmount failed" is similar to what I was receiving before. The "Permission denied" messages seem to be new.

[Jeff Balderson (jbalders)]

Performing:

chmod ugo+x /lib/partman/choose_method/70dont_use/choices

gets rid of the "Permission denied" problem, but doesn't seem to fix the original problem. I've poked around a bit but am still unclear as to why it's even attempting to mount the unconfigured RAID components.

[Sam Bashton (sam-bashton)]

I'm still seeing this issue on amd64 with the updated installer.

[Launchpad Janitor (janitor)]

This bug was fixed in the package partman-target - 54ubuntu2

---------------
partman-target (54ubuntu2) hardy; urgency=low

  * Don't clear partitions that aren't marked as using a filesystem.
  * Run clear_partitions from finish.d, not commit.d; commit.d is run in all
    sorts of places, such as while configuring RAID, where we don't want to
    clear partitions yet (LP: #198106).

 -- Colin Watson <email address hidden> Thu, 06 Mar 2008 13:24:59 +0000

[Jeff Balderson (jbalders)]

This appears to resolve my issue. I was able to get past partitioning and RAID configuration on a system with the original partition layout:

IDE1 master (hda) - 20.0 GB ST320413A
> #1 8.2 MB ("do not use")
> #2 641.6 MB K raid
> #4 4.1 GB K raid
> #5 510.0 MB K raid
> #6 3.1 GB K raid
> #7 5.1 GB K raid
> #8 6.6 GB K raid
IDE2 master (hdc) - 20.0 GB ST320413A
> #1 8.2 MB ("do not use")
> #2 641.6 MB K raid
> #4 4.1 GB K raid
> #5 510.0 MB K raid
> #6 3.1 GB K raid
> #7 5.1 GB K raid
> #8 6.6 GB K raid

/dev/md0 (hda2,hdc2 - ext3 - /)
/dev/md1 (hda4,hdc4 - ext3 - /usr)
/dev/md2 (hda5,hdc5 - ext3 - /tmp)
/dev/md3 (hda6,hdc6 - swap)
/dev/md4 (hda7,hdc7 - ext3 - /var)
/dev/md5 (hda8,hdc8 - ext3 - /export)

Thanks for working on the problem.

[Evan (ev)]

Sam,

Are you still seeing this in the latest daily CD build?

http://cdimage.ubuntu.com/daily/current/

Thanks!

[Sam Bashton (sam-bashton)]

I'm using the PXE installer as the test machine doesn't have a CD drive, but no, I'm not seeing the problem since Colin's update.

Let me know if you need me to test the CD installer too, I can find another machine to test on.

Thanks