disk-detect/s390-dasd/s390-zfcp: Restructure installer and put DASD and FCP configuration into disk detection

Bug #1559193 reported by bugproxy on 2016-03-18
16
This bug affects 2 people
Affects Status Importance Assigned to Milestone
hw-detect (Debian)
Fix Released
Unknown
hw-detect (Ubuntu)
Undecided
Dimitri John Ledkov
s390-dasd (Debian)
Fix Released
Unknown
s390-dasd (Ubuntu)
Undecided
Dimitri John Ledkov
s390-zfcp (Debian)
Fix Released
Unknown
s390-zfcp (Ubuntu)
Undecided
Dimitri John Ledkov

Bug Description

== Comment: #1 - Hendrik Brueckner <email address hidden> - 2016-03-18 09:23:45 ==
On Linux on z Systems, there are two major disk storage environments,
the direct-attached storage disk (DASD) and SCSI over Fibre-Channel.

There are the s390-dasd and s390-zfcp d-i modules to configure and
enable DASDs and FCP devices. Note that on s390x, disks are not
available by default to Linux and must be enabled in advance.

The s390-dasd and s390-zfcp both provide the harddrive-detection
dependency and, thus, each could fulfill the dependency to silenty
ignore the other. That behavior does not allow to mix DASDs and
SCSI disk on single installation (except you call both manually,
for example, in the expert mode).

To improve and provide a "guided" flow, I have split the harddrive
detection dependency for s390-dasd and s390-zfcp as follows:

- s390-dasd provides harddrive-detection-dasd
- s390-zfcp provides harddrive-detection-zfcp

disk-detect depends on
  -> harddrive-detection-dasd
  -> harddrive-detection-zfcp

and continues to provide the harddrive-detection.

With this split, the guided installation will install disk-detect to
solve the harddrive-detection dependency. In turn, disk-detect will
then rely on the s390-dasd and s390-zfcp d-i modules to provide DASD
and FC-attached SCSI disks. If both modules fail, the user perceives
the default disk-detect behavior, for example, users might configure
iSCSI.

The other nice benefit of this dependency split is the seamlessly
enablement of multipath with disk-detect (through preseeding).
For s390, multipathing should be always considered when SCSI is used.
I probably will extend the s390-zfcp module to set disk-detect's
multipath debconf variable for usability.

There are already Debian bug reports with patches to provide this solution:

disk-detect: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818586
s390-dasd: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818591
s390-zfcp: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=818592

bugproxy (bugproxy) on 2016-03-18
tags: added: architecture-s39064 bugnameltc-139323 severity-high targetmilestone-inin1604
Changed in ubuntu:
assignee: nobody → Skipper Bug Screeners (skipper-screen-team)

Thank you for taking the time to report this bug and helping to make Ubuntu better. It seems that your bug report is not filed about a specific source package though, rather it is just filed against Ubuntu in general. It is important that bug reports be filed about source packages so that people interested in the package can find the bugs about it. You can find some hints about determining what package your bug might be about at https://wiki.ubuntu.com/Bugs/FindRightPackage. You might also ask for help in the #ubuntu-bugs irc channel on Freenode.

To change the source package that this bug is filed about visit https://bugs.launchpad.net/ubuntu/+bug/1559193/+editstatus and add the package name in the text box next to the word Package.

[This is an automated message. I apologize if it reached you inappropriately; please just reply to this message indicating so.]

tags: added: bot-comment
dann frazier (dannf) on 2016-03-18
Changed in ubuntu:
assignee: Skipper Bug Screeners (skipper-screen-team) → Dimitri John Ledkov (xnox)
affects: ubuntu → hw-detect (Ubuntu)
Changed in s390-dasd (Ubuntu):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in s390-zfcp (Ubuntu):
assignee: nobody → Dimitri John Ledkov (xnox)
Changed in hw-detect (Debian):
status: Unknown → New
Changed in s390-dasd (Debian):
status: Unknown → New
Changed in s390-zfcp (Debian):
status: Unknown → New
tags: added: block-proposed
Dimitri John Ledkov (xnox) wrote :

Both s390-dasd and s390-zfcp are priority standard, and thus are always installed in d-i.

I do not think that creating two virtual provides, and changing hw-detect from arch:all -> arch:any is required.

The only effective changes are bumping up the XB-Installer-Menu-Item level, which has already been done in Ubuntu for s390-dasd, but not for s390-zfcp.

Let me fix s390-zfcp installer priority, and i believe that would then resolve the whole lot, i.e. dasd/zfcp will be auto-discovered if any are present.

Changed in hw-detect (Ubuntu):
status: New → Invalid
Changed in s390-dasd (Ubuntu):
status: New → Invalid
Changed in s390-zfcp (Ubuntu):
status: New → Fix Committed
tags: removed: block-proposed

------- Comment From <email address hidden> 2016-03-31 08:32 EDT-------
Hi Dimitri,

(In reply to comment #6)
> Both s390-dasd and s390-zfcp are priority standard, and thus are always
> installed in d-i.

Whether they are installed automatically or later through anna is not relevant
to the root problem.

>
> I do not think that creating two virtual provides,

This is required to be automatically triggered by disk-detect to solve the "virtual dependency". Apart from the package dependency, the installer uses this dependency to define required pre-reqs in the installation workflow. That means, disk-detect provides harddrive-detection to detect disks for partitioning. The virtual dependency for s390-dasd, s390-zfcp provides dasd and zfcp disk detection. If both provide harddrive-detection, one is actually sufficient to proceed the installation. In the worst case, a user can only configure DASDs, but not zfcp.

> and changing hw-detect from arch:all -> arch:any is required.

I had to change this to any because of some build errors caused by the s390x-specific dependencies.

>
> The only effective changes are bumping up the XB-Installer-Menu-Item level,
> which has already been done in Ubuntu for s390-dasd, but not for s390-zfcp.

I think that this influences just how they appear in the list of installer menu (apart from the waypoints of the base-installer).

>
> Let me fix s390-zfcp installer priority, and i believe that would then
> resolve the whole lot, i.e. dasd/zfcp will be auto-discovered if any are
> present.

Launchpad Janitor (janitor) wrote :

This bug was fixed in the package s390-zfcp - 1.0.0ubuntu1

---------------
s390-zfcp (1.0.0ubuntu1) xenial; urgency=medium

  * Bump s390-zfcp menu item, to be before disk-detect, similar to
    s390-dasd. LP: #1559193

 -- Dimitri John Ledkov <email address hidden> Thu, 31 Mar 2016 13:16:57 +0100

Changed in s390-zfcp (Ubuntu):
status: Fix Committed → Fix Released
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-04-01 09:19 EDT-------
Obviously some of the changes are breaking the zfcp installation now. Installer (no DASDs, but zfcp devices) dropped into the disk-detect menu, without asking for zfcp device initialization. See LTC bug 139913 which is being mirrored.

Changed in hw-detect (Debian):
status: New → Fix Released
Changed in s390-dasd (Debian):
status: New → Fix Released
Changed in s390-zfcp (Debian):
status: New → Fix Released
Changed in hw-detect (Ubuntu):
status: Invalid → Confirmed
Changed in s390-dasd (Ubuntu):
status: Invalid → Confirmed
Changed in s390-zfcp (Ubuntu):
status: Fix Released → Confirmed
Dimitri John Ledkov (xnox) wrote :

s390-dasd & s390-zfcp synced; hw-detect merged and is in the unapproved quee. s390-dasd/zfcp should not land in release pocket without hw-detect package. Hence blocking proposed migration for the whole lot for now.

tags: added: block-proposed
Changed in s390-zfcp (Ubuntu):
status: Confirmed → Fix Committed
Changed in s390-dasd (Ubuntu):
status: Confirmed → Fix Committed
Changed in hw-detect (Ubuntu):
status: Confirmed → In Progress
tags: removed: block-proposed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package hw-detect - 1.117ubuntu1

---------------
hw-detect (1.117ubuntu1) xenial; urgency=low

  * Merge 1.117 from debian to pick up fix for LP: #1559193.

  * Resynchronise with Debian. Remaining changes:
    - Add an 'archdetect-deb' package, containing /usr/bin/archdetect. Add
      an archdetect(1) manual page.
    - Add support for activating fakeraid devices with mdadm.
    - disk-detect.sh: Do not check the kernel command line for any option
      to enable dmraid support. If functional dmraid arrays are found, the
      user will be asked if they wish to activate them.
    - 'dmraid -c -s' changed its output format; cope with both old and new.
    - Make dmraid logging a bit neater.
    - Offer iSCSI targets for preseeding if partman-iscsi is available and
      no disk devices are found.
    - disk-detect.sh: Use mpath[a-z] now rather than mpath[0-9] to identify
      multipath devices and thus know to install partman-multipath.
    - disk-detect.sh: run update-dev after installing multipath and sg3.
    - debian/control: Depends on multipath-udeb (>= 0.5.0), which introduces
      the multipath device naming changes above.
    - Remove dm-emc from the multipath modules, since it's gone since
      2.6.27.
    - Improve checks for dm-* drivers being present to avoid relying on them
      being built as modules.
    - Bump question to load driver injection disk from medium to high, and
      raise driver-injection-disk package priority to standard. Thus the
      udeb will be loaded by default, but before installing any debs from
      the OEMDRV a confirmation question will be asked (default true, can be
      pre-seeded)
    - Drop priorities of a couple of ethdetect questions to medium.
    - Exit zero if you continue all the way through ethdetect's errors about
      having no network interfaces.
    - Remove FireWire Ethernet support.
    - disk-detect.sh: multipath: reload udev rules before re-add devices
      to guarantee all devices may get new udev properties
    - disk-detect.sh: multipath: deactivate LVM volumes before multipath
      discovery to unlock individual paths
    - disk-detect.sh: multipath: deactivate md arrays before multipath
      discovery to unlock individual paths
    - disk-detect.sh: use iscsi-start wrapper in partman-iscsi to be able to set
      a custom iSCSI initiatorName from the installer.
    - disk-detect.sh: load SCSI device handlers before SCSI low-level
      device drivers.

hw-detect (1.117) unstable; urgency=medium

  * Team upload

  [ Hendrik Brueckner ]
  * Improve and split harddrive detection into DASD and SCSI dependency
    on s390x (Closes: #818586)

hw-detect (1.116) unstable; urgency=medium

  [ Colin Watson ]
  * Use HTTPS for Vcs-* URLs, and link to cgit rather than gitweb.

hw-detect (1.115) unstable; urgency=medium

  [ Colin Watson ]
  * Compress devnames-static.gz using "gzip -n" (Lintian).

 -- Dimitri John Ledkov <email address hidden> Mon, 04 Apr 2016 14:37:02 +0100

Changed in hw-detect (Ubuntu):
status: In Progress → Fix Released
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-04-06 07:01 EDT-------
Installer does not fall into DASD nor into zfcp setup. Comes up with disk detect. Disk setup only possible after choosing "Go back". -> broken!
Will provide more detailed information later today.

tags: added: severity-critical
removed: severity-high
Dimitri John Ledkov (xnox) wrote :

yes, all of the requested changes are now in, and whilst both s390-dasd and s390-zfcp components are loaded in the d-i, none of those steps are executed.

Returning to main menu, and executing the steps does work. But obviously this is broken at the moment.

bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-04-06 09:47 EDT-------
Hi Dimitri,
when fixing this, please consider the following three cases:
1.) DASD(s) available, at least one configured
2.) DASD(s) available, but none configured
3.) no DASD available
in all three above cases the s390-zfcp installer module should be run, if at least one zfcp device is available.

See also LP1564912 (LTC 139913) and LP1557719 (LTC 139094).

Dimitri John Ledkov (xnox) wrote :

s390-dasd (0.0.36ubuntu1) xenial; urgency=medium

  * Run update-dev, to trigger device addition and hence loading of the
    dasd modules. Previously this was only loaded after hw/disk-detect
    which is too late, as this package step is already done at that point.

 -- Dimitri John Ledkov <email address hidden> Wed, 06 Apr 2016 16:57:36 +0100

s390-zfcp (1.0.2ubuntu1) xenial; urgency=medium

  * Run update-dev, to trigger device addition and hence loading of the
    zfcp module. Previously this was only loaded after hw/disk-detect
    which is too late, as this package step is already done at that point.

 -- Dimitri John Ledkov <email address hidden> Wed, 06 Apr 2016 17:15:34 +0100

Changed in s390-dasd (Ubuntu):
status: Fix Committed → Fix Released
Changed in s390-zfcp (Ubuntu):
status: Fix Committed → Fix Released
bugproxy (bugproxy) wrote :

------- Comment From <email address hidden> 2016-04-07 05:35 EDT-------
(In reply to comment #14)
> Hi Dimitri,
> when fixing this, please consider the following three cases:
> 1.) DASD(s) available, at least one configured
> 2.) DASD(s) available, but none configured
> 3.) no DASD available
> in all three above cases the s390-zfcp installer module should be run, if at
> least one zfcp device is available.
>
> See also LP1564912 (LTC 139913) and LP1557719 (LTC 139094).

Restructuring works fine now, as requested above, and DASD handling works also as expected.
Successfully verfied with installer version 445 + udeb packages from 2016-04-07 s390-dasd 0.0.36ubuntu1, s390-zfcp 1.0.2ubuntu1, multipath-udeb 0.5.0+git1.656f8865-5ubuntu2, disk-detect 1.117ubuntu1

Thanks a lot :-) Closed.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.