Ubuntu1804 installer is not able to detect SAN disks(mulitpath disks) with 'No disk drive was detected' message.

[bugproxy]

Problem Description:
===================
Ubuntu1804 installer is not able to detect SAN disks(mulitpath disks) with 'No disk drive was detected' message.

Steps to re-create:
==================

> on Witherspoon test system, tried to install latest ubuntu1804 build using netboot install.

> Selected appropriate fields and then when it entres detect disk screen

> It throws below messgae

  lqqqqqqqqqqqqqqqqqqqqqqqqqqu [!] Detect disks tqqqqqqqqqqqqqqqqqqqqqqqqqqqk
  x x
  x No disk drive was detected. If you know the name of the driver needed x
  x by your disk drive, you can select it from the list. x
  x x
  x Driver needed for your disk drive: x
  x x
  x continue with no disk drive x
  x login to iSCSI targets x
  x cryptoloop x
  x ipr x
  x virtio_scsi x
  x none of the above x
  x x
  x <Go Back> x
  x x
  mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

> Installer not able to detect the SAN disk(mulitpath disks) and so not able to proceed with installation.

> Unable to proceed with ubutnu1804 installation as installer not able to detect the disk (SAN disks) using netboot install method

> Attached /var/log/syslog messages from installer

Attached /var/log/syslog files from installer shell prompt

== Comment: #13 - Frederic Bonnard <email address hidden> - 2018-04-30 09:00:21 ==
Hi,
- did you install install with "disk-detect/multipath/enable=true" ? ( https://help.ubuntu.com/lts/serverguide/multipath-setting-up-dm-multipath.html.en )
- check /var/log/syslog to see you didn't fail to download any udeb related to disk management and make sure there isn't any error .
- check availability of disks with lsblk or "find /dev/disk/ -ls"

F.

== Comment: #14 - INDIRA P. JOGA <email address hidden> - 2018-05-02 04:55:55 ==
(In reply to comment #12)
> Hi Indira, thanks for the bug. I have some questions I would like to
> understand better:
>
> What are the device subsystem that this disks are connected to? Is it a
> Fibre Channel connection?
>
> Do you have explicit two connections to the same disk? I.e, you would see
> two disks that the multipathd would need to detect that they are the same,
> or, your subsystem would handle the multi path configuration?
>
> How many disks do you have?

Hi Breno,

System has both SAS card with mulitpath & SAN card with mulitpath.
Yes we have two connections to same disk.

FYI -

SAN -
mpathb (3600507680183050d28000000000002a3) dm-1 IBM,2145
size=100G features='1 queue_if_no_path' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| |- 2:0:1:1 sdaa 65:160 active ready running
| `- 3:0:1:1 sdag 66:0 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  |- 2:0:0:1 sdx 65:112 active ready running
  `- 3:0:0:1 sdad 65:208 active ready running

SAS -
mpathl (35000c50076564663) dm-35 IBM,ST300MP0064
size=279G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 1:0:8:0 sdi 8:128 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
  `- 1:0:19:0 sds 65:32 active ready running

== Comment: #15 - INDIRA P. JOGA <email address hidden> - 2018-05-02 05:01:21 ==
(In reply to comment #13)
> Hi,
> - did you install install with "disk-detect/multipath/enable=true" ? (
> https://help.ubuntu.com/lts/serverguide/multipath-setting-up-dm-multipath.
> html.en )
> - check /var/log/syslog to see you didn't fail to download any udeb related
> to disk management and make sure there isn't any error .
> - check availability of disks with lsblk or "find /dev/disk/ -ls"
>
> F.

Yes i have tried both with multipath enable true & without multipath in both cases it failed. when problem reported guest has 3 disks and still not able to detect those disks.

== Comment: #19 - Breno Leitao <email address hidden> - 2018-05-03 09:02:54 ==
Let's get this mirrored to canonical for their awareness.

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

Default Comment by Bridge

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-05-04 03:55 EDT-------
Hi,
I tried with the bionic iso and a virtual machine with a multipath setup based on backend file and everything went fine :
---
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none' io='native'/>
<source file='/home/fbonnard/ubuntuimage'/>
<backingStore/>
<target dev='sda' bus='scsi'/>
<serial>0001</serial>
<alias name='scsi0-0-0-0'/>
<address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>
<disk type='file' device='disk'>
<driver name='qemu' type='raw' cache='none' io='native'/>
<source file='/home/fbonnard/ubuntuimage'/>
<backingStore/>
<target dev='sdb' bus='scsi'/>
<serial>0001</serial>
<alias name='scsi0-0-0-1'/>
<address type='drive' controller='0' bus='0' target='0' unit='1'/>
</disk>
---

I passed disk-detect/multipath/enable=true to the installer and everything went fine :
root@vm80:/home/ubuntu# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 3G 0 disk
??sda1 8:1 0 7M 0 part
??sda2 8:2 0 3G 0 part
??mpatha 253:0 0 3G 0 mpath
??mpatha-part1 253:1 0 7M 0 part
??mpatha-part2 253:2 0 3G 0 part /
sdb 8:16 0 3G 0 disk
??sdb1 8:17 0 7M 0 part
??sdb2 8:18 0 3G 0 part
??mpatha 253:0 0 3G 0 mpath
??mpatha-part1 253:1 0 7M 0 part
??mpatha-part2 253:2 0 3G 0 part /

root@vm80:/home/ubuntu# multipath -ll
mpatha (0QEMU_QEMU_HARDDISK_0001) dm-0 QEMU,QEMU HARDDISK
size=3.0G features='0' hwhandler='0' wp=rw
|-+- policy='service-time 0' prio=1 status=active
| `- 0:0:0:0 sda 8:0 active ready running
`-+- policy='service-time 0' prio=1 status=enabled
`- 0:0:0:1 sdb 8:16 active ready running
root@vm80:/home/ubuntu# ls -ltr /dev/mapper/
total 0
crw------- 1 root root 10, 236 May 4 08:40 control
lrwxrwxrwx 1 root root 7 May 4 08:40 mpatha -> ../dm-0
lrwxrwxrwx 1 root root 7 May 4 08:40 mpatha-part1 -> ../dm-1
lrwxrwxrwx 1 root root 7 May 4 08:40 mpatha-part2 -> ../dm-2

root@vm80:/home/ubuntu# mount |grep ext4
/dev/mapper/mpatha-part2 on / type ext4 (rw,relatime,errors=remount-ro,data=ordered)

[Steve Langasek (vorlon)]

What is the Linux kernel driver required for accessing the disks in question?

This doesn't appear to be a multipath issue at all, but simply an issue of the right driver not being available in the netboot installer.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-05-04 11:16 EDT-------
Right vorlon.
Actually this report is a bit unclear : 2 types of systems were tried : WitherSpoon system as well as VM with 2 installation methods : netboot and cdrom .

It's reported that it failed with netboot then succeeded later on (so I suspected some driver udeb not downloaded/available at this point).
But then Chethan reported : "The things are not working still w.r.t boslcp3 KVM guest. We tried to define a new guest and used both net/cdrom install method, it did not succeed"
which I tried (VM/cdrom) and this worked for me.

So Indira/Chethan, to avoid network issues or daily netboot instabilities, could you please try the cdrom
installation method, and let us know (in particular on a VM, which I can more easily reproduce here) ?
typically boslcp3g1 xml and cdrom installation.

Compared to what I tried, in your qemu guest boslcp3g1 xml, the disk devices specified point to /dev/disk/by-id/dm-uuid-part[1-3]-mpath-3600507680183050d28000000000002a4 and whether the bus is virtio or scsi, both should be supported : in this case the disks are seen through the VM's controller not the host which is supported by the bionic iso from my tests : I tried both configurations and they worked : disks are seen.
Now it's just that with scsi, a multipath device is detected directly, which was not the case with virtio, but disks are still seen (maybe I miss some more configuration).

Here are my virtio and scsi xml as well

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2018-05-04 11:17 EDT-------

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2018-05-04 11:18 EDT-------

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-05-12 05:33 EDT-------
(In reply to comment #24)
> Right vorlon.
> Actually this report is a bit unclear : 2 types of systems were tried :
> WitherSpoon system as well as VM with 2 installation methods : netboot and
> cdrom .
>
> It's reported that it failed with netboot then succeeded later on (so I
> suspected some driver udeb not downloaded/available at this point).
> But then Chethan reported : "The things are not working still w.r.t boslcp3
> KVM guest. We tried to define a new guest and used both net/cdrom install
> method, it did not succeed"
> which I tried (VM/cdrom) and this worked for me.
>
> So Indira/Chethan, to avoid network issues or daily netboot instabilities,
> could you please try the cdrom
> installation method, and let us know (in particular on a VM, which I can
> more easily reproduce here) ?
> typically boslcp3g1 xml and cdrom installation.
>
> Compared to what I tried, in your qemu guest boslcp3g1 xml, the disk devices
> specified point to
> /dev/disk/by-id/dm-uuid-part[1-3]-mpath-3600507680183050d28000000000002a4
> and whether the bus is virtio or scsi, both should be supported : in this
> case the disks are seen through the VM's controller not the host which is
> supported by the bionic iso from my tests : I tried both configurations and
> they worked : disks are seen.
> Now it's just that with scsi, a multipath device is detected directly, which
> was not the case with virtio, but disks are still seen (maybe I miss some
> more configuration).
>
> Here are my virtio and scsi xml as well

Hi Federic,

I am facing issues with netboot installation on kvm guests, example boslcp3g1 guest which is using SAN disk (mulitpath disks).I have tried CDROM installation & installer is able to detect the disks properly.

To recreate the issue we can go with boslcp3g1 guest. Currently boslcp3 host is running with 4 guests already. Will try to bring down one guest & work on recreation using boslcp3g1 guest.

Regards,
Indira

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-05-14 04:52 EDT-------
Hi Indira,
thanks for the feedback. Let me know about your netboot tests.

F.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-05-16 09:42 EDT-------
(In reply to comment #28)
> Hi Indira,
> thanks for the feedback. Let me know about your netboot tests.
>
> F.

Hi Federic,

I am able to see the issue on boslcp3g1 guest as below. Tried netboot installation today on boslcp3g1 guest and installer fails to detect disks as below. Guest is using SAN disks ( multipath disks - 3 disks are used in guest xml)

lqqqqqqqqqqqqqqqqqqqqqqqqqqu [!] Detect disks tqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x x
x No disk drive was detected. If you know the name of the driver needed x
x by your disk drive, you can select it from the list. x
x x
x Driver needed for your disk drive: x
x x
x continue with no disk drive x
x login to iSCSI targets x
x cryptoloop x
x virtio_scsi x
x none of the above x
x x
x <Go Back> x
x x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

Can you please look into the issue.

Regards,
Indira

------- Comment From <email address hidden> 2018-05-16 09:42 EDT-------
Attached guest xml- boslcp3g1

[bugproxy (bugproxy)]

------- Comment (attachment only) From <email address hidden> 2018-05-16 09:49 EDT-------

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2018-05-16 10:36 EDT-------
I am able to install other guests which are using SAS disks /qemu images using netboot installer method.

------- Comment From <email address hidden> 2018-05-16 10:44 EDT-------
Hi Indira,
after connecting to the vm, I see that the kernel that the installer runs is 4.15.0-15
which is not the one used by the latest netboot which uses 4.15.0-20.
Also, the logs show "no packages matching running kernel 4.15.0-13-generic in archive"
which is not a good sign as it fails to find several kernel udebs that may lead to missing support.

Could you try with the latest netboot ?
Thanks,

F.

[Andrew Cloke (andrew-cloke)]

Please let us know if this issue is persisting with the latest netboot.

[Manoj Iyer (manjo)]

Please try http://ports.ubuntu.com/ubuntu-ports/dists/bionic-updates/main/installer-ppc64el/current/images/netboot/ and report back if the issue is now fixed. I tested on a Power9 boston system and it was able to detect disks. Please test with your SAN setup.

[Andrew Cloke (andrew-cloke)]

After discussing with IBM, lowering to a medium priority.

[bugproxy (bugproxy)]

------- Comment From <email address hidden> 2019-02-07 10:24 EDT-------
No updates from bug reporter since May 2018. Closing as insufficient data...