open-iscsi duplicates exsiting block device on initiator login

[iMac]

Binary package hint: open-iscsi

I have setup Maverick as a domU, running standard linux-image-2.6.35-23-server from current stable+updates+proposed.

The kernel sets up the disks as sda and sdb via blkfront rather than xvda and xvdb I have seen on similarly configured Debian 6 domUs. The host system is running Debian 6, and loads this maverick system via pygrub+grub-pc.

I try to connect to an iscsi target to attach my 2TB ocfs2 filesystem and the iscsi driver attempts to use sda which is already in use, resulting in an sysfs kernel error:

sysfs: cannot create duplicate filename '/class/block/sda'

The details are in the attached dmesg. My fstab contains only UUID references, and my udev foo is not good enough to workaround. I tried rules similar to the following to move the xen devices, however it just created additional non-symlinked devices that were identical named xvda,xvdb.

KERNEL=="sda", SUBSYSTEMS=="xen", NAME="xvda"

I expect this might be an issue with blkfront (linux-image) or udev, however it is as a result of issuing the initiator login, so I'll start here.

ProblemType: Bug
DistroRelease: Ubuntu 10.10
Package: open-iscsi 2.0.871-0ubuntu5
ProcVersionSignature: Ubuntu 2.6.35-23.37-server 2.6.35.7
Uname: Linux 2.6.35-23-server x86_64
Architecture: amd64
Date: Mon Nov 8 16:01:51 2010
ProcEnviron:
 LANG=C
 SHELL=/bin/bash
SourcePackage: open-iscsi

[iMac (imac-netstatz)]

Same issue with linux-image-2.6.35-23-virtual and linux-image-2.6.35-23-server. Looks like xenpv + iscsi is likely a problem for all Ubuntu flavors.

[iMac (imac-netstatz)]

Similar dmesg with the virtual kernel shown here. A hard power off (xm destroy) is the only way to restart the system.

[ 704.493694] scsi 0:0:0:0: Direct-Access IET VIRTUAL-DISK 0 PQ: 0 ANSI: 4
[ 704.493901] sd 0:0:0:0: Attached scsi generic sg0 type 0
[ 704.505016] sd 0:0:0:0: [sda] 3907018752 512-byte logical blocks: (2.00 TB/1.81 TiB)
[ 704.505139] sd 0:0:0:0: [sda] Write Protect is off
[ 704.505144] sd 0:0:0:0: [sda] Mode Sense: 77 00 00 08
[ 704.505384] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[ 704.505421] ------------[ cut here ]------------
[ 704.505429] WARNING: at /build/buildd/linux-2.6.35/fs/sysfs/dir.c:451 sysfs_add_one+0xc5/0x130()
[ 704.505433] sysfs: cannot create duplicate filename '/class/block/sda'
[ 704.505437] Modules linked in: crc32c iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi parport_pc ppdev binfmt_misc lp parport xen_fbfront joydev fb_sys_fops sysimgblt xen_kbdfront sysfillrect syscopyarea
[ 704.505472] Pid: 2036, comm: async/0 Not tainted 2.6.35-23-virtual #40-Ubuntu
[ 704.505476] Call Trace:
[ 704.505484] [<ffffffff810607df>] warn_slowpath_common+0x7f/0xc0
[ 704.505491] [<ffffffff81006b3d>] ? xen_force_evtchn_callback+0xd/0x10
[ 704.505497] [<ffffffff810608d6>] warn_slowpath_fmt+0x46/0x50
[ 704.505503] [<ffffffff811bf255>] sysfs_add_one+0xc5/0x130
[ 704.505509] [<ffffffff811bfaab>] sysfs_do_create_link+0x13b/0x210
[ 704.505514] [<ffffffff811bfbb3>] sysfs_create_link+0x13/0x20
[ 704.505522] [<ffffffff81384d2c>] device_add_class_symlinks+0x6c/0x100
[ 704.505527] [<ffffffff81385c58>] device_add+0x238/0x440
[ 704.505534] [<ffffffff811b6021>] register_disk+0x41/0x180
[ 704.505541] [<ffffffff812ab12c>] add_disk+0x8c/0x160
[ 704.505547] [<ffffffff813d6a9b>] sd_probe_async+0x12b/0x220
[ 704.505553] [<ffffffff810865d0>] ? async_thread+0x0/0xf0
[ 704.505558] [<ffffffff810864aa>] run_one_entry+0x7a/0x1a0
[ 704.505564] [<ffffffff8108662d>] async_thread+0x5d/0xf0
[ 704.505569] [<ffffffff81056be0>] ? default_wake_function+0x0/0x20
[ 704.505575] [<ffffffff8107eaf6>] kthread+0x96/0xa0
[ 704.505581] [<ffffffff8100aee4>] kernel_thread_helper+0x4/0x10
[ 704.505587] [<ffffffff8100a313>] ? int_ret_from_sys_call+0x7/0x1b
[ 704.505593] [<ffffffff815a4e9d>] ? retint_restore_args+0x5/0x6
[ 704.505599] [<ffffffff8100aee0>] ? kernel_thread_helper+0x0/0x10
[ 704.505603] ---[ end trace dd350111cd760489 ]---

[iMac (imac-netstatz)]

This problem is resolved for me in Natty 11.04. Xen pv_ops kernels use xvda, so there is no longer a conflict for sda during initiator login.

[clayg (clay-gerrard)]

I've run into the same issue with a lucid guest running on XCP. Everything I can find says that the first block device *should* be xvda, but mine is sda, and that causes the open-iscsi to sysfs error.

Is there a way to force xen or this udev to move the block device out of the way?

# udevadm info --query=all --name=/dev/sda
P: /devices/vbd-51712/block/sda
N: sda
S: block/202:0
S: disk/by-path/xen-vbd-51712
E: UDEV_LOG=3
E: DEVPATH=/devices/vbd-51712/block/sda
E: SUBSYSTEM=block
E: DEVNAME=sda
E: ID_PATH=xen-vbd-51712
E: ID_PART_TABLE_TYPE=dos
E: MAJOR=202
E: MINOR=0
E: DEVTYPE=disk
E: DEVLINKS=/dev/block/202:0 /dev/disk/by-path/xen-vbd-51712

[iMac (imac-netstatz)]

I tried udev stuff initially, before the fix in Natty, but I couldn't get around this issue. Maybe my udev foo is not strong enough. It seemed to me that if udev actually had a say, then the order would be /dev/sda /dev/sdb. I think the issue was the kernel implementation of the hypervisor disk. It might be worthwhile looking at the downstream Natty patch that fixed the issue and seeing if it can be applied to the Lucid kernel source.

[clayg (clay-gerrard)]

I was mistaken, I'm only seeing this problem on maverick. FWIW running lucid or natty on domU moves the first hd to /dev/xvda and iscsi works just fine.

Also if I boot my maverick domU via HVM-boot-policy="BIOS order" instead of pygrub, the first hd still goes to /dev/sda - but iscsi initiator will mount the first target onto /dev/sdb instead of conflicting.

More information regarding the resolution in nattry: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/684875