Drives mistakenly reported as removable

Bug #1532062 reported by Robert Clark on 2016-01-08
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
curtin
Medium
Unassigned
curtin (Ubuntu)
Medium
Unassigned

Bug Description

I have multiple servers which report their drives are removable, even though the drives are the main drives in the device (/dev/sda, /dev/sdb, etc). One of such machines (an IBM system x3650 M1) I have pulled a report from lsblk and lshw below and included them. These are configured using the RAID controller that came with the machine. I can provide any other information needed. I am seeing the same on a HP proliant DL100 g2. In this case the removable tag is triggering a problem with curtin since curtin currently refuses to install to drives reported as removable by lsblk.

output of: lsblk -l
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 100G 0 disk
sda1 8:1 1 100G 0 part /
sdb 8:16 1 173.2G 0 disk
sdb1 8:17 1 172.2G 0 part /var/lib/ceph/osd/ceph-0
sdb2 8:18 1 1023M 0 part
sr0 11:0 1 1024M 0 rom

output of: sudo lshw -class disk
  *-disk:0
       description: SCSI Disk
       physical id: 0.0.0
       bus info: scsi@2:0.0.0
       logical name: /dev/sda
       size: 99GiB (107GB)
       capabilities: partitioned partitioned:dos
       configuration: sectorsize=512
  *-disk:1
       description: SCSI Disk
       physical id: 0.1.0
       bus info: scsi@2:0.1.0
       logical name: /dev/sdb
       size: 173GiB (186GB)
       capabilities: gpt-1.00 partitioned partitioned:gpt
       configuration: guid=66a282c8-da87-4502-902d-ce5c7b4da275 sectorsize=512
  *-disk:2 UNCLAIMED
       description: SCSI Disk
       physical id: 1.0.0
       bus info: scsi@2:1.0.0
  *-disk:3 UNCLAIMED
       description: SCSI Disk
       physical id: 1.1.0
       bus info: scsi@2:1.1.0
  *-cdrom
       description: DVD reader
       product: UJDA780 DVD/CDRW
       vendor: MATSHITA
       physical id: 0.0.0
       bus info: scsi@1:0.0.0
       logical name: /dev/cdrom
       logical name: /dev/sr0
       version: CA21
       capabilities: removable audio cd-r cd-rw dvd
       configuration: ansiversion=5 status=nodisc

Related branches

Scott Moser (smoser) wrote :

Just a note, 'RM' is what curtin is reading as 'removable'.
Not that this is definitive, but http://linoxide.com/linux-command/linux-lsblk-command/ indicates that RM means removable.

Scott Moser (smoser) wrote :

hm... actually, reading your 'lsblk' output above, those devices do *not* show as removable.

could you collect output of this:
lsblk --bytes --pairs --output=ALIGNMENT,DISC-ALN,DISC-GRAN,DISC-MAX,DISC-ZERO,FSTYPE,GROUP,KNAME,LABEL,LOG-SEC,MAJ:MIN,MIN-IO,MODE,MODEL,MOUNTPOINT,NAME,OPT-IO,OWNER,PHY-SEC,RM,RO,ROTA,RQ-SIZE,SIZE,STATE,TYPE,UUID

and then, also just get the output that curtin would find itself.
sudo python -c 'import json, curtin.block; print(json.dumps(curtin.block._lsblock(), indent=1))'

and
dpkg-query --show util-linux

thanks.

Robert Clark (returntoreptar) wrote :
Download full text (6.0 KiB)

Sure, output is below (sorry its a bit hard to structure it in an easy to read manner here):

lsblk --bytes --pairs --output=ALIGNMENT,DISC-ALN,DISC-GRAN,DISC-MAX,DISC-ZERO,FSTYPE,GROUP,KNAME,LABEL,LOG-SEC,MAJ:MIN,MIN-IO,MODE,MODEL,MOUNTPOINT,NAME,OPT-IO,OWNER,PHY-SEC,RM,RO,ROTA,RQ-SIZE,SIZE,STATE,TYPE,UUID

ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0" DISC-MAX="0" DISC-ZERO="0" FSTYPE="" GROUP="disk" KNAME="sda" LABEL="" LOG-SEC="512" MAJ:MIN="8:0" MIN-IO="512" MODE="brw-rw----" MODEL="main " MOUNTPOINT="" NAME="sda" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="107363696640" STATE="running" TYPE="disk" UUID=""
ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0" DISC-MAX="0" DISC-ZERO="0" FSTYPE="" GROUP="disk" KNAME="sda1" LABEL="" LOG-SEC="512" MAJ:MIN="8:1" MIN-IO="512" MODE="brw-rw----" MODEL="" MOUNTPOINT="/" NAME="sda1" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="107362631168" STATE="" TYPE="part" UUID=""
ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0" DISC-MAX="0" DISC-ZERO="0" FSTYPE="" GROUP="disk" KNAME="sdb" LABEL="" LOG-SEC="512" MAJ:MIN="8:16" MIN-IO="512" MODE="brw-rw----" MODEL="storage " MOUNTPOINT="" NAME="sdb" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="186004799488" STATE="running" TYPE="disk" UUID=""
ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0" DISC-MAX="0" DISC-ZERO="0" FSTYPE="" GROUP="disk" KNAME="sdb1" LABEL="" LOG-SEC="512" MAJ:MIN="8:17" MIN-IO="512" MODE="brw-rw----" MODEL="" MOUNTPOINT="/var/lib/ceph/osd/ceph-0" NAME="sdb1" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="184929992192" STATE="" TYPE="part" UUID=""
ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0" DISC-MAX="0" DISC-ZERO="0" FSTYPE="" GROUP="disk" KNAME="sdb2" LABEL="" LOG-SEC="512" MAJ:MIN="8:18" MIN-IO="512" MODE="brw-rw----" MODEL="" MOUNTPOINT="" NAME="sdb2" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="1072693760" STATE="" TYPE="part" UUID=""
ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0" DISC-MAX="0" DISC-ZERO="0" FSTYPE="" GROUP="cdrom" KNAME="sr0" LABEL="" LOG-SEC="512" MAJ:MIN="11:0" MIN-IO="512" MODE="brw-rw----" MODEL="UJDA780 DVD/CDRW" MOUNTPOINT="" NAME="sr0" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="1073741312" STATE="running" TYPE="rom" UUID=""

output of sudo python -c 'import json, curtin.block; print(json.dumps(curtin.block._lsblock(), indent=1))'
{
 "sdb2": {
  "GROUP": "disk",
  "OPT-IO": "0",
  "DISC-MAX": "0",
  "LABEL": "",
  "MODEL": "",
  "RQ-SIZE": "128",
  "MODE": "brw-rw----",
  "ROTA": "1",
  "RM": "1",
  "RO": "0",
  "device_path": "/dev/sdb2",
  "DISC-ZERO": "0",
  "UUID": "",
  "STATE": "",
  "MOUNTPOINT": "",
  "FSTYPE": "",
  "SIZE": "1072693760",
  "MAJ:MIN": "8:18",
  "DISC-GRAN": "0",
  "NAME": "sdb2",
  "LOG-SEC": "512",
  "DISC-ALN": "0",
  "ALIGNMENT": "0",
  "MIN-IO": "512",
  "OWNER": "root",
  "KNAME": "sdb2",
  "TYPE": "part",
  "PHY-SEC": "512"
 },
 "sdb1": {
  "GROUP": "disk",
  "OPT-IO": "0",
  "DISC-MAX": "0",
  "LABEL": "",
  "MODEL": "",
  "RQ-SIZE": "128",
  "MODE": "brw-rw--...

Read more...

Scott Moser (smoser) wrote :

ok, so this is wierd, lets focus on 'sda' and see if we can't prune this down.
From your statements in the bug opening:
$ lsblk -l
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 100G 0 disk
sda1 8:1 1 100G 0 part /

but then, when we do the longer 'lsblk --bytes --pairs --output=...' command in comment 3, we see:
$ lsblk --bytes --pairs --output=ALIGNMENT,DISC-ALN,DISC-GRAN,DISC-MAX,DISC-ZERO,FSTYPE,GROUP,KNAME,LABEL,LOG-SEC,MAJ:MIN,MIN-IO,MODE,MODEL,MOUNTPOINT,NAME,OPT-IO,OWNER,PHY-SEC,RM,RO,ROTA,RQ-SIZE,SIZE,STATE,TYPE,UUID

ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0" DISC-MAX="0" DISC-ZERO="0" FSTYPE="" GROUP="disk" KNAME="sda" LABEL="" LOG-SEC="512" MAJ:MIN="8:0" MIN-IO="512" MODE="brw-rw----" MODEL="main " MOUNTPOINT="" NAME="sda" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="107363696640" STATE="running" TYPE="disk" UUID=""
ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0" DISC-MAX="0" DISC-ZERO="0" FSTYPE="" GROUP="disk" KNAME="sda1" LABEL="" LOG-SEC="512" MAJ:MIN="8:1" MIN-IO="512" MODE="brw-rw----" MODEL="" MOUNTPOINT="/" NAME="sda1" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="107362631168" STATE="" TYPE="part" UUID=""

So it *seems* that in the --output= invocation, it reports they *are* removable, but not in the less verbose invocation.
Lets try to pair that down, can you try:
$ full=ALIGNMENT,DISC-ALN,DISC-GRAN,DISC-MAX,DISC-ZERO,FSTYPE,GROUP,KNAME,LABEL,LOG-SEC,MAJ:MIN,MIN-IO,MODE,MODEL,MOUNTPOINT,NAME,OPT-IO,OWNER,PHY-SEC,RM,RO,ROTA,RQ-SIZE,SIZE,STATE,TYPE,UUID
$ short="NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT"
$ lsblk --pairs "--output=$full" /dev/sda
$ lsblk --pairs "--output=$short" /dev/sda

and then without /dev/sda on it.

See if that shows RM= on any of those. It woudl seem like there must be buffer overflow or something.

Robert Clark (returntoreptar) wrote :

Sounds good, here is the output from: lsblk --pairs "--output=$full" /dev/sda
ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0B" DISC-MAX="0B" DISC-ZERO="0" FSTYPE="" GROUP="disk" KNAME="sda" LABEL="" LOG-SEC="512" MAJ:MIN="8:0" MIN-IO="512" MODE="brw-rw----" MODEL="main " MOUNTPOINT="" NAME="sda" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="100G" STATE="running" TYPE="disk" UUID=""
ALIGNMENT="0" DISC-ALN="0" DISC-GRAN="0B" DISC-MAX="0B" DISC-ZERO="0" FSTYPE="" GROUP="disk" KNAME="sda1" LABEL="" LOG-SEC="512" MAJ:MIN="8:1" MIN-IO="512" MODE="brw-rw----" MODEL="" MOUNTPOINT="/" NAME="sda1" OPT-IO="0" OWNER="root" PHY-SEC="512" RM="1" RO="0" ROTA="1" RQ-SIZE="128" SIZE="100G" STATE="" TYPE="part" UUID=""

And the output from lsblk --pairs "--output=$short" /dev/sda
NAME="sda" MAJ:MIN="8:0" RM="1" SIZE="100G" RO="0" TYPE="disk" MOUNTPOINT=""
NAME="sda1" MAJ:MIN="8:1" RM="1" SIZE="100G" RO="0" TYPE="part" MOUNTPOINT="/"

I may be misreading it, however it looks to me like the initial output is telling the same story, in lsblk -l it looks to me like it has the below structure with formatting, which seems to me that it is being reported wrong all the way around. That being said I don't have any experience with lsblk so I could be completely off.

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 1 100G 0 disk
sda1 8:1 1 100G 0 part /

Robert Clark (returntoreptar) wrote :

Sorry I should've known that wouldn't format correctly once I posted, I have tried to structure the lsblk -l output here: http://pastebin.com/hQktMbXg.

Scott Moser (smoser) wrote :

shoot. you're right. i was just reading columns wrong. I was reading the 'RO' as 'RM'.
thanks for your patience. At least lsblk is providing consitent information, so that is good.

Scott Moser (smoser) wrote :

Robert,
I took a quick gander at util-linux source, and it is just reading
  /sys/class/block/<KERNEL_NAME>/removable

There isnt' any util-linux specific knowledge that says "this is removable", but rather its just telling you what the kernel is telling it.

Would you mind posting/attaching:
 sudo lshw
?

I really suspect at this point that kernel is reporting "correctly", and that curtin probably has an incorrect idea of what "removable" means. So I'm goign to move this off util-linux and to curtin for now.

affects: util-linux (Ubuntu) → curtin (Ubuntu)
Changed in curtin:
status: New → Confirmed
Changed in curtin (Ubuntu):
status: New → Confirmed
Robert Clark (returntoreptar) wrote :

Sure no problem! I have attached the output of sudo lshw as a file, since it was a bit long.

Scott Moser (smoser) on 2016-01-11
Changed in curtin:
importance: Undecided → Medium
Changed in curtin (Ubuntu):
importance: Undecided → Medium
Phillip Susi (psusi) wrote :

I'm guessing that this particular scsi controller supports hot plug and so identifies the drives as removable since technically you can yank out the raid bays and replace them while the system is on. The whole concept of whether a drive is "removable" or not is a hand waving best guess that really needs to not be used for anything, so yea, the bug seems to be that curtain is looking at it in the first place.

Scott Moser (smoser) wrote :

more info here, and I'm going to merge Robert's change.
On a system the kernel team has, the value of 'removable' changed even from one version of a kernel to another.
$ uname -r
4.2.0-27-generic
$ cat /sys/class/block/sda/removable
0

But on 4.4.0-2-generic that same sysatem shows that drive as removable.

so, that just gives more evidence that its nto terribly useful.

Scott Moser (smoser) on 2016-02-08
Changed in curtin:
status: Confirmed → Fix Committed
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package curtin - 0.1.0~bzr351-0ubuntu1

---------------
curtin (0.1.0~bzr351-0ubuntu1) xenial; urgency=medium

  * New upstream snapshot.
    * partitioning: limited support for odd ordering of partition
      numbers (LP: #1543263). Specifically targetted at MAAS and
      powerVM support.
    * many upstream test improvements (LP: #1533770)
    * general upstream code improvements
    * use mkfs.vfat rather than mkfs.fat to support precise.
    * use removable devices for installation if no non-removable devices are
      found [Robert Clark] (LP: #1532062)
    * mkfs: fix for lack of uuid in btrfs tools on precise or trusty
    * added 'curtin mkfs' command for easily making filesystems.
    * mdadm: fix issues exposed by use via block_meta (LP: #1531520)
    * improvements and small bug fix for oauth on systems with bad clock
    * support bcache installation on precise
    * fix bug in install_grub to partition when storage_config
      provided. (LP: #1523779)
    * url_helper: raise import error on lack of oauth only when oauth used
    * block_meta: handle 'preserve' flag for raid devices (LP: #1522147)
    * close file descriptors from --config= arguments
    * xenial: disable update-motd during an apt-get update (LP: #1527710)
    * curthooks: know kernel mapping for xenial (4.4.0)
    * fix python executable selection when 'curtin --help' is called
    * subp: add decode parameter, defaulting to replace (LP: #1526127)
    * support passing an integer or valid float to human2bytes
    * Use /proc/mounts to find missing mountpoints

 -- Scott Moser <email address hidden> Fri, 12 Feb 2016 17:07:33 -0500

Changed in curtin (Ubuntu):
status: Confirmed → Fix Released

This bug is believed to be fixed in curtin in 17.1. If this is still a problem for you, please make a comment and set the state back to New

Thank you.

Changed in curtin:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers