nonexistent partition in /dev, & lsblk/disk util misbehaving with randomized disks

[DiagonalArg]

I have two 2T disks with no partition table and which have been filled with random data. GNU parted shows nothing on this disk:

GNU Parted 3.2
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Error: /dev/sda: unrecognised disk label
Model: ATA WDC WD2002FAEX-0 (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: unknown
Disk Flags:

but when I check with `lsblk`, I see one disk with partitions and the other without:

Code:
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 1.8T 0 disk
└─sda2 8:2 0 1.6T 0 part

NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdb 8:16 0 1.8T 0 disk

and the Disks utility agrees (see: http://ibin.co/2SQv4KbH06Mz).

Also, /dev/disk/by-id contains an entry pointing to sda and one pointing to sda2. So it really does think there's a second partition there.

Thinking there was something wrong, I re-filled the disks with random data. Now sda is properly (in the Disks utility as just an "unknown disk"), but sdb shows a second partition, and identically to how sda was showing it before (ie the Disks utility output is the same with 181GB at the beginning and 128GB at the end).

I am running all this from the Ubuntu Gnome 15.10 installer disk.
---
ApportVersion: 2.19.1-0ubuntu5
Architecture: amd64
CurrentDesktop: GNOME
DistroRelease: Ubuntu 15.10
InstallationDate: Installed on 2015-08-31 (130 days ago)
InstallationMedia: Ubuntu-GNOME 15.10 "Wily Werewolf" - Release amd64 (20151021)
Package: util-linux 2.26.2-6ubuntu3
PackageArchitecture: amd64
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 XDG_RUNTIME_DIR=<set>
 LANG=en_US.UTF-8
 SHELL=/bin/bash
ProcVersionSignature: Ubuntu 4.2.0-23.28-generic 4.2.6
Tags: wily
Uname: Linux 4.2.0-23-generic x86_64
UpgradeStatus: No upgrade log present (probably fresh install)
UserGroups: adm cdrom dip lpadmin plugdev sambashare sudo
_MarkForUpload: True

[DiagonalArg (diagonalarg)]

It's not really a linux utility problem, nor a gnome-disk-utility problem. Whatever is identifying devices is making a mistake.

[DiagonalArg (diagonalarg)]

Prompted by a chat on #ubuntu-bugs, I'll point this out:

(1) Since an MBR has a signature at the end, 0x55AA, then what we're seeing should not be happening. It's possible in all but 1 in 65536 cases to definitively identify that there is no partition table on the disk.

(2) Even in that small chance that the signature is there "randomly" (on a random filled disk), partition entries would also show up randomly. They would not show up specifically at 181GB into the disk.

Therefore, something that we shouldn't expect is going on here.

[DiagonalArg (diagonalarg)]

apport information

[DiagonalArg (diagonalarg)]

apport information

[Phillip Susi (psusi)]

Did you reboot after filling the drive with random data? Also please run sudo blkid -p /dev/sda ( or whichever disk is effected ) and post the output.

[DiagonalArg (diagonalarg)]

Yes, I have rebooted many times. Right now, I'm using them in a working system. What's more, I have two systems prepared similarly, one with raid and one without, and they are showing exactly the same problem. Here, have a look at one of these systems (raid). All the disks sd{a,b,c} are random filled with no luks header. They are mounted with an external header. sd{a,b} contain root/home and sdc (not mounted) is just data.

@Tyan-S2927:~$ ls -l /dev/disk/by-id/wwn-*
lrwxrwxrwx 1 root root 9 Jan 14 11:16 /dev/disk/by-id/wwn-0x50014ee207XXXXXX -> ../../sdb
lrwxrwxrwx 1 root root 9 Jan 14 11:16 /dev/disk/by-id/wwn-0x50014ee208XXXXXX -> ../../sdc
lrwxrwxrwx 1 root root 10 Jan 14 11:16 /dev/disk/by-id/wwn-0x50014ee208XXXXXX-part2 -> ../../sdc2
lrwxrwxrwx 1 root root 10 Jan 14 11:16 /dev/disk/by-id/wwn-0x50014ee208XXXXXX-part3 -> ../../sdc3
lrwxrwxrwx 1 root root 10 Jan 14 11:16 /dev/disk/by-id/wwn-0x50014ee208XXXXXX-part4 -> ../../sdc4
lrwxrwxrwx 1 root root 9 Jan 14 11:16 /dev/disk/by-id/wwn-0x50014ee25XXXXXX -> ../../sda
lrwxrwxrwx 1 root root 10 Jan 14 11:16 /dev/disk/by-id/wwn-0x50014ee25XXXXXX-part2 -> ../../sda2

@Tyan-S2927:~$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 1.8T 0 disk
├─sda2 8:2 0 1.6T 0 part
└─luks.root1 252:0 0 1.8T 0 crypt /home
sdb 8:16 0 1.8T 0 disk
└─luks.root2 252:1 0 1.8T 0 crypt
sdc 8:32 0 1.8T 0 disk
├─sdc2 8:34 0 499.7G 0 part
├─sdc3 8:35 0 1000.3G 0 part
└─sdc4 8:36 0 130.7G 0 part
sdd 8:48 1 28.9G 0 disk
└─sdd1 8:49 1 28.9G 0 part /boot

@Tyan-S2927:~$ sudo blkid -p /dev/sda
@Tyan-S2927:~$ sudo blkid -p /dev/sdb
@Tyan-S2927:~$ sudo blkid -p /dev/sdc

@Tyan-S2927:~$ mount | column -t
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
udev on /dev type devtmpfs (rw,nosuid,relatime,size=12326324k,nr_inodes=3081581,mode=755)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,noexec,relatime,size=2475548k,mode=755)
/dev/mapper/luks.root1 on / type btrfs (rw,noatime,nodiratime,compress=lzo,space_cache,autodefrag,subvolid=258,subvol=/@)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
tmpfs on /run/lock type tmpfs (rw,nosuid,nodev,noexec,relatime,size=5120k)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgr...

[DiagonalArg (diagonalarg)]

Here, the formatting is slightly better in this pastebin:

http://pastebin.com/dRh9RCfM

[DiagonalArg (diagonalarg)]

Also, have a look at the syslog:

http://paste.ubuntu.com/14498427/

In addition to searching for /dev/sd{a,b,c}, you can also search for the word "truncate" in there.

[Phillip Susi (psusi)]

It seems that your random data just happens to look like an Atari partition table to the kernel. Parted and ( it would seem ) blkid do not support this format so they don't recognize any partitions, but the kernel does, and so gives you partitions.

[DiagonalArg (diagonalarg)]

Phillip, the Atari Partition Table spec is located here:
http://drac030.krap.pl/APT_spec.pdf

You will notice that the header entry (bottom of p.2 to top of p.3) has a signature at offset 1 ("APT"). It also has a sanity check at offset 5. In addition, each partition entry points to both the previous and next entry. There are clearly sufficient tests here to see if there is really an Atari Partition Table present or not.

I have filled six 2T disks, each with different random data, and I'm always getting the same result. There is no way this is happening randomly.

[Phillip Susi (psusi)]

The kernel code is looking for:

1) Each partition has the 1 bit set in its flags byte
2) Each partition's ID bytes are all ascii numbers
3) Each partition starts and ends within the bounds of the disk

When these conditions are met, it sets up the partitions and states that the partition table type is "AHDI", which your syslog shows, so as unlikely as this seems, it must have happened.

I suppose you can get a hex dump of the boot sector of this drive and see:

sudo dd if=/dev/sda count=1 | hd > bootsect.txt

[Phillip Susi (psusi)]

Sorry, in #2 make that "is alphanumeric".

[DiagonalArg (diagonalarg)]

Phillip - Thanks for this explanation. While I'm willing to look into the first two, please first note that (3) clearly does not hold. If you look in the syslog I linked (above), you will see repeatedly:

Jan 8 18:27:08 Tyan-S2927 kernel: [ 2.804642] sdc: p2 size 1157233300 extends beyond EOD, truncated
Jan 8 18:27:08 Tyan-S2927 kernel: [ 2.804747] sdc: p3 size 3023914186 extends beyond EOD, truncated
Jan 8 18:27:08 Tyan-S2927 kernel: [ 2.804814] sdc: p4 size 3400475220 extends beyond EOD, truncated

Whatever it's seeing are not within the bounds of the disk.

[Launchpad Janitor (janitor)]

[Expired for util-linux (Ubuntu) because there has been no activity for 60 days.]

[Andreas Schildbach (schildbach)]

I've got this problem too, for one drive. I'm using a LUKS2 header, containing an encrypted BTRFS filesystem that spans the whole drive (without any partition table). I've got this same setup on thre drives, one of them "dreams up" two AHDI partitions that I can see in lsblk.

[DiagonalArg (diagonalarg)]

Seven years and two weeks later!