Nailgun-agent should use 512 block size for calculation

[Andrey Danin]

Affects 7.0, 8.0, 9.0. Was reproduced with 7.0

Fuel provides wrong disk size for disks with 4k logical block size. In my case it was HGST Ultrastar 7K6000 SAS drive with 4k block and 4.5TB but Fuel reported 38TB.

In order to calculate a disk size Nailgun-agent gathers a size in blocks from Ohai, a block size form /sys/block/{disk_name_ie_sda}/queue/logical_block_size and then multiplies them. It's wrong because Ohai always returns a size aligned to 512 block size.

Since 4k disks become more popular we will most likely have problems with a lot of installations in future.

Workaround:
Manually remove these lines [1] from /usr/bin/nailgun-agent on the bootstrap node then restart nailgun-agent

[1] https://github.com/openstack/fuel-nailgun-agent/blob/7.0/agent#L375-L377

[Krzysztof Szukiełojć (kszukielojc)]

Assigning to it as we are changing code in place where space is read. So we probably will fix it with it.

[Krzysztof Szukiełojć (kszukielojc)]

Andrey, can you check if `blockdev --getsize64 /dev/sda` returns correct disk size?

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/278948
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-agent/commit/?id=76f48ff6c6a3996a7800a34cd97c5bfd4539107f
Submitter: Jenkins
Branch: master

commit 76f48ff6c6a3996a7800a34cd97c5bfd4539107f
Author: Alexey Sheplyakov <email address hidden>
Date: Thu Feb 11 12:30:26 2016 +0300

    Fix hard drive size miscalculation

    On Linux ohai gets the block device size from /sys/block/$device/size.
    That size is always measured in units of 512 bytes even if the "physical"
    block size of the device in question is different [1][2]. On the other hand
    /sys/block/$device/queue/logical_block_size is the smallest unit which
    the device can address, and /sys/block/$device/queue/physical_block_size
    is the smallest unit the physical storage device can write atomically.
    Typically SATA/SAS drives having the size >= 2 TB have 4 KB physical
    sectors and expose 512 "logical" block size to the operating system.
    However some hard drives (for instance, HGST Ultrastar 7K6000 SAS drive)
    expose the actual physical sector size (that is, 4 KB) for the efficiency
    reasons. As a result nailgun-agent miscomputes the size of the hard drive
    (it's 8x off!)

    [1] http://lxr.free-electrons.com/source/include/linux/types.h?v=4.4#L124
    [2] http://lxr.free-electrons.com/source/drivers/scsi/sd.c?v=4.4#L2340

    Change-Id: Iae36b11dce8e6f43d7ee4bddac5098c633883ed6
    Closes-Bug: #1544816

[Alexander Gordeev (a-gordeev)]

Hello Maintenance team

Bug is h/w specific, but still high. Could be reproduced only at any nodes with 4K native[1] disks. Both work-around and fix are trivial.

[1] https://en.wikipedia.org/wiki/Advanced_Format#4K_native

[Vitaly Sedelnik (vsedelnik)]

Retargeting to 6.1-updates and 7.0-updates as this issue is not customer found (yet). We will consider including it into 8.0-mu-1.

[Andrey Danin (gcon-monolake)]

Krzysztof,

[root@bootstrap ~]# blockdev --getsize64 /dev/sda
5000981078016

[root@bootstrap ~]# parted /dev/sda
GNU Parted 2.1
Using /dev/sda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
Model: HGST HUS726050AL4214 (scsi)
Disk /dev/sda: 5001GB
Sector size (logical/physical): 4096B/4096B

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/291049

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/7.0
Review: https://review.openstack.org/291050

[Maksim Malchuk (mmalchuk)]

mos 9.0 #465 iso contain the fix.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/291049
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-agent/commit/?id=9e716cbddc3634c256cbcb24a5c43fb550a52d47
Submitter: Jenkins
Branch: stable/8.0

commit 9e716cbddc3634c256cbcb24a5c43fb550a52d47
Author: Alexey Sheplyakov <email address hidden>
Date: Thu Feb 11 12:30:26 2016 +0300

    Fix hard drive size miscalculation

    On Linux ohai gets the block device size from /sys/block/$device/size.
    That size is always measured in units of 512 bytes even if the "physical"
    block size of the device in question is different [1][2]. On the other hand
    /sys/block/$device/queue/logical_block_size is the smallest unit which
    the device can address, and /sys/block/$device/queue/physical_block_size
    is the smallest unit the physical storage device can write atomically.
    Typically SATA/SAS drives having the size >= 2 TB have 4 KB physical
    sectors and expose 512 "logical" block size to the operating system.
    However some hard drives (for instance, HGST Ultrastar 7K6000 SAS drive)
    expose the actual physical sector size (that is, 4 KB) for the efficiency
    reasons. As a result nailgun-agent miscomputes the size of the hard drive
    (it's 8x off!)

    [1] http://lxr.free-electrons.com/source/include/linux/types.h?v=4.4#L124
    [2] http://lxr.free-electrons.com/source/drivers/scsi/sd.c?v=4.4#L2340

    Change-Id: Iae36b11dce8e6f43d7ee4bddac5098c633883ed6
    Cherry-pick from: 76f48ff6c6a3996a7800a34cd97c5bfd4539107f
    Closes-Bug: #1544816

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/291050
Committed: https://git.openstack.org/cgit/openstack/fuel-nailgun-agent/commit/?id=748971cc947c6ea5d5848ce1915c5e4d8e0ebfda
Submitter: Jenkins
Branch: stable/7.0

commit 748971cc947c6ea5d5848ce1915c5e4d8e0ebfda
Author: Alexey Sheplyakov <email address hidden>
Date: Thu Feb 11 12:30:26 2016 +0300

    Fix hard drive size miscalculation

    On Linux ohai gets the block device size from /sys/block/$device/size.
    That size is always measured in units of 512 bytes even if the "physical"
    block size of the device in question is different [1][2]. On the other hand
    /sys/block/$device/queue/logical_block_size is the smallest unit which
    the device can address, and /sys/block/$device/queue/physical_block_size
    is the smallest unit the physical storage device can write atomically.
    Typically SATA/SAS drives having the size >= 2 TB have 4 KB physical
    sectors and expose 512 "logical" block size to the operating system.
    However some hard drives (for instance, HGST Ultrastar 7K6000 SAS drive)
    expose the actual physical sector size (that is, 4 KB) for the efficiency
    reasons. As a result nailgun-agent miscomputes the size of the hard drive
    (it's 8x off!)

    [1] http://lxr.free-electrons.com/source/include/linux/types.h?v=4.4#L124
    [2] http://lxr.free-electrons.com/source/drivers/scsi/sd.c?v=4.4#L2340

    Change-Id: Iae36b11dce8e6f43d7ee4bddac5098c633883ed6
    Cherry-pick from: 76f48ff6c6a3996a7800a34cd97c5bfd4539107f
    Closes-Bug: #1544816

[Vadim Rovachev (vrovachev)]

Reproduced w/o patch.
Fixed with patch.
package with fix: nailgun-agent
version: 8.0.0-1~u14.04+mos153

Steps to reproduce:
add kvm machine to master node
destroy kvm machine
using "virsh edit" add line:
<blockio logical_block_size='4096' physical_block_size='4096'/>
to disk block.
Delete node from fuel DB(fuel node --node-id <node-id> --delete-from-db --force)
start kvm machine.
W/o patch disk size of this kvm machine displayed as 0.8 TB, but actual disk size 100GB.
After apply patch all is ok :)

[Vladimir Jigulin (vjigulin)]

Reproduced on mos7.0. Verified on mos7.0+mu6 updates.
Used <blockio logical_block_size='4096' physical_block_size='4096'/> in slave vm config

[Alexey Stupnikov (astupnikov)]

We no longer support MOS5.1, MOS6.0, MOS6.1
We deliver only Critical/Security fixes to MOS7.0, MOS8.0.
We deliver only High/Critical/Security fixes to MOS9.2.