provisioning onto 4K native disks fails

[Andrey Danin]

Affects 7.0, 8.0, 9.0. Reproduced with 7.0

When the first disk discovered by the bootstrap is a disk with 4k logical block size, the node with OS installed won't bring up any NICs after reboot.

Fuel-agent prepares config files for cloud-init in ISO9660 image witch has 2k block size. Then this image is written to /dev/sda. After reboot Cloud-init tries to mount it but it fails.

We need to change an image format from ISO to ext2 with 4k blocks and switch to data source ConfigDrive with disk label == 'config-2' (see [1])

Workaround: in my particular case I applied the patch [2] to collocate the ISO image with the OS filesystem which was on different disk with 512 block size.

[1] https://github.com/number5/cloud-init/blob/0.7.5/cloudinit/sources/DataSourceConfigDrive.py#L247-L255
[2] https://review.openstack.org/#/c/279107/1/fuel_agent/drivers/nailgun.py

[Alexander Gordeev (a-gordeev)]

so, we're dealing here with 4K native (or 4Kn) disks [1]

https://en.wikipedia.org/wiki/Advanced_Format#4K_native [1]

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/279485

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/279485
Committed: https://git.openstack.org/cgit/openstack/fuel-agent/commit/?id=a07e21c252053482bf79e245abdbec9eec231d97
Submitter: Jenkins
Branch: master

commit a07e21c252053482bf79e245abdbec9eec231d97
Author: Alexander Gordeev <email address hidden>
Date: Thu Feb 11 17:35:51 2016 +0300

    Always add configdrive partition to OS disk

    configdrive partition was put onto the first disk from the list of
    all available disks.
    When dealing with large number of nodes, sometimes disk names many
    vary from one node to another, so it's hard to predict onto which
    disk configdrive partition will land.

    Since now, configdive partition is always tied to OS disk.

    Change-Id: I6550b7c541050083991b4874f7230fc1606487df
    Related-Bug: #1544818

[Bug Checker Bot (bug-checker)]

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

actual result

expected result

steps to reproduce

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

[Alexander Gordeev (a-gordeev)]

> We need to change an image format from ISO to ext2 with 4k blocks and switch to data source ConfigDrive with disk label == 'config-2' (see [1])

that's it. We're using NoCloud data source right now it will not allow us to use any other FS with 4k blocks. (vfat isn't an option)

so, in order to fix, we need to modify do_configdrive action in manager.py https://github.com/openstack/fuel-agent/blob/57539aed3192969e884acb7cbba9952a291052c3/fuel_agent/manager.py#L343-L365

so, it will be producing ext2 image instead of iso. http://cloudinit.readthedocs.org/en/latest/topics/datasources.html#version-2

[Bug Checker Bot (bug-checker)]

(This check performed automatically)
Please, make sure that bug description contains the following sections filled in with the appropriate data related to the bug you are describing:

actual result

expected result

steps to reproduce

For more detailed information on the contents of each of the listed sections see https://wiki.openstack.org/wiki/Fuel/How_to_contribute#Here_is_how_you_file_a_bug

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/302397

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/302397
Committed: https://git.openstack.org/cgit/openstack/fuel-agent/commit/?id=8f5c03dcacfd20d278c4c1389762638fb14e1a02
Submitter: Jenkins
Branch: master

commit 8f5c03dcacfd20d278c4c1389762638fb14e1a02
Author: Dmitry Guryanov <email address hidden>
Date: Wed Apr 6 21:01:16 2016 +0300

    change configdrive format to ConfigDrive version 2

    We put configdrive with is9660 filesystem to a
    partition on a hard disk. New hard disks may have
    4K sectors, but blocksize of iso9660 fs is 2K so
    it will not work.

    To fix this bug we should use another filesystem (ext2)
    and another config drive format (files, directory
    structure), because NoCloud format, which is currently
    used support only vfat and iso9660 filesystems.

    Change-Id: Ia0f244f19bab3dfaceef8a092ad03667675a5557
    Closes-Bug: #1544818

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/318127

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/318127
Committed: https://git.openstack.org/cgit/openstack/fuel-agent/commit/?id=e74e783128daca56c05b7b6568e91cd6fc908f96
Submitter: Jenkins
Branch: stable/mitaka

commit e74e783128daca56c05b7b6568e91cd6fc908f96
Author: Dmitry Guryanov <email address hidden>
Date: Thu May 19 19:02:23 2016 +0300

    change configdrive format to ConfigDrive version 2

    We put configdrive with is9660 filesystem to a
    partition on a hard disk. New hard disks may have
    4K sectors, but blocksize of iso9660 fs is 2K so
    it will not work.

    To fix this bug we should use another filesystem (ext2)
    and another config drive format (files, directory
    structure), because NoCloud format, which is currently
    used support only vfat and iso9660 filesystems.

    Change-Id: Ia0f244f19bab3dfaceef8a092ad03667675a5557
    Closes-Bug: #1544818
    (cherry picked from commit 8f5c03dcacfd20d278c4c1389762638fb14e1a02)

[Dmitry Guryanov (dguryanov)]

I don't have hardware with 4K disk, so I've just checked, that configdrive formatted as ext2 with blocksize 4096 bytes. There should be no problems since ext2 works on 4K drives well.

[OpenStack Infra (hudson-openstack)]

Related fix proposed to branch: master
Review: https://review.openstack.org/332768

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/332768
Committed: https://git.openstack.org/cgit/openstack/fuel-agent/commit/?id=91680ad7e6fbf452f8a4a4a0115521a384ffba32
Submitter: Jenkins
Branch: master

commit 91680ad7e6fbf452f8a4a4a0115521a384ffba32
Author: Alexander Gordeev <email address hidden>
Date: Wed Jun 22 14:34:13 2016 +0300

    Remove genisoimage from deps

    genisoimage isn't needed anymore since fuel-agent started to produce
    non ISO9660 config drive image.

    Change-Id: I19beac5a253aa2db5eeb39c018c5bd90ebcefcd6
    Related-Bug: #1544818

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/8.0
Review: https://review.openstack.org/350108

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/350108
Committed: https://git.openstack.org/cgit/openstack/fuel-agent/commit/?id=d58de668f6b86aac2ff526a6e1eb1486ccb41bc5
Submitter: Jenkins
Branch: stable/8.0

commit d58de668f6b86aac2ff526a6e1eb1486ccb41bc5
Author: Dmitry Guryanov <email address hidden>
Date: Thu May 19 19:02:23 2016 +0300

    change configdrive format to ConfigDrive version 2

    We put configdrive with is9660 filesystem to a
    partition on a hard disk. New hard disks may have
    4K sectors, but blocksize of iso9660 fs is 2K so
    it will not work.

    To fix this bug we should use another filesystem (ext2)
    and another config drive format (files, directory
    structure), because NoCloud format, which is currently
    used support only vfat and iso9660 filesystems.

    Change-Id: Ia0f244f19bab3dfaceef8a092ad03667675a5557
    Closes-Bug: #1544818
    (cherry picked from commit e74e783128daca56c05b7b6568e91cd6fc908f96)

[Sergii Rizvan (srizvan)]

Note for QA:
It's very difficult to find hardware with 4K disk, that's why we can verify the patch just by checking that after provisioning cloud-init drive has ext2 format (the last partition):

root@node-2:~# parted /dev/vda print list
Model: Virtio Block Device (virtblk)
Disk /dev/vda: 107GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
 1 1049kB 26.2MB 25.2MB primary bios_grub
 2 26.2MB 236MB 210MB primary
 3 236MB 446MB 210MB ext2 primary
 4 446MB 26.3GB 25.8GB primary
 5 26.3GB 107GB 81.0GB primary
 6 107GB 107GB 21.0MB ext2 primary

[Alexander Gordeev (a-gordeev)]

@Sergii,

For VM that shouldn't be a problem to verify. Please check the comments here https://review.openstack.org/#/c/302397/9/fuel_agent/manager.py

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/7.0
Review: https://review.openstack.org/359881

[TatyanaGladysheva (tgladysheva)]

Verified on MOS 8.0 + MU3 updates.

Before updates:
After provisioning the last partition doesn't have ext2 format:
root@node-1:~# parted /dev/vda print list
Model: Virtio Block Device (virtblk)
Disk /dev/vda: 53.7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
 1 1049kB 26.2MB 25.2MB primary bios_grub
 2 26.2MB 236MB 210MB primary
 3 236MB 446MB 210MB ext2 primary
 4 446MB 19.8GB 19.3GB primary
 5 19.8GB 30.6GB 10.8GB primary
 6 30.6GB 52.1GB 21.5GB primary
 7 52.1GB 53.5GB 1335MB primary
 8 53.5GB 53.5GB 21.0MB primary

After updates:
After provisioning the last partition has ext2 format:
root@node-3:~# parted /dev/vda print list
Model: Virtio Block Device (virtblk)
Disk /dev/vda: 53.7GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
 1 1049kB 26.2MB 25.2MB primary bios_grub
 2 27.3MB 237MB 210MB primary
 3 238MB 448MB 210MB ext2 primary
 4 449MB 19.8GB 19.4GB primary
 5 19.8GB 30.6GB 10.8GB primary
 6 30.6GB 52.2GB 21.5GB primary
 7 52.2GB 53.5GB 1355MB primary
 8 53.5GB 53.6GB 21.0MB ext2 primary

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/359881
Committed: https://git.openstack.org/cgit/openstack/fuel-agent/commit/?id=0cc992ba32b1608fcbfe2bc9db4e15491e9a6cb3
Submitter: Jenkins
Branch: stable/7.0

commit 0cc992ba32b1608fcbfe2bc9db4e15491e9a6cb3
Author: Dmitry Guryanov <email address hidden>
Date: Thu May 19 19:02:23 2016 +0300

    change configdrive format to ConfigDrive version 2

    We put configdrive with is9660 filesystem to a
    partition on a hard disk. New hard disks may have
    4K sectors, but blocksize of iso9660 fs is 2K so
    it will not work.

    To fix this bug we should use another filesystem (ext2)
    and another config drive format (files, directory
    structure), because NoCloud format, which is currently
    used support only vfat and iso9660 filesystems.

    Change-Id: Ia0f244f19bab3dfaceef8a092ad03667675a5557
    Closes-Bug: #1544818
    (cherry picked from commit d58de668f6b86aac2ff526a6e1eb1486ccb41bc5)

[Vladimir Jigulin (vjigulin)]

Verified on 7.0+mu6
I have a problem with verifying on vm that is probably related to libvirt but after some research with committer we decided to close the bug with justification of ext2 partition as well