Configdrive writing script fails for gpt devices

Bug #1633063 reported by Vladyslav Drok
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
ironic-lib
Fix Released
Undecided
Vladyslav Drok
ironic-python-agent
Fix Released
Undecided
Vladyslav Drok

Bug Description

Here is a piece of log describing the problem:

2016-10-13 11:50:14.086 420 ERROR root [-] Command failed: prepare_image, error: Error writing configdrive to device: Writing configdrive to device /dev/vda failed with exit code 1. stdout: fix the GPT to use all of the space (an extra 22978560 blocks) or continue with
copy_configdrive_to_disk.sh: Fixing GPT to use all of the space on device /dev/vda
The operation has completed successfully.
copy_configdrive_to_disk.sh: Adding configdrive partition to /dev/vda
Information: Moved requested sector from 22937566 to 22935552 in
order to align on 2048-sector boundaries.
The operation has completed successfully.
copy_configdrive_to_disk.sh: Writing Configdrive contents in /tmp/configdrive to /dev/vda2048
copy_configdrive_to_disk.sh: Error writing Configdrive to /dev/vda2048
. stderr: Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
Warning: Not all of the space available to /dev/vda appears to be used, you can fix the GPT to use all of the space (an extra 22978560 blocks) or continue with the current setting?
dd: failed to open '/dev/vda2048': Invalid argument
.
2016-10-13 11:50:14.086 420 ERROR root Traceback (most recent call last):
2016-10-13 11:50:14.086 420 ERROR root File "/usr/local/lib/python2.7/site-packages/ironic_python_agent/extensions/base.py", line 159, in run
2016-10-13 11:50:14.086 420 ERROR root File "/usr/local/lib/python2.7/site-packages/ironic_python_agent/extensions/standby.py", line 550, in prepare_image
2016-10-13 11:50:14.086 420 ERROR root File "/usr/local/lib/python2.7/site-packages/ironic_python_agent/extensions/standby.py", line 220, in _write_configdrive_to_partition
2016-10-13 11:50:14.086 420 ERROR root ConfigDriveWriteError: Error writing configdrive to device: Writing configdrive to device /dev/vda failed with exit code 1. stdout: fix the GPT to use all of the space (an extra 22978560 blocks) or continue with
2016-10-13 11:50:14.086 420 ERROR root copy_configdrive_to_disk.sh: Fixing GPT to use all of the space on device /dev/vda
2016-10-13 11:50:14.086 420 ERROR root The operation has completed successfully.
2016-10-13 11:50:14.086 420 ERROR root copy_configdrive_to_disk.sh: Adding configdrive partition to /dev/vda
2016-10-13 11:50:14.086 420 ERROR root Information: Moved requested sector from 22937566 to 22935552 in
2016-10-13 11:50:14.086 420 ERROR root order to align on 2048-sector boundaries.
2016-10-13 11:50:14.086 420 ERROR root The operation has completed successfully.
2016-10-13 11:50:14.086 420 ERROR root copy_configdrive_to_disk.sh: Writing Configdrive contents in /tmp/configdrive to /dev/vda2048
2016-10-13 11:50:14.086 420 ERROR root copy_configdrive_to_disk.sh: Error writing Configdrive to /dev/vda2048
2016-10-13 11:50:14.086 420 ERROR root . stderr: Error: The backup GPT table is not at the end of the disk, as it should be. This might mean that another operating system believes the disk is smaller. Fix, by moving the backup to the end (and removing the old backup)?
2016-10-13 11:50:14.086 420 ERROR root Warning: Not all of the space available to /dev/vda appears to be used, you can fix the GPT to use all of the space (an extra 22978560 blocks) or continue with the current setting?
2016-10-13 11:50:14.086 420 ERROR root dd: failed to open '/dev/vda2048': Invalid argument
2016-10-13 11:50:14.086 420 ERROR root .
2016-10-13 11:50:14.086 420 ERROR root

The reason of this seems to be incorrect commands in the bash script.

Apart from that, if doing dd right after creating a partition, it may fail like this:

dd if=/tmp/configdrive of=/dev/vda2 bs=64K oflag=direct
dd: failed to open '/dev/vda2': Invalid argument

And partprobe may not help in this case.

Vladyslav Drok (vdrok)
Changed in ironic-python-agent:
assignee: nobody → Vladyslav Drok (vdrok)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic-python-agent (master)

Fix proposed to branch: master
Review: https://review.openstack.org/385957

Changed in ironic-python-agent:
status: New → In Progress
Vladyslav Drok (vdrok)
description: updated
Changed in ironic-lib:
assignee: nobody → Vladyslav Drok (vdrok)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic-lib (master)

Fix proposed to branch: master
Review: https://review.openstack.org/387664

Changed in ironic-lib:
status: New → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic-python-agent (master)

Reviewed: https://review.openstack.org/385957
Committed: https://git.openstack.org/cgit/openstack/ironic-python-agent/commit/?id=7bda3408f5bef4349c4e3537f3c8c0826a5dead8
Submitter: Jenkins
Branch: master

commit 7bda3408f5bef4349c4e3537f3c8c0826a5dead8
Author: Vladyslav Drok <email address hidden>
Date: Thu Oct 13 15:28:04 2016 +0300

    Fix config drive writing script

    When the deployment happens on a GPT disk with config drive, writing
    it to disk fails. Three reasons for that:

    * parted should be used instead of partprobe to determine the type of
      the disk;
    * gdisk -l sorts the partitions by their number, sort them by start
      sector instead;
    * after sgdisk completion, the configdrive device is not immediately
      visible in the /dev folder, udevadm settle needed here too.

    Closes-Bug: #1633063
    Change-Id: Ifed89e343f9db4cf303baf7f8823342f6041f202

Changed in ironic-python-agent:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic-lib (master)

Reviewed: https://review.openstack.org/387664
Committed: https://git.openstack.org/cgit/openstack/ironic-lib/commit/?id=939c026950756a926056a9ee24a18b02a150bb82
Submitter: Jenkins
Branch: master

commit 939c026950756a926056a9ee24a18b02a150bb82
Author: Vladyslav Drok <email address hidden>
Date: Mon Oct 17 23:12:02 2016 +0300

    Wait for a config drive device to be available

    A failure may occur if the dd to config drive partition happens
    before it actually becomes available. This change adds a udevadm
    settle call that ensures that config drive device is present in
    /dev folder.

    Closes-Bug: #1633063
    Change-Id: I5526f791371e7a6f4cdad151bb163707504e76cf

Changed in ironic-lib:
status: In Progress → Fix Released
Revision history for this message
Thierry Carrez (ttx) wrote : Fix included in openstack/ironic-lib 2.3.0

This issue was fixed in the openstack/ironic-lib 2.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic-python-agent (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/392557

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to ironic-lib (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/392573

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic-python-agent (stable/newton)

Reviewed: https://review.openstack.org/392557
Committed: https://git.openstack.org/cgit/openstack/ironic-python-agent/commit/?id=f52e88301068ade6add74906843e2063ef87826a
Submitter: Jenkins
Branch: stable/newton

commit f52e88301068ade6add74906843e2063ef87826a
Author: Vladyslav Drok <email address hidden>
Date: Thu Oct 13 15:28:04 2016 +0300

    Fix config drive writing script

    When the deployment happens on a GPT disk with config drive, writing
    it to disk fails. Three reasons for that:

    * parted should be used instead of partprobe to determine the type of
      the disk;
    * gdisk -l sorts the partitions by their number, sort them by start
      sector instead;
    * after sgdisk completion, the configdrive device is not immediately
      visible in the /dev folder, udevadm settle needed here too.

    Closes-Bug: #1633063
    Change-Id: Ifed89e343f9db4cf303baf7f8823342f6041f202
    (cherry picked from commit 7bda3408f5bef4349c4e3537f3c8c0826a5dead8)

tags: added: in-stable-newton
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to ironic-lib (stable/newton)

Reviewed: https://review.openstack.org/392573
Committed: https://git.openstack.org/cgit/openstack/ironic-lib/commit/?id=6abe92e190058f277d69859f0e68f4b3fd086047
Submitter: Jenkins
Branch: stable/newton

commit 6abe92e190058f277d69859f0e68f4b3fd086047
Author: Vladyslav Drok <email address hidden>
Date: Mon Oct 17 23:12:02 2016 +0300

    Wait for a config drive device to be available

    A failure may occur if the dd to config drive partition happens
    before it actually becomes available. This change adds a udevadm
    settle call that ensures that config drive device is present in
    /dev folder.

    Closes-Bug: #1633063
    Change-Id: I5526f791371e7a6f4cdad151bb163707504e76cf
    (cherry picked from commit 939c026950756a926056a9ee24a18b02a150bb82)

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ironic-lib 2.3.0

This issue was fixed in the openstack/ironic-lib 2.3.0 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ironic-lib 2.1.2

This issue was fixed in the openstack/ironic-lib 2.1.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ironic-python-agent 1.5.1

This issue was fixed in the openstack/ironic-python-agent 1.5.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ironic-lib 2.1.2

This issue was fixed in the openstack/ironic-lib 2.1.2 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ironic-python-agent 1.5.1

This issue was fixed in the openstack/ironic-python-agent 1.5.1 release.

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/ironic-python-agent 2.0.0

This issue was fixed in the openstack/ironic-python-agent 2.0.0 release.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.