When utilizing a standalone ironic install ( via bifrost https://github.com/juliakreger/bifrost ) to re-deploy physical and virtual nodes that already had a configuration drive, I've found that the config drive silently fails to be written.
This is on ironic master branch as of yesterday afternoon and IPA's master branch as of today. My understanding is cleaning will largely resolve this issue in the case of redeployments, although a node that is added to ironic with a pre-existing configdrive from a previous deployment will fail in this same manor.
Investigation has revealed two things are occurring:
1) CoreOS is auto-mounting the pre-existing configuration drive which prevents the kernel view of the disk devices from being updating after a disk image has been written out, resulting in deployment writing out to an _existing_ configdrive partition that is not in the partition table on disk. Upon reboot, only the original partition from the disk image is present. This can be bypassed by setting the kernel parameter "coreos.configdrive=0", although it also means that the masked units in oem/cloud-config.yml are not working as expected. This was identified and verified by logging into the IPA node and checking the mounted devices.
2) The execution of partx in ironic_python_agent/shell/copy_configdrive_to_disk.sh does not appear to work (verified in a VM with the disk bus being virtio) as expected after new image is written out via ironic_python_agent/shell/write_image.sh. This was determined by logging into the node running IPA which was booted with coreos.configdrive=0 (having short circuited Ironic's reboot and complete deployment step) and investigating the state after the IPA debug logging indicated that it successfully wrote to the _existing configdrive device_. The partition table on disk showed a single partition (/dev/vda1), where as the kernel device listing showed two partitions (/dev/vda1, /dev/vda2). Manually re-running `partx -u /dev/vda` successfully re-synced the table, however as result of the first execution apparently not registering (possible race condition with partx and device usage?), the node lacks a configdrive upon reboot.
Setting coreos.configdrive=0 and deploying a locally built IPA ramdisk with 877f66826cd1b50163c67e73c8ebb4590c0f7ec8 reverted, as JayF requested I try in IRC, results in a the configuration drive always being written out if a stale partition existed as it is wiped and partprobe updates the kernel partition table.
+------------------------+-------------------------------------------------------------------------+
| Property | Value |
+------------------------+-------------------------------------------------------------------------+
| instance_uuid | None |
| target_power_state | None |
| properties | {u'memory_mb': u'512', u'cpu_arch': u'x86_64', u'local_gb': u'10', |
| | u'cpus': u'1'} |
| maintenance | False |
| driver_info | {u'ssh_port': 22, u'ssh_username': u'ironic', u'deploy_kernel': |
| | u'http://192.168.122.1:8080/coreos_production_pxe.vmlinuz', |
| | u'deploy_ramdisk': u'http://192.168.122.1:8080 |
| | /coreos_production_pxe_image-oem.cpio.gz', u'ssh_key_filename': |
| | u'/home/ironic/.ssh/id_rsa', u'ssh_address': u'127.0.0.1', |
| | u'ssh_virt_type': u'virsh'} |
| extra | {} |
| last_error | None |
| created_at | 2015-03-18T16:17:01+00:00 |
| target_provision_state | active |
| driver | agent_ssh |
| updated_at | 2015-03-18T22:30:16+00:00 |
| maintenance_reason | None |
| instance_info | {u'root_gb': 10, u'image_source': |
| | u'http://192.168.122.1:8080/deployment_image.qcow2', u'image_checksum': |
| | u'684649dec72ecfc600842ff8af836f57', u'image_url': |
| | u'http://192.168.122.1:8080/deployment_image.qcow2', |
| | u'image_disk_format': u'raw', u'configdrive': |
| | u'http://192.168.122.1:8080/configdrive-a8cb6624-0d9f-c882-affc- |
| | 046ebb96ec01.iso.gz'} |
| driver_internal_info | |
| chassis_uuid | |
| provision_state | deploying |
| reservation | None |
| power_state | power on |
| console_enabled | False |
| uuid | a8cb6624-0d9f-c882-affc-046ebb96ec01 |
+------------------------+-------------------------------------------------------------------------+
After a conversation with Alex Crawford from CoreOS, it sounds like this *is* a race. The ConfigDrive is mounted before oem/cloud-
A short-term fix will be to modify the coreos-