Deploying ramdisk instances with Ironic 2023.1 failed - image_source

[Alexey Tsitsin]

deploying instances with a Ramdisk failed:
Ironic Bifrost version stable/2023.1

https://docs.openstack.org/ironic/2023.1/admin/ramdisk-boot.html

[DEFAULT]
...
enabled_deploy_interfaces = direct,ramdisk

$vmbc with libvirt is used for development purposes:
$sudo /opt/virtualbmc/bin/vbmc add diskless-test --port 6230 --username test --password test

$disk-image-create -o ~/output/ramdisk debian-minimal ironic-ramdisk-base openssh-server dhcp-all-interfaces

$source /opt/stack/bifrost/bin/activate
$export OS_CLOUD=bifrost

~/bifrost/bifrost-cli enroll ~/bifrost/playbooks/inventory/baremetal.yml

diskless-test:
  driver: ipmi
  driver_info:
    ipmi_address: "127.0.0.1"
    ipmi_port: "6231"
    ipmi_password: "test"
    ipmi_username: "test"
  ipv4_address: 192.168.103.101
  ipv4_gateway: 192.168.103.254
  ipv4_subnet_mask: 255.255.255.0
  network_mtu: 1500
  name: diskless-test
  nics:
  - mac: 52:54:00:e6:20:55
  properties:
    cpu_arch: x86_64
  uuid: 00000000-0000-0000-0000-200000000101
  instance-info:
    kernel: http://192.168.103.99:8081/iso/ramdisk.kernel
    ramdisk: http://192.168.103.99:8081/iso/ramdisk.initramfs

$baremetal node set diskless --instance-info kernel=http://192.168.103.99:8081/iso/ramdisk.kernel --instance-info ramdisk=http://192.168.103.99:8081/iso/ramdisk.initramfs

$ baremetal node deploy diskless
Failed to validate deploy or power info for node 00000000-0000-0000-0000-200000000101: Node 00000000-0000-0000-0000-200000000101 failed to validate deploy image info. Some parameters were missing. Missing are: ['instance_info.image_source'] (HTTP 400)

Even if:
Before the Xena release, the image_source field was also required:

--instance-info image_source=http://path/to/ramdisk.initramfs

/var/log/syslog:Aug 5 11:18:57 ironic01 ironic[123909]: 2023-08-05 11:18:57.073 123909 DEBUG ironic.api.method [None req-a4011e4c-acb6-44db-a641-afe369cff923 bifrost_user - - - - -] Client-side error: Failed to validate deploy or power info for node 00000000-0000-0000-0000-200000000101: Node 00000000-0000-0000-0000-200000000101 failed to validate deploy image info. Some parameters were missing. Missing are: ['instance_info.image_source'] format_exception /opt/stack/bifrost/lib/python3.10/site-packages/ironic/api/method.py:124#033[00m

If we provide image_source=http://192.168.103.99:8081/iso/ramdisk.kernel

So the error will be:

$ baremetal node set diskless --instance-info kernel=http://192.168.103.99:8081/iso/ramdisk.kernel --instance-info ramdisk=http://192.168.103.99:8081/iso/ramdisk.initramfs --instance-info image_source=http://192.168.103.99:8081/iso/ramdisk.kernel
(bifrost) ubuntu@ironic01:~/bifrost$ baremetal node deploy diskless
Failed to validate deploy or power info for node 00000000-0000-0000-0000-200000000101: image_source's "image_checksum", or "image_os_hash_algo" and "image_os_hash_value" must be provided in instance_info for node 00000000-0000-0000-0000-200000000101 (HTTP 400)

If we provide image_checksum:

baremetal node set diskless --instance-info kernel=http://192.168.103.99:8081/iso/ramdisk.kernel --instance-info ramdisk=http://192.168.103.99:8081/iso/ramdisk.initramfs --instance-info image_source=http://192.168.103.99:8081/iso/ramdisk.initramfs --instance-info image_checksum=http://192.168.103.99:8081/iso/ramdisk.initramfs.sha256

$ baremetal node deploy diskless

$ baremetal node list
+--------------------------------------+------------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+------------+---------------+-------------+--------------------+-------------+
| 00000000-0000-0000-0000-200000000101 | diskless | None | power on | deploy failed | False |
+--------------------------------------+------------+---------------+-------------+--------------------+-------------+

| instance_info | {'kernel': 'http://192.168.103.99:8081/iso/ramdisk.kernel', 'ramdisk': 'http://192.168.103.99:8081/iso/ramdisk.initramfs', 'image_source': 'http://192.168.103.99:8081/iso/ramdisk.initramfs', 'image_type': 'partition', 'image_checksum': 'http://192.168.103.99:8081/iso/ramdisk.initramfs.sha256'} |
| instance_uuid | None |
| last_error | Failed to prepare to deploy: Some parameters were missing in node's instance_info. Missing are: ['root_gb']

However we understand that there is no any disks because the server is diskless

If we provide root_gb manually:

$ baremetal node set diskless --instance-info kernel=http://192.168.103.99:8081/iso/ramdisk.kernel --instance-info ramdisk=http://192.168.103.99:8081/iso/ramdisk.initramfs --instance-info image_source=http://192.168.103.99:8081/iso/ramdisk.initramfs --instance-info image_checksum=http://192.168.103.99:8081/iso/ramdisk.initramfs.sha256 --instance-info root_gb=10

$ baremetal node list
+--------------------------------------+------------+---------------+-------------+--------------------+-------------+
| UUID | Name | Instance UUID | Power State | Provisioning State | Maintenance |
+--------------------------------------+------------+---------------+-------------+--------------------+-------------+
| 00000000-0000-0000-0000-200000000101 | diskless | None | power off | deploy failed | False |
+--------------------------------------+------------+---------------+-------------+--------------------+-------------+

| instance_info | {'kernel': 'http://192.168.103.99:8081/iso/ramdisk.kernel', 'ramdisk': 'http://192.168.103.99:8081/iso/ramdisk.initramfs', 'image_source': 'http://192.168.103.99:8081/iso/ramdisk.initramfs', 'image_type': 'partition', 'image_checksum': 'http://192.168.103.99:8081/iso/ramdisk.initramfs.sha256', 'root_gb': 10, 'image_url': '******', 'preserve_ephemeral': False, 'swap_mb': 0, 'ephemeral_gb': 0, 'root_mb': 10240, 'ephemeral_mb': 0, 'ephemeral_format': None, 'configdrive': '******'} |
| instance_uuid | None |
| last_error | Deploy step deploy.prepare_instance_boot failed: Failed to install a bootloader when deploying node 00000000-0000-0000-0000-200000000101: Installing GRUB2 boot loader to device /dev/vda failed with Unexpected error while running command. |
| | Command: mount /dev/vda2 /tmp/tmpa57e_eko |
| | Exit code: 32 |
| | Stdout: '' |
| | Stderr: 'mount: /tmp/tmpa57e_eko: wrong fs type, bad option, bad superblock on /dev/vda2, missing codepage or helper program, or other error.\n'.

The logic seems broken in 2023.1 Ironic booting with a ramdrive

[Alexey Tsitsin (daydrim)]

Traceback:

ERROR ironic.conductor.utils KeyError: 'image_source'

ERROR ironic.conductor.utils [None req-406c6af0-b596-4c0f-9a66-12ffbf581f56 bifrost_user - - - - -] Node 00000000-0000-0000-0000-200000000004>
ERROR ironic.conductor.utils Traceback (most recent call last):
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic/conductor/deployments.py", line 276, in do_next_d>
ERROR ironic.conductor.utils result = interface.execute_deploy_step(task, step)
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic/drivers/base.py", line 376, in execute_deploy_step
ERROR ironic.conductor.utils return self._execute_step(task, step)
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic/drivers/base.py", line 299, in _execute_step
ERROR ironic.conductor.utils return getattr(self, step['step'])(task)
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic_lib/metrics.py", line 60, in wrapped
ERROR ironic.conductor.utils result = f(*args, **kwargs)
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic/conductor/task_manager.py", line 153, in wrapper
ERROR ironic.conductor.utils return f(*args, **kwargs)
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic/drivers/modules/ramdisk.py", line 76, in deploy
ERROR ironic.conductor.utils task.driver.boot.prepare_instance(task)
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic_lib/metrics.py", line 60, in wrapped
ERROR ironic.conductor.utils result = f(*args, **kwargs)
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic/drivers/modules/pxe_base.py", line 242, in prepar>
ERROR ironic.conductor.utils instance_image_info = pxe_utils.get_instance_image_info(
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic/common/pxe_utils.py", line 736, in get_instance_i>
ERROR ironic.conductor.utils _get_image_properties()
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic/common/pxe_utils.py", line 689, in _get_image_pro>
ERROR ironic.conductor.utils d_info['image_source'],
ERROR ironic.conductor.utils KeyError: 'image_source'
ERROR ironic.conductor.utils
DEBUG ironic.common.pxe_utils [None req-406c6af0-b596-4c0f-9a66-12ffbf581f56 bifrost_user - - - - -] Cleaning up PXE config for node 00000000>
DEBUG ironic.drivers.modules.boot_mode_utils [None req-406c6af0-b596-4c0f-9a66-12ffbf581f56 bifrost_user - - - - -] Deploy boot mode is uefi >
DEBUG ironic.drivers.modules.image_cache [None req-406c6af0-b596-4c0f-9a66-12ffbf581f56 bifrost_user - - - - -] Starting clean up for master >
ERROR ironic.conductor.utils [None req-406c6af0-b596-4c0f-9a66-12ffbf581f56 bifrost_user - - - - -] Cleanup failed for node 00000000-0000-000>
ERROR ironic.conductor.utils Traceback (most recent call last):
ERROR ironic.conductor.utils File "/opt/stack/bifrost/lib/python3.10/site-packages/ironic/conductor/deployments.py", line 276,...

[Jay Faulkner (jason-oldos)]

This looks like potentially a configuration issue, we test this in CI so it's unlikely broken systemmatically. I'll ask a bifrost expert to look at this bug to potentially help.

[Launchpad Janitor (janitor)]

[Expired for Bifrost because there has been no activity for 60 days.]