Subcloud local install - Failure to set instdev parameter when we use ks-setup.cfg

[Shrikumar Sharma]

Brief Description
------------------------
DC Debian: Subcloud local install - Failure to set instdev parameter when we use ks-setup.cfg

error:

Export %ks-early hook script /tmp/lat/lat_ks_early/0_script
2022-11-14 23:40:52.367 kickstart ks-early info: UEFI Firmware execution
2022-11-14 23:40:52.384 kickstart ks-early info: *****************************************************
2022-11-14 23:40:52.386 kickstart ks-early info: *** Pre Partition - Volume Remove & Wipe Disk ***
2022-11-14 23:40:52.389 kic Waiting for disks to be available
2022-11-14 23:40:52.400 kickstart ks-early info: Disks became available
2022-11-14 23:40:52.403 kickstart ks-early info: Install traits: controller
2022-11-14 23:40:52.406 kickstart ks-early info: controller LAT size parameters: FSZ=32 BSZ=512 RSZ=20480 VSZ=20480
2022-11-14 23:40:52.409 kickstart ks-early info: controller /proc/cmdline:net.naming-scheme=vSTX7_0 BOOT_IMAGE=/bzImage-std initrd=/initrd instdate=@1668469252 instw=60 instiso=instboot biosplusefi=1 instnet=0 ks=file:///kickstart/kickstart.cfg rdinit=/install instname=debian instbr=starlingx instab=0 insturl=file://NOT_SET prestage ip= BLM=2506 FSZ=32 BSZ=512 RSZ=20480 VSZ=20480 instl=/ostree_repo instdev= defaultkernel=vmlinuz*[!t]-amd64 force_install traits=controller console=ttyS0,115200 serial
2022-11-14 23:40:52.448 kickstart ks-early info: controller Controller Function
2022-11-14 23:40:52.449 kickstart ks-early info: controller Standard system
2022-11-14 23:40:52.476 kickstart ks-early warn: controller timezone not set ; 'timezone --nontp --utc UTC'
Error no kernel argument instdev=...
ks-setup.cfg
BOARD_VENDOR=$(cat /sys/devices/virtual/dmi/id/board_vendor)
BOARD_NAME=$(cat /sys/devices/virtual/dmi/id/board_name)
PRODUCT_NAME=$(cat /sys/devices/virtual/dmi/id/product_name)
wlog "BOARD_VENDOR=${BOARD_VENDOR}"
wlog "BOARD_NAME=${BOARD_NAME}"
wlog "PRODUCT_NAME=${PRODUCT_NAME}"# Setup rootfs/boot device based on server type
if [[ ${PRODUCT_NAME} == *"ProLiant e910t"* ]]; then
    rootfs_device=/dev/nvme1n1
elif [[ ${PRODUCT_NAME} == *"Proteus I_Mix"* ]]; then
    rootfs_device=/dev/nvme0n1
elif [[ ${PRODUCT_NAME} == *"TRITON"* ]]; then
    rootfs_device=/dev/nvme0n1
elif [[ ${PRODUCT_NAME} == *"PowerEdge XR11"* ]]; then
    rootfs_device=/dev/disk/by-path/pci-0000:00:17.0-ata-1.0
fi

boot_device=${rootfs_device}
wlog "rootfs_device=${rootfs_device}"
wlog "boot_device=${boot_device}"
subcloud disks - from install-values file:
[sysadmin@controller-0 prestage-files(keystone_admin)]$ cat ~/subcloud-4/subcloud4-install-values.yaml | grep device
rootfs_device: /dev/disk/by-path/pci-0000:00:17.0-ata-1.0
boot_device: /dev/disk/by-path/pci-0000:00:17.0-ata-1.0
Product name
sysadmin@controller-0:~$ cat /sys/devices/virtual/dmi/id/product_name
PowerEdge XR11

Severity
-------------
<Critical: System/Feature is not usable after the defect>

Steps to Reproduce
--------------------
Generate prestaged.iso (use the ks-setup.cfg script described above)

burn prestaged.iso into a USB

boot subcloud from USB

Expected Behavior
-----------------
Kickstart should be able to set the "instdev" parameter according to the PRODUCT_NAME

Actual Behavior
-----------------
It fails to set "instdev", then the install from USB fails

Reproducibility
-----------------
100% reproducible

System Configuration
----------------------
DC

Load info (eg: 2022-03-10_20-00-07)
------------------------------------
[sysadmin@controller-0 prestage-files(keystone_admin)]$ cat /etc/build.info
SW_VERSION="22.12"
BUILD_TARGET="Host Installer"
BUILD_TYPE="Formal"
BUILD_ID="2022-11-11_18-00-09"
SRC_BUILD_ID="222"JOB="wrcp-master-debian"
BUILD_BY="jenkins"
BUILD_NUMBER="159"
BUILD_HOST="yow-wrcp-lx.wrs.com"
BUILD_DATE="2022-11-11 23:00:09 +0000"

Last Pass
-----------
new test scenario.

Timestamp/Logs
-----------------
kickstart logs:

[ 17.379387] Buffer I/O error on dev sr0, logical block 0, async page read
[ 18.375406] attempt to access beyond end of device
[ 18.375406] sr0: rw=0, want=182248, limit=182240
Devices should be properly configured.
Mon Nov 14 23:40:52 UTC 2022
Export %ks-early hook script /tmp/lat/lat_ks_early/0_script
2022-11-14 23:40:52.367 kickstart ks-early info: UEFI Firmware execution
2022-11-14 23:40:52.384 kickstart ks-early info: *****************************************************
2022-11-14 23:40:52.386 kickstart ks-early info: *** Pre Partition - Volume Remove & Wipe Disk ***
2022-11-14 23:40:52.389 kic Waiting for disks to be available
2022-11-14 23:40:52.400 kickstart ks-early info: Disks became available
2022-11-14 23:40:52.403 kickstart ks-early info: Install traits: controller
2022-11-14 23:40:52.406 kickstart ks-early info: controller LAT size parameters: FSZ=32 BSZ=512 RSZ=20480 VSZ=20480
2022-11-14 23:40:52.409 kickstart ks-early info: controller /proc/cmdline:net.naming-scheme=vSTX7_0 BOOT_IMAGE=/bzImage-std initrd=/initrd instdate=@1668469252 instw=60 instiso=instboot biosplusefi=1 instnet=0 ks=file:///kickstart/kickstart.cfg rdinit=/install instname=debian instbr=starlingx instab=0 insturl=file://NOT_SET prestage ip= BLM=2506 FSZ=32 BSZ=512 RSZ=20480 VSZ=20480 instl=/ostree_repo instdev= defaultkernel=vmlinuz*[!t]-amd64 force_install traits=controller console=ttyS0,115200 serial
2022-11-14 23:40:52.448 kickstart ks-early info: controller Controller Function
2022-11-14 23:40:52.449 kickstart ks-early info: controller Standard system
2022-11-14 23:40:52.476 kickstart ks-early warn: controller timezone not set ; 'timezone --nontp --utc UTC'
Error no kernel argument instdev=...[ 19.507516] EXT4-fs (sdb5): 2 orphan inodes deleted
[ 19.512405] EXT4-fs (sdb5): recovery complete
[ 19.517884] EXT4-fs (sdb5): mounted filesystem with ordered data mode. Opts: (null)
Save install-fail.log to partition fluxdata /var (/dev/sdb5)
[ 31.540423] sysrq: Resetting
generate prestaged.iso script output logs:

generate prestaged.iso with ks-setup.cfg:
sudo ./gen-prestaged-iso.sh --input /opt/dc-vault/loads/22.12/bootimage.iso --setup ks-setup.cfg --images container-image1.tar.gz --output /opt/backups/prestaged.iso
/var/rootdirs/opt/backups/prestage-files/updateiso_build_f778mP
mount: /var/rootdirs/opt/backups/prestage-files/stx-iso-utils_mnt_7LGpwZ: WARNING: source write-protected, mounted read-only.
INFO: Generating prestage.iso from params:
INFO: Parameters:
INFO: COMMON_ARGS: initrd=/initrd instdate=@1668456790 instw=60 instiso=instboot biosplusefi=1 instnet=0 ks=file:///kickstart/kickstart.cfg rdinit=/install instname=debian instbr=starlingx instab=0 insturl=file://NOT_SET prestage ip= BLM=2506 FSZ=32 BSZ=512 RSZ=20480 VSZ=20480 instl=/ostree_repo instdev= defaultkernel=vmlinuz*[!t]-amd64
Warning: creating filesystem that does not conform to ISO-9660.
isohybrid: Warning: more than 1024 cylinders: 3682
isohybrid: Not all BIOSes will be able to boot this device
Note: we can see the the "INFO: Parameters" is empty.

gen-prestaged-iso.sh script:

Alarms
--------------
n/a

Test Activity
--------------
 Feature Testing

Workaround
---------------
use --param to define the boot and rootfs device

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/metal/+/865207
Committed: https://opendev.org/starlingx/metal/commit/15e971b59e02042ebac3f19bab197b62a5b88693
Submitter: "Zuul (22348)"
Branch: master

commit 15e971b59e02042ebac3f19bab197b62a5b88693
Author: Shrikumar Sharma <email address hidden>
Date: Mon Nov 21 23:13:41 2022 +0000

    Fix failure to set instdev parameter when we use ks-setup.cfg

    When we create the prestage iso with an external script,
    ks-setup.cfg, we may not provide the rootfs_device or
    boot_device parameter. This is a valid scenario where these
    parameters are defined in ks-setup.cfg. An installation failure
    is observed in this case.

    The cause of the failure is that the prestage code is handled in
    a pre-part hook. This commit moves it to a ks-early hook.

    In this commit, a provision for the execution of a custom script
    named ks-addon.cfg is also made. This script is a bash script
    that must execute in the last post hook.

    Test Plan:
    PASS: Verify that the installation succeeds when the rootfs
          and boot device parameters are only specified via
          the ks-setup.cfg.

    PASS: Verify that the external script, ks-addon.cfg, is executed
          after the install and configurations are done.

    PASS: Verify that the logs from the execution of ks-addon.cfg
          are present in kickstart.log.

    Closes-bug: 1997305

    Signed-off-by: Shrikumar Sharma <email address hidden>
    Change-Id: Ica1735aef3ab457cf0609ebee6aac45671e97987