DC subcloud pre-install-failed

Bug #1893990 reported by Anujeyan Manokeran
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
StarlingX
Fix Released
High
Don Penney

Bug Description

Brief Description
-----------------
     DC subcloud pre-install-failed. Workaround given to create a directory sudo mkdir /www/pages/updates/rel-20.06/Packages . With the workaround below error was not seen.

dcmanager subcloud list
+----+-----------+------------+--------------+--------------------+---------+
| id | name | management | availability | deploy status | sync |
+----+-----------+------------+--------------+--------------------+---------+
| 5 | subcloud7 | unmanaged | offline | pre-install-failed | unknown |
| 6 | subcloud6 | unmanaged | offline | pre-install-failed | unknown |
+----+-----------+------------+--------------+--------------------+---------+
[sysadmin@controller-0 ~(keystone_admin)]$ pwd

Expected Behavior
------------------
No failure as description during the install.

Actual Behavior
----------------
As per description pre-install failed.

Reproducibility
---------------
Seen on both subcloud . Expected to see in both subcloud

System Configuration
--------------------
Distributed system DC-1

Branch/Pull Time/Commit
-----------------------
stx master as of 2020-08-26

Last Pass
---------
Loads prior to 2020-08-26

Timestamp/Logs
--------------
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager [-] Failed to update iso ['/usr/local/bin/gen-bootloader-iso.sh', '--input', '/opt/dc-vault/loads/20.06/bootimage.iso', '--www-root', '/opt/platform/iso/20.06', '--id', u'subcloud6', '--boot-hostname', u'subcloud6', '--timeout', '5', '--param', u'vlan=eno2.103:eno2', '--boot-ip', '[2620:10a:a001:a103::1129]', '--default-boot', '3', '--param', 'True', '--boot-netmask', '64', '--boot-interface', u'eno2.103', '--param', 'rootfs_device=/dev/disk/by-path/pci-0000:00:17.0-ata-5.0', '--addon', u'/opt/dc/ansible/subcloud6/ks-addon.cfg', '--base-url', u'http://[2620:10a:a001:a103::1162]:8080/iso/20.06'], : Exception: Failed to update iso ['/usr/local/bin/gen-bootloader-iso.sh', '--input', '/opt/dc-vault/loads/20.06/bootimage.iso', '--www-root', '/opt/platform/iso/20.06', '--id', u'subcloud6', '--boot-hostname', u'subcloud6', '--timeout', '5', '--param', u'vlan=eno2.103:eno2', '--boot-ip', '[2620:10a:a001:a103::1129]', '--default-boot', '3', '--param', 'True', '--boot-netmask', '64', '--boot-interface', u'eno2.103', '--param', 'rootfs_device=/dev/disk/by-path/pci-0000:00:17.0-ata-5.0', '--addon', u'/opt/dc/ansible/subcloud6/ks-addon.cfg', '--base-url', u'http://[2620:10a:a001:a103::1162]:8080/iso/20.06'],
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager Traceback (most recent call last):
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/dcmanager/manager/subcloud_manager.py", line 457, in run_deploy
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager payload['install_values'])
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/dccommon/subcloud_install.py", line 411, in prep
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager self.update_iso(override_path, iso_values)
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager File "/usr/lib/python2.7/site-packages/dccommon/subcloud_install.py", line 368, in update_iso
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager raise Exception(msg)
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager Exception: Failed to update iso ['/usr/local/bin/gen-bootloader-iso.sh', '--input', '/opt/dc-vault/loads/20.06/bootimage.iso', '--www-root', '/opt/platform/iso/20.06', '--id', u'subcloud6', '--boot-hostname', u'subcloud6', '--timeout', '5', '--param', u'vlan=eno2.103:eno2', '--boot-ip', '[2620:10a:a001:a103::1129]', '--default-boot', '3', '--param', 'True', '--boot-netmask', '64', '--boot-interface', u'eno2.103', '--param', 'rootfs_device=/dev/disk/by-path/pci-0000:00:17.0-ata-5.0', '--addon', u'/opt/dc/ansible/subcloud6/ks-addon.cfg', '--base-url', u'http://[2620:10a:a001:a103::1162]:8080/iso/20.06'],
2020-09-02 18:09:13.129 108860 ERROR dcmanager.manager.subcloud_manager

sudo /usr/local/bin/gen-bootloader-iso.sh --input /opt/dc-vault/loads/20.06/bootimage.iso --www-root /opt/platform/iso/20.06 --id subcloud6 --boot-hostname subcloud6 --timeout 5 --param vlan=eno2.103:eno2 --boot-ip [2620:10a:a001:a103::1129] --default-boot 3 --param True --boot-netmask 64 --boot-interface eno2.103 --param rootfs_device=/dev/disk/by-path/pci-0000:00:17.0-ata-5.0 --addon /opt/dc/ansible/subcloud6/ks-addon.cfg --base-url http://[2620:10a:a001:a103::1162]:8080/iso/20.06
mount: /dev/loop0 is write-protected, mounting read-only
rsync: link_stat "/www/pages/updates/rel-20.06/Packages" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]
gen-bootloader-iso.sh[2572863]: Failed to copy patches repo from /www/pages/updates/rel-20.06
ls: cannot access /opt/platform/iso/20.06/nodes: No such file or directory
Cache was expired
0 files removed

Test Activity
-------------
Install subcloud

Ghada Khalil (gkhalil)
Changed in starlingx:
assignee: nobody → Don Penney (dpenney)
Revision history for this message
Ghada Khalil (gkhalil) wrote :

stx.5.0 / high priority - this was introduced by recent code changes to support DC subcloud pre-patching:
https://review.opendev.org/#/c/745355/
https://review.opendev.org/#/c/749099/

description: updated
tags: added: stx.5.0 stx.config stx.distcloud
Changed in starlingx:
importance: Undecided → High
status: New → Triaged
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to utilities (master)

Fix proposed to branch: master
Review: https://review.opendev.org/749623

Changed in starlingx:
status: Triaged → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to utilities (master)

Reviewed: https://review.opendev.org/749623
Committed: https://git.openstack.org/cgit/starlingx/utilities/commit/?id=f43e37b4ea5ed58a55a660cab0b76a3ce0b6532e
Submitter: Zuul
Branch: master

commit f43e37b4ea5ed58a55a660cab0b76a3ce0b6532e
Author: Don Penney <email address hidden>
Date: Wed Sep 2 22:42:41 2020 -0400

    Fix gen-bootloader-iso.sh error on unpatched controller

    On a system that has never been patched, the updates repository may
    not yet have a Packages directory. This update fixes the patch setup
    in gen-bootloader-iso.sh to check for this case and handle it without
    failing.

    This update also improves the platform-kickstarts-pxeboot check to
    look for a pxeboot kickstart, to ensure there isn't a false-positive
    match against an extracted pxe-network-installer patch, which would
    also have a /pxeboot directory.

    Change-Id: I8e24a3ff123f4c2649be52b1a0ce77d830e342f3
    Closes-Bug: 1893990
    Signed-off-by: Don Penney <email address hidden>

Changed in starlingx:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to utilities (f/centos8)

Fix proposed to branch: f/centos8
Review: https://review.opendev.org/c/starlingx/utilities/+/792213

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.