STX installation failed

[Juan Carlos Alonso]

Title
-----
STX ISO installation FAILED

Brief Description
-----------------
ISO from 2019-03-12 failed during installation:

09:36:03.670 - INFO - Starting installer ...
Starting Dump dmesg to /var/log/dmesg...
Pane is deadK
  File "/usr/lib64/python2.7/site-packages/pyanaconda/iutil.py", line 40, in <module>
    import subprocess32 as subprocess
ImportError: No module named subprocess32
Wait for Plymouth Boot Screen to Quit

Severity
--------
<Critical: System/Feature is not usable after the defect>

Steps to Reproduce
------------------
Current steps to install ISO

Expected Behavior
------------------
Installation success

Actual Behavior
----------------
Installation failed

Reproducibility
---------------
<Reproducible/Intermittent>
100%

System Configuration
--------------------
Any configuration in Virtual and Bare Metal environments

[Juan Carlos Alonso (juancarlosa)]

ISO: http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/20190312T163454Z/

[Ghada Khalil (gkhalil)]

Marking as release gating; critical priority given installs are failing.

Assigning to Cindy's to investigate. This is the first load after the CentOS 7.6 rebase.

[Ghada Khalil (gkhalil)]

This appears to be an issue with the installer build scripts; assigning to Scott for further investigation.

[Juan Carlos Alonso (juancarlosa)]

Issue present on ISO 20190313
http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/20190313T142005Z/outputs/iso/

[Scott Little (slittle1)]

Workaround fix is available as of http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/20190313T203628Z/outputs/iso/

[Scott Little (slittle1)]

The CENGN build does not delete stx-tools/centos-mirror-tools/output between build. This speeds builds by reusing rpms, tarballs and other files downloaded by prior builds. This works great for versioned files like rpms. It dos not work at all for non-versioned files like squashfs.img. 'download_mirrors.sh', via it's daughter script 'dl_other_from_centos_repo.sh', was happy to say 'Already downloaded' and left us with the 7.4 version of squashfs.img, rather than replace it with the 7.6 version.

Sort term work around will be to delete all the non-versioned files created by 'dl_other_from_centos_repo.sh' as found in the CENGN build directories.

A longer term solution will be to fix dl_other_from_centos_repo.sh. Possibilities are ...
1) checksum the file vs a checksum stored in a modified other_downloads.lst
2) Store the files in a versioned subdirectory. Perhaps add symlinks to bridge the gap from old non-versioned path to versioned path so that the rest of the build scripts can stay as is.

[Scott Little (slittle1)]

Workaround is to delete directories ...
   .../stx-tools/centos-mirror-tools/output/stx-r1/CentOS/pike/Binary/EFI
   .../stx-tools/centos-mirror-tools/output/stx-r1/CentOS/pike/Binary/images
   .../stx-tools/centos-mirror-tools/output/stx-r1/CentOS/pike/Binary/isolinux
   .../stx-tools/centos-mirror-tools/output/stx-r1/CentOS/pike/Binary/LiveOS

and then re-run the 'download_mirrors.sh' step.

The installer is now building correctly, and has been verified in several labs.

A longer term fix will be tracked by https://storyboard.openstack.org/#!/story/2005234