Ansible Bootstrap Configuration failed using stx6.0 image. /var/www/pages/helm_charts/stx-platform doesn't exist

[Alexandru Dimofte]

Brief Description
-----------------
Installing latest stx6.0 image 20211217T221324Z the setup failed at "Ansible Bootstrap Configuration" on all configurations.

Severity
--------
<Critical: System/Feature is not usable due to the defect>

Steps to Reproduce
------------------
Install latest stx6.0 image

Expected Behavior
------------------
Installation should work.

Actual Behavior
----------------
2021-12-19 00:51:36,066 p=11154 u=sysadmin | fatal: [localhost]: FAILED! => changed=true
  cmd:
  - /usr/local/sbin/helm-upload
  - stx-platform
  - /opt/extracharts/armada-0.1.0.tgz
  delta: '0:00:00.003541'
  end: '2021-12-19 00:51:36.047498'
  msg: non-zero return code
  rc: 1
  start: '2021-12-19 00:51:36.043957'
  stderr: ''
  stderr_lines: []
  stdout: /var/www/pages/helm_charts/stx-platform doesn't exist.
  stdout_lines: <omitted>

Reproducibility
---------------
100% reproducible

System Configuration
--------------------
One node system, Two node system, Multi-node system, Dedicated storage

Branch/Pull Time/Commit
-----------------------
6.0 20211217T221324Z

Last Pass
---------
For stx6.0 we don't have yet a green image

Timestamp/Logs
--------------
Will be attached

Test Activity
-------------
Sanity

Workaround
----------
-

[Ghada Khalil (gkhalil)]

screening: stx.6.0 / critical - issue causing a red sanity.
Note: This issue is specific to the r/stx.6.0 release. It's not applicable to the stx master branch

[Ghada Khalil (gkhalil)]

Assigning to the build team to triage as it appears the r/stx.6.0 builds are picking stuff from the stx master branch

[Bin Qian (bqian20)]

stx.6.0 shouldn't have the changes to relocate /www to /var/www
https://opendev.org/starlingx/integ/src/branch/r/stx.6.0/kubernetes/helm/centos/files/helm-upload#L26
vs starlingx master
https://opendev.org/starlingx/integ/src/branch/master/kubernetes/helm/centos/files/helm-upload#L26
The review for the changes https://review.opendev.org/c/starlingx/integ/+/810019

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: r/stx.6.0
Review: https://review.opendev.org/c/starlingx/tools/+/822350

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/tools/+/822350
Committed: https://opendev.org/starlingx/tools/commit/67cd5e3c4c165641965b329eb717ad6c37167f8c
Submitter: "Zuul (22348)"
Branch: r/stx.6.0

commit 67cd5e3c4c165641965b329eb717ad6c37167f8c
Author: Davlet Panech <email address hidden>
Date: Mon Dec 20 13:27:36 2021 -0500

    Update layer cfg files for r/stx.6.0

    Closes-Bug: 1955329
    Signed-off-by: Davlet Panech <email address hidden>
    Change-Id: I7d2353a9464b16c442bc374e5f50a5a9ad7d042c

[Alexandru Dimofte (adimofte)]

I executed sanity on latest RC6.0 image. The issue is still visible:
2021-12-22T14:54:47: sw-patch-agent[727819]: base.py(415): WARNING: Failed to synchronize cache for repo 'platform-base', ignoring this repo.
2021-12-22T14:54:47: sw-patch-agent[727819]: patch_agent.py(539): ERROR: Could not find software group: updates-controller-worker
2021-12-22T14:54:47: sw-patch-agent[727819]: patch_functions.py(68): ERROR: Uncaught exception
Traceback (most recent call last):
  File "/usr/sbin/sw-patch-agent", line 15, in <module>
    main()
  File "/usr/lib64/python2.7/site-packages/cgcs_patch/patch_agent.py", line 924, in main
    pa.query()
  File "/usr/lib64/python2.7/site-packages/cgcs_patch/patch_agent.py", line 541, in query
    for pkg in pkggrp.packages_iter():
AttributeError: 'NoneType' object has no attribute 'packages_iter'
[sysadmin@controller-0 ~(keystone_admin)]$

[Alexandru Dimofte (adimofte)]

Using the RC6.0 20211221T231755Z image the issue is still visible. I attached the collected logs from bare-metal duplex configuration.

[Ghada Khalil (gkhalil)]

Re-opening as sanity is still red: http://lists.starlingx.io/pipermail/starlingx-discuss/2021-December/012576.html

[Al Bailey (albailey1974)]

The updated comment indicates the issue is related to the patch setup tracked in
https://bugs.launchpad.net/starlingx/+bug/1955076

I think the helm path issue is resolved.

[Ghada Khalil (gkhalil)]

Based on Al's comment above, I will mark this LP as Fix Released since the originally reported issue was addressed. We'll track the remaining sanity issue via https://bugs.launchpad.net/starlingx/+bug/1955076