stx-openstack-helm build failure: garbd

[Scott Little]

Brief Description
-----------------
CENGN flock layer, master branch build failure in stx-openstack-helm-1.0-140.tis.src.rpm.
It appears to be a post build unit test that is failing.

The same issue was seen in the monolithic build.

Severity
--------
Major

Steps to Reproduce
------------------
build-pkgs

Expected Behavior
------------------
all packages build

Actual Behavior
----------------
stx-openstack-helm failes to build

Reproducibility
---------------
100%

System Configuration
--------------------
N/A

Branch/Pull Time/Commit
-----------------------
master
2021-12-15 05:30:00Z

Last Pass
---------
2021-12-14

Timestamp/Logs
--------------
03:48:29 Failed to build packages: stx-openstack-helm-1.0-140.tis.src.rpm

...
Update Complete. ⎈Happy Helming!⎈
Saving 1 charts
Downloading helm-toolkit from repo http://localhost:8879/charts
BUILDSTDERR: 2021-12-15T03:48:24.932Z DEBUG [7] Incoming request: /charts/charts/helm-toolkit-0.2.19.tgz {"reqID": "eb6411b5-d26a-4803-ac7c-e36c8ff213ce"}
BUILDSTDERR: 2021-12-15T03:48:24.932Z INFO [7] Request served {"path": "/charts/charts/helm-toolkit-0.2.19.tgz", "comment": "", "latency": "256.277µs", "clientIP": "127.0.0.1", "method": "GET", "statusCode": 200, "reqID": "eb6411b5-d26a-4803-ac7c-e36c8ff213ce"}
Deleting outdated charts
if [ -d garbd ]; then helm lint garbd; fi
==> Linting garbd
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: template: garbd/templates/deployment.yaml:50:13: executing "garbd/templates/deployment.yaml" at <$envAll.Values.pod.tolerations.garbd.enabled>: nil pointer evaluating interface {}.enabled
BUILDSTDERR: Error: 1 chart(s) linted, 1 chart(s) failed
BUILDSTDERR: make[1]: *** [lint-garbd] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/stx-openstack-helm-1.0/helm-charts'
BUILDSTDERR: make: *** [garbd] Error 2

Test Activity
-------------
Build

Workaround
----------
N/A

[Scott Little (slittle1)]

I don't see anything in the change log relating to stx-openstack-helm.

Perhaps an upstream change impacting unit test?

[Scott Little (slittle1)]

This issue was NOT reproduced on the r/stx.6.0 branch.

This rules out an upstream issues.

perhaps one of these ...
./cgcs-root/stx/openstack-armada-app b36261adbdaa048940389d8c32454f3f22f94da9 2021-12-13 19:32:15 +0000 Hugo Nicodemos <email address hidden> Add toleration to garbd pod
./cgcs-root/stx/openstack-armada-app 9dc289bfc77d3638c9107e30392a28c17328dedb 2021-12-09 17:09:19 +0000 Hugo Nicodemos <email address hidden> Add NoExecute toleration for mariadb and rabbitmq
./cgcs-root/stx/cert-manager-armada-app 2b6cf573b3a0d485b7a972d2d92645d5684c6c14 2021-12-02 13:00:31 -0600 Tracey Bogue <email address hidden> Debian packaging: Fix manual pycache removal

[Ghada Khalil (gkhalil)]

This seems to be related to this change in stx master on Dec 14
https://review.opendev.org/c/starlingx/openstack-armada-app/+/821591

which may be fixed by this follow-up change today
https://review.opendev.org/c/starlingx/openstack-armada-app/+/821845

[Ghada Khalil (gkhalil)]

screening: build issue in stx master only; does not affect r/stx.6.0 release

[Kyle MacLeod (kmacleod)]

From my local build seeing this same error, I see this in the build.log:

if [ -d garbd ]; then helm lint garbd; fi
==> Linting garbd
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: template: garbd/templates/deployment.yaml:50:13: executing "garbd/templates/deployment.yaml" at <$envAll.Values.pod.tolerations.garbd.enabled>: nil pointer evaluating interface {}.enabled
Error: 1 chart(s) linted, 1 chart(s) failed
make[1]: *** [lint-garbd] Error 1
make[1]: Leaving directory `/builddir/build/BUILD/stx-openstack-helm-1.0/helm-charts'
make: *** [garbd] Error 2
RPM build errors:
error: Bad exit status from /var/tmp/rpm-tmp.8M9M4j (%build)
    Bad exit status from /var/tmp/rpm-tmp.8M9M4j (%build)
Child pid '155408' is dead
Child dead, killing orphans
Child return code was: 1
EXCEPTION: [Error()]
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/mockbuild/trace_decorator.py", line 88, in trace
    result = func(*args, **kw)
  File "/usr/lib/python2.7/site-packages/mockbuild/util.py", line 547, in do
    raise exception.Error("Command failed. See logs for output.\n # %s" % (command,), child.returncode)
Error: Command failed. See logs for output.
 # bash --login -c /usr/bin/rpmbuild -bb --target x86_64 --nodeps /builddir/build/SPECS/stx-openstack-helm.spec

There are a couple of commits yesterday and today which look relevant, so hopefully this is already fixed:

c34e08c 2021-12-15 10:06 Hugo Nicodemos (HEAD, starlingx/master, m/master) Fix typo on garbd

 stx-openstack-helm/stx-openstack-helm/helm-charts/garbd/values.yaml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

b36261a 2021-12-13 12:04 Hugo Nicodemos Add toleration to garbd pod

 stx-openstack-helm/stx-openstack-helm/helm-charts/garbd/templates/deployment.yaml | 3 +++
 stx-openstack-helm/stx-openstack-helm/helm-charts/garbd/templates/job-image-repo-sync.yaml | 3 +++
 stx-openstack-helm/stx-openstack-helm/helm-charts/garbd/values.yaml | 4 ++++
 3 files changed, 10 insertions(+)

[Ghada Khalil (gkhalil)]

Assigning to Hugo Nicodemos as he is the author of the 2 commits I reference above.
He can confirm if the build issue is fixed by his 2nd review: https://review.opendev.org/c/starlingx/openstack-armada-app/+/821845

[Douglas Lopes Pereira (douglaspereira)]

Fix released by Hugo as part of https://review.opendev.org/c/starlingx/openstack-armada-app/+/821845