FluxCD: Apps cannot be reapplied if http port is updated

[Thiago Paiva Brito]

Brief Description
-----------------
For the FluxCD apps, the helmrepository defined on the fluxcd-manifests folder is only changed at upload time. If we change the port at run-time, the app loses the ability to download new charts since the helmrepository will still be configured to the old port.

Severity
--------
Major: System/Feature is usable but degraded

Steps to Reproduce
------------------
system service-parameter-modify http config http_port=8877;system service-parameter-apply http
Apply app that was uploaded before OR remove/apply an app that was already applied

Expected Behavior
------------------
App fwk downloads the charts and app installs in any case

Actual Behavior
----------------
- If app is uploaded and not applied: App fwk fails to download the charts from the helmrepository
- If app was already applied: App can be reapplied but not removed/applied
- "kubectl get helmrepository" -A shows errors for all resources

Reproducibility
---------------
3 out of 3 attempts

System Configuration
--------------------
All configurations

Branch/Pull Time/Commit
-----------------------
master @ 80f97188 Merge "Fix service_parameter_get_one return in max_cpu_frequency semantic check"

Last Pass
---------
N/A

Timestamp/Logs
--------------
N/A

Test Activity
-------------
Feature Testing

Workaround
----------
- Revert to port 8080
- Remove and reapply the app

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/844340
Committed: https://opendev.org/starlingx/config/commit/a9eb21d688ab98f3446b06681f9c30cb1bf6d2c3
Submitter: "Zuul (22348)"
Branch: master

commit a9eb21d688ab98f3446b06681f9c30cb1bf6d2c3
Author: Thiago Brito <email address hidden>
Date: Wed Jun 1 18:08:01 2022 -0300

    Fix HelmRepository CRDs on lighttpd port change

    This commit fixes the problem of changing the lighttpd port in 2 parts:
    - Added a callback to the puppet runtime manifest apply to call a method
    that will fix the addres on the custom resource if it is already created
    on puppet
    - Modified the FluxCDKustumizeOperator to change the helmrepository.yaml
    when the app is uploaded/applied/reapplied and removed that
    responsibility from kube_app.py.

    TEST PLAN
    PASS change the http port and check if the HelmRepository resource was
         updated on kubernetes
    PASS check if the resource definition on base/helmrepository.yaml was
         uploaded and that helmrepository-orig.yaml was created
    PASS upload the snmp-app and verify that the default port for it's
         helmrepository.yaml was updated and helmrepository-orig.yaml was
         created.

    Closes-Bug: #1977471

    Signed-off-by: Thiago Brito <email address hidden>
    Change-Id: I4ac50bc7dabdb589a1774f2c13dba1f5d16432c5

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/c/starlingx/config/+/845269

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/starlingx/config/+/845269
Committed: https://opendev.org/starlingx/config/commit/5438a4c688e6296338d01371871706440213bd02
Submitter: "Zuul (22348)"
Branch: master

commit 5438a4c688e6296338d01371871706440213bd02
Author: Thiago Brito <email address hidden>
Date: Thu Jun 9 09:45:41 2022 -0300

    Reverting FLUXCD_CRD_HELM_REL_* change on kube_app

    On [1], during a refactoring activity to export some variables to
    constants, the IDE did some code substitutions on a file that wasn't
    meant to be made. This commit will fix it and unblock the bootstrap
    failure.

    [1] https://review.opendev.org/c/starlingx/config/+/844340

    Closes-Bug: #1977471

    Signed-off-by: Thiago Brito <email address hidden>
    Change-Id: Id8f02057bb335f21314e9c06642fe501b786fe80