Reviewed:  https://review.opendev.org/c/starlingx/openstack-armada-app/+/896818
Committed: https://opendev.org/starlingx/openstack-armada-app/commit/eaa8b41cb0ce3d251903d7dfe3ece3b0a58a7a09
Submitter: "Zuul (22348)"
Branch:    master

commit eaa8b41cb0ce3d251903d7dfe3ece3b0a58a7a09
Author: Lucas de Ataides <email address hidden>
Date:   Thu Sep 28 14:03:49 2023 -0300

    Allow VMs to be created via volumes
    
    After STX-Openstack upversioned to Antelope, we noticed that it was not
    possible to create VMs by volume, as they would be stuck on ERROR
    status. The first proposed solution was to create a patch containing
    [1] and [2], because, as specified in [3], Nova now requires
    a service token in order to be able to manipulate Cinder volumes. This
    unfortunately did not solve the issue by itself, as now an error message
    showed up on the nova-conductor pods with the following (not full error
    message, only important part): "nova.exception.RescheduledException:
    Build of instance 2f32c7ea-1720-4f61-bce8-dbe970c40b0c was re-scheduled:
    Secret not found: no secret with matching uuid 'a7f3ae2e-cee7-4f04-9402
    -a78047747654". This UUID was not the same one present when issuing
    `virsh secret-list` on Cinder, Nova and Libvirt containers.
    
    Turns out openstack-helm and openstack-helm-infra have a Ceph UUID
    hardcoded in them, in Cinder [4], Nova [5] [6] and Libvirt [7] values.
    By changing these values to the UUID that libvirt was trying to find
    (7f3ae2e-cee7-4f04-9402-a78047747654), and it worked to solve the issue.
    However, it is not a good practice to use hardcoded values, and,
    searching on where this UUID was coming from, it turns out it was
    defined by the platform's Ceph configuration under
    `/etc/ceph/ceph.conf`.
    
    This still leaves the question, why was this working on Ussuri and
    stopped working on Antelope? First of all, the Ceph official
    documentation [8] [9] about using it with OpenStack explains the
    process of adding the secret to libvirt, to store the ceph admin
    keyring. You can see that the secret uuid is generated "on the fly" and
    both docs mention that old/hard-coded value
    (i.e., 457eb676-33da-42ec-9a8c-9293d545c337). This is the reason why it
    used to work until our upversion to OpenStack Antelope/2023.1: this
    UUID does not really matter (as long as nova and libvirt have the same
    value for it)! It is a given UUID to the libvirt secret that will store
    ceph keyring [10], and the key ring will ensure proper communication
    between our services and the platform ceph.
    
    What changed between Ussuri and Antelope (2023.1), is that now there is
    a specific method [11] to set a default value (Ceph's Cluster UUID) for
    this UUID when it is not specified in the driver configuration.
    
    What this change does is dynamically read this `/etc/ceph/ceph.conf`
    file to search for the UUID value, and use it to override the [4] [5]
    [6] and [7] values. It also adds the patch including the Nova service
    token configuration. The combination of these 2 changes allows VMs to be
    created by volumes.
    
    [1] https://opendev.org/openstack/openstack-helm/commit/91c8a5baf2cf2f0dddded57d88f00ea11dd4ff4a
    [2] https://opendev.org/openstack/openstack-helm/commit/7d39af25fddbf5fc67e15c92a9265f28567a214e
    [3] https://docs.openstack.org/releasenotes/cinder/2023.1.html#upgrade-notes
    [4] https://opendev.org/openstack/openstack-helm/src/branch/master/cinder/values.yaml#L942
    [5] https://opendev.org/openstack/openstack-helm/src/branch/master/nova/values.yaml#L594
    [6] https://opendev.org/openstack/openstack-helm/src/branch/master/nova/values.yaml#L1432
    [7] https://opendev.org/openstack/openstack-helm-infra/src/branch/master/libvirt/values.yaml#L100
    [8] https://github.com/ceph/ceph/blob/main/doc/rbd/rbd-openstack.rst
    [9] https://docs.huihoo.com/ceph/v0.80.5/rbd/rbd-openstack/index.html
    [10] https://opendev.org/starlingx/openstack-armada-app/src/branch/master/python3-k8sapp-openstack/k8sapp_openstack/k8sapp_openstack/helm/libvirt.py#L60
    [11] https://opendev.org/openstack/cinder/commit/6464d37d1661f5dfd58edec399a8e7e39c757b03#diff-9b122c182b4333b747e7fd7e07f73d68ff30256a
    
    Test Plan:
    PASS: Build openstack-helm, python3-k8sapp-openstack and
          stx-openstack-helm-fluxcd
    PASS: Upload / apply / remove STX-Openstack
    PASS: Create a VM by an image
    PASS: Create a volume and launch a VM from it
    PASS: Create a VM using the `boot-from-volume` flag
    PASS: Delete a VM created by a volume
    
    Closes-Bug: 2037463
    
    Change-Id: Ia00bb8dbe3460ce817d69049f97f56a96ad6a298
    Signed-off-by: Lucas de Ataides <email address hidden>