Need a way to set cross_az_attach in nova.conf

[Vern Hart]

We have a deployment with multiple availability zones and we want to ensure that volumes attached to instances come from the same zone. There is a nova.conf directive, cross_az_attach, which, if false, requires volumes to be in the same az as the instance. More importantly, when nova boots from volume where the source is an image, then nova will attempt to create the volume in the same availability zone.

Unfortunately, we can't use config-flags for this because cross_az_attach is in the [cinder] section of nova.conf.

[Nobuto Murata (nobuto)]

https://jaas.ai/nova-cloud-controller#charm-config-cross-az-attach

Do you actually need it to nova-compute as well?

[Vern Hart (vern)]

Yes, this needs to be in nova.conf on computes as well. We already had cross_az_attach=false in nova-cloud-controller. When cross_az_attach=false is in nova.conf on compute node, we see error messages that tell us it's trying to create the volume in the same zone:

  nova.exception.BuildAbortException: Build of instance 850f7e99-5111-4d80-9e75-dabb3b20d80b aborted: Invalid input received: Availability zone \'zone2\' is invalid. (HTTP 400)

When cross_az_attach is missing (default is true) from the nova.conf on the compute we get errors that show it's trying to use volumes from the default backend (which is zone1), which will fail in this environment.

[Nobuto Murata (nobuto)]

Sounds like you are testing this new behavior in Ussuri?

https://opendev.org/openstack/nova/commit/07a24dcef7ce6767b4b5bab0c8d3166cbe5b39c0
> * `Bug 1694844 <https://bugs.launchpad.net/nova/+bug/1694844>`_. This is
> fixed in the 21.0.0 (Ussuri) release by using the volume zone for the
> server being created if the server is created without an explicit zone,
> :oslo.config:option:`default_schedule_zone` is None, and the volume zone
> does not match the value of
> :oslo.config:option:`default_availability_zone`.

[Vern Hart (vern)]

Perhaps this should be something that comes from nova-cloud-controller relation?

[Vern Hart (vern)]

Nobuto: If I understand that commit, that's for the scenario where you are not specifying a zone for the nova instance. We are always explicitly setting the zone for new VMs so this wouldn't apply to us. Additionally, this deployment is using train so that commit doesn't exist here.

[Billy Olsen (billy-olsen)]

Patches proposed - https://review.opendev.org/q/topic:%22bug%252F1899084%22+(status:open%20OR%20status:merged)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-nova-cloud-controller/+/786090
Committed: https://opendev.org/openstack/charm-nova-cloud-controller/commit/198c9e7d6e7197bc26198737124df7a658b8df56
Submitter: "Zuul (22348)"
Branch: master

commit 198c9e7d6e7197bc26198737124df7a658b8df56
Author: Billy Olsen <email address hidden>
Date: Tue Apr 13 11:38:56 2021 -0700

    Set cross_az_attach setting on cloud-compute relation

    Communicate to compute services the cross_az_attach config setting.
    Since the cross_az_attach setting needs to be applied at the compute
    node, update the relation settings to specify the cross_az_attach
    policy configured.

    Change-Id: I71e97453453d5d091449caf547e68c6455d091cf
    Closes-Bug: #1899084

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/c/openstack/charm-nova-compute/+/786091
Committed: https://opendev.org/openstack/charm-nova-compute/commit/17e44e12e328922ec9e488ff29212e3e5cc94da6
Submitter: "Zuul (22348)"
Branch: master

commit 17e44e12e328922ec9e488ff29212e3e5cc94da6
Author: Billy Olsen <email address hidden>
Date: Tue Apr 13 11:41:46 2021 -0700

    Set cross_az_attach mode from ncc relation configuration

    The cross_az_attach property needs to be configured on the compute
    nodes. The policy is set on the ncc service and is propigated to the
    compute nodes on the cloud-compute relation. Update the relevant cinder
    config setting based on the value provided.

    Note, the default value for cross_az_attach is aligned with the nova
    default of True.

    Closes-Bug: #1899084
    Change-Id: I7d00b50acbfe05dfd943a3511126b507fc570aeb