Instance architecture should be reflected in the instance domain

[Belmiro Moreira]

"""
It would be great if Nova supports instances with a different architecture than the host.
An use case would be run aarch64 guests in a x86_64 compute node.
"""

The issue is that nova always uses the architecture from the host when defining the instance domain and not what's defined in the image architecture.
Also, because of this the emulator is not correctly defined.

Almost all the pieces are already there!
- CONF.libvirt.hw_machine_type / or using the instance metadata
(it's defined as expected in the instance domain, I'm using "virt-4.0")

- CONF.libvirt.virt_type
(it's defined as expected in the instance domain, I'm using "qemu")

- Defined the image architecture to "aarch64". Actually Nova reads this property from the image but doesn't use it.

===

The instance creation fails because:

Nova only uses in the domain definition:
<type machine="virt-4.0">hvm</type>

and then libvirt uses the host architecture in the domain definition.
In my case this results in using the x86_64 emulator.

When hardcoding the right architecture in the guest.os_mach_type it works as expected.
if self.os_mach_type is not None:
    type_node.set("arch", 'aarch64')
    type_node.set("machine", self.os_mach_type)

The domain is created correctly:
<type arch='aarch64' machine='virt-4.0'>hvm</type>

<emulator>/usr/bin/qemu-system-aarch64</emulator>

Environment
===========

Tested using the master branch (29/10/2020)

Other
=====

I'm now opening target bugs for the generic issue reported in https://bugs.launchpad.net/nova/+bug/1863728

[Jessie Lass (lassimus)]

I have a functioning implementation here:
I'm happy to take on work related to emulation support for the community.

https://review.opendev.org/c/openstack/nova/+/772156

As a note, I think I uncovered a hole in coverage for native aarch64 cloud users that was masked by the previous implementation.