hosts within two availability zones

[Yingxin]

There have been a lot of bug fixes related to this topic, but it still exists somehow. Some previous fix-released bugs for example:
https://bugs.launchpad.net/nova/+bug/1200479
https://bugs.launchpad.net/nova/+bug/1196893
https://bugs.launchpad.net/nova/+bug/1277230

The mailing list has already decided not to allow hosts in different AZs (http://lists.openstack.org/pipermail/openstack-dev/2014-April/031803.html), but it can still be reproduced by following 3 steps:

#### start repro ####

1) create two host aggregates "foo", "bar" to the default AZ:
$ nova aggregate-create foo
+----+------+-------------------+-------+----------+
| Id | Name | Availability Zone | Hosts | Metadata |
+----+------+-------------------+-------+----------+
| 58 | foo | - | | |
+----+------+-------------------+-------+----------+
$ nova aggregate-create bar
+----+------+-------------------+-------+----------+
| Id | Name | Availability Zone | Hosts | Metadata |
+----+------+-------------------+-------+----------+
| 59 | bar | - | | |
+----+------+-------------------+-------+----------+

2) assign a host "node2" to both aggregates
$ nova aggregate-add-host foo node2
+----+------+-------------------+---------+----------+
| Id | Name | Availability Zone | Hosts | Metadata |
+----+------+-------------------+---------+----------+
| 58 | foo | - | 'node2' | |
+----+------+-------------------+---------+----------+
$ nova aggregate-add-host bar node2
+----+------+-------------------+---------+----------+
| Id | Name | Availability Zone | Hosts | Metadata |
+----+------+-------------------+---------+----------+
| 59 | bar | - | 'node2' | |
+----+------+-------------------+---------+----------+

3) change "foo" to a named AZ called "az"
$ nova aggregate-update foo foo az
Aggregate 58 has been successfully updated.
+----+------+-------------------+---------+------------------------+
| Id | Name | Availability Zone | Hosts | Metadata |
+----+------+-------------------+---------+------------------------+
| 58 | foo | az | 'node2' | 'availability_zone=az' |
+----+------+-------------------+---------+------------------------+

#### end repro ####

The third step should NOT happen because it causes "node2" belong to both default AZ and "az" AZ, logically:
$ nova aggregate-details foo
+----+------+-------------------+---------+------------------------+
| Id | Name | Availability Zone | Hosts | Metadata |
+----+------+-------------------+---------+------------------------+
| 58 | foo | az | 'node2' | 'availability_zone=az' |
+----+------+-------------------+---------+------------------------+
$ nova aggregate-details bar
+----+------+-------------------+---------+----------+
| Id | Name | Availability Zone | Hosts | Metadata |
+----+------+-------------------+---------+----------+
| 59 | bar | - | 'node2' | |
+----+------+-------------------+---------+----------+

Interesting thing is, "node2" is actually only belong to the availibility zone "az" if we list all the AZs. Thanks to the previous bug fixings:
$ nova availability-zone-list
+-----------------------+----------------------------------------+
| Name | Status |
+-----------------------+----------------------------------------+
| internal | available |
| |- node1 | |
| | |- nova-conductor | enabled :-) 2015-12-07T13:45:59.000000 |
| | |- nova-consoleauth | enabled :-) 2015-12-07T13:45:59.000000 |
| | |- nova-scheduler | enabled :-) 2015-12-07T13:46:02.000000 |
| | |- nova-cert | enabled :-) 2015-12-07T13:46:01.000000 |
| az | available |
| |- node2 | |
| | |- nova-compute | enabled :-) 2015-12-07T13:46:04.000000 |
+-----------------------+----------------------------------------+

[Sean Dague (sdague)]

I don't really understand what the problem is. Nova only thinks this is in one az. It can still be in multiple aggregates.

[Yingxin (cyx1231st)]

What I'm doing is to successfully make the host"node2" to two different AZs: the default AZ and another AZ named "az".

Host "node2" is belonging to two different aggregates "foo" and "bar", however, "foo" belongs to AZ "az", but "bar" belongs to AZ "default". That's why I think this is a bug.

[Hans Lindgren (hanlind)]

I believe everything works as expected. Where I think your reasoning is wrong is what you think makes a host belong to the default AZ.

As I see things "bar" does not belong to an AZ at all, only aggregates that explicitly define the AZ attribute "belongs" to an AZ (strictly speaking, only hosts in that aggregate belong to the AZ, but anyway).

What makes a host belong to the default AZ? Simply that none of the aggregates it has been added to has the AZ attribute set. So, from your steps above, up until 3) node2 belongs to the default AZ and after 3) it only belongs to the AZ named "az", which also is what the "nova availability-zone-list" command shows.

[Yingxin (cyx1231st)]

As a newbie of OpenStack, I see the documentation[1] says that "Host aggregates can be regarded as a mechanism to further partition an availability zone". So it makes me believe that a host aggregate must belong to an availability zone in order to partition an AZ.

And what you said "the aggregate does not belong to an AZ at all if its AZ attribute is unset" confuses me.

Yes, the internal representation shows that 'node2' is actually belongs to the AZ named "az". But I think the scheduler behavior in step 3) should be treated as a bug. It should be fixed by returning error in executing `nova aggregate-update foo foo az`, or by updating the AZ field of aggregate "bar" to "az".

[1] http://docs.openstack.org/developer/nova/aggregates.html

[Sylvain Bauza (sylvain-bauza)]

So, Hans is right but the model is maybe a bit confusing. Let me explain :

The default AZ (eg. "nova") defined by the default_availability_zone Opt is only for allowing the AZ REST resource to show all the hosts behind a single AZ if not set. That doesn't imply that the host is attached to an aggregate explicitely defined as an AZ for partitioning.