cell conductor tries to connect the other cells's DB when it is started

[Shi Yan]

Description
===========

We observe some nova-conductor behavior during its startup, and think that might be not that correct, would like to see if any improvement can be made for this.

Steps to reproduce
==================

- We have a large cellv2 setup, the nova services are running in victoria version.
- Just start/restart nova-conductor in one cell and check logs

Actual result
=============

The conductor will try to connect other cell's DB in order to get get_minimum_version_all_cells.
Since each cell is guarded by firewall, the connection attempt is failed and has to wait for its timeout(60 seconds) to start the following nova-conductor services.

This will be fine for the superconductor but the cell conductor usually cannot reach other cell's DB. Wonder if this behavior can be changed, or there are some other considerations behind this?

[Uggla (rene-ribaud)]

Sorry this is not a bug and more a feature request.
Thanks to open a blueprint for that one.

[Uggla (rene-ribaud)]

Note: If you think this is a bug, thanks answer and provide the following info:
- steps to reproduce
- the version of Nova and the novaclient (or os-client)
- logs (on debug level)
- environment details depending on the bug
        libvirt/kvm versions, VMWare version, ...
        storage type (ceph, LVM, GPFS, ...)
        network type (nova-network or neutron)

[Shi Yan (yanshi-403)]

Thanks, Could I double confirm such observed behavior(cell conductor will also try to connect all the cell dbs during its startup) is expected?

[Sam Morrison (sorrison)]

Been looking into this, this is definitely a bug. The issue is because cell conductors still need to be configured with the API database to allow for some upcalls https://docs.openstack.org/nova/latest/admin/cells.html#operations-requiring-upcalls . Unless this doc is out of date?

When api_database is set it'll cause nova-conductor to check all cells service versions.

It has in nova/utils.py
if CONF.api_database.connection is not None:

I think this check needs to be different, I'm not sure though if there is a good way to determine if the conductor is running at api level of cell level. The check for the api_database isn't good enough I think.