manually add a disabled service to nova-manage

[Scott Devoid]

There is no way in the nova-manage commands to register an additional service node, e.g. I had to manually add the service through the database, which seems like the wrong way to do things. I would suggest:

nova-manage service add --host compute123 --service nova-compute

I would also suggest that any command that does this automatically sets the service to disabled.

[Boris Pavlovic (boris-42)]

If nova.conf is ok and then you run it on some host it will
automatically create DB entry for this compute node in services and compute_nodes tables.

So I don't understand why do you want to add it by hand with nova-manage?

[Scott Devoid (scott-devoid)]

Oh, I didn't realize it did that when the node started. Then the issue is that it brings the node in as "enabled". In our PXE-boot case, we want to bring nodes in as "disabled" and manually switch them when they are ready.

I suspect there are other operations folk who are in the same position: you bring a machine up with the basic OS but there may be firmware updates or performance checks before you're comfortable bringing a node into production.

[Scott Devoid (scott-devoid)]

Suggested usage:
nova-manage service add --host=<hostname> --service=<service-name> --disabled

Use case:
To pre-seed an existing system with disabled services and prevent nova-scheduler from using them until a later time.

[Scott Devoid (scott-devoid)]

I can confirm that this is still an issue in Havana.

[Matt Fischer (mfisch)]

The use case is valid. If I bring-up an extra nova-compute node I would like to run some tests on it to ensure that it can properly spin-up instances before I allow user to schedule jobs on it. I'm just not sure that this is the right solution. Perhaps this should be a mailing list discussion?

[Facundo Farias (facundo-farias)]

Hi Matt, any advancements on this topic? Thanks

[vaibhav (vaibhav-bhatkar)]

How about adding option "service_enabled=<True|False>" in nova.conf which decides the state of service in DB?
This option will default to "True" making system work as it is now. ipxe case can be handled by setting the "service_enabled"
to "False". We can update the DB accordingly on every "nova" restart. So, once we are sure about the setup we can
change "service_enabled" to "True" and restart "nova".

[Scott Devoid (scott-devoid)]

There are a few cases where that approach might work: all our nova.conf files are templates and managed with configuration management software. This would effectively punt the decision from OpenStack to the operator, and whatever software they are using to manage it--which I'm not opposed to.

Would the switch work in the reverse direction? "service_enabled=False" && service nova restart when the database reports the host as up? Would it switch it to disabled? I worry that this could lead to inconsistent state. The more *dumb* config option name would be "register_service_on_startup=<True|False>".

At the end of the day though, I don't see why nova implements a service-tracking framework if you can't *add* services to it. There are also terms that make *no* sense. "Host" is almost never a FQDN, but it's used for the message queues. You can't delete hosts, even if you've already loaded the rack onto a pallet and sent it to the scrapper. And of course the "status" field only reports "XXX" in the most limited of cases.

[vaibhav (vaibhav-bhatkar)]

If we try to support switch in reverse direction, i.e, changing from True to False then first
questions come is what will happen to running instances on compute. As you have rightly
said this is inconsistent. I was thinking that we support "service_enabled=False" only if
there is no entry for the service in the DB. Following is rough algorithm (I am going
to use your config as "service_enabled" has many implications :-) ),

IF There is no entry in the DB for the service THEN
    IF register_service_on_startup == False THEN
        make the entry in the DB for the service with status set to DISABLED
    ELSE
        make the entry in the DB for the service with status set to ENABLED
ELSE IF register_service_on_startup == TRUE AND status for the service in the DB is DISABLED
THEN
    chagne the status for the service in the DB to ENABLED
ELSE IF register_service_on_startup == False AND There is entry in the DB for the service
THEN
    log warning and do nothing
END IF

[Scott Devoid (scott-devoid)]

Well, at least in Essex and Havana, when you disable the nova-compute service on a node that prevents new instances from being scheduled on the node, but it does not affect running instances on the node. So you don't have to worry about that.

> IF register_service_on_startup == False THEN
        make the entry in the DB for the service with status set to DISABLED

I would expect there to be no entry in the DB if register_service_on_startup = False.

[Scott Devoid (scott-devoid)]

@vaibhav, are you still working on this issue? If so, can you link to a review so I can track it? If not, can you release the bug so someone else can take a look at it? Thanks!

[Matt Riedemann (mriedem)]

nova-manage is deprecated, why not use the service API to add the service?

[Matt Riedemann (mriedem)]

I guess there isn't an API extension to add a service, only list/update/delete:

http://git.openstack.org/cgit/openstack/nova/tree/nova/api/openstack/compute/contrib/services.py

[Scott Devoid (scott-devoid)]

I'm fine with using the nova service API endpoint, but we need a service-create command in the client and a create API endpoint.

Adding to that, I see that there *is* a delete endpoint, but no service-delete subcommand or service-disable --delete flag. Should we add this? Should I file a separate bug for that against python-novaclient?

[Scott Devoid (scott-devoid)]

I can document one additional place where this feature would alleviate a pain point:

When populating host-aggregates with nova-compute hosts, the host-aggregates API checks that the host exists in the services table with the nova-compute binary. This poses a problem, though if you are bringing new hardware online and only have a subset of the nova-compute services working (and therefore present in the database).

If you could manually add hosts, this would allow you to pre-populate the services and host-aggregates APIs with the hosts.

[Christopher Yeoh (cyeoh-0)]

I don't think we need a separate bug report for service delete and python-novaclient, but if you have time a nova-spec proposal for a create command for the service endpoint which could cover the delete and create novaclient functionality would be very helpful.

Scott - is this something you want to implement your self or is it just something you'd like to see added?

[Scott Devoid (scott-devoid)]

At the moment I do not have time to implement this myself.

[Qiu Yu (unicell)]

Actually there's an existing config controlling whether or not to register service in enabled state. CONF.enable_new_services for controller nova.conf, since Grizzly version

I think it should be enough to address the original requirement described by Scott in #2

[Markus Zoeller (markus_z) (mzoeller)]

Looks like this is implemented with commit [1] (a new service gets
created in "disabled" mode). This disabled service can then be enabled
with "nova service-enable <hostname> <binary>" [2].

References:
[1] https://git.openstack.org/cgit/openstack/nova/commit/?id=5a25de893f34cb9b05996406488188b6ed47fca1
[2] http://docs.openstack.org/cli-reference/nova.html#nova-service-enable