cinder services doesn't honor _topic setting in config

[Endre Karlson]

I found this by porting the c.services module to designate, I can't see that the CONF.volume_topic is read anywhere?

2014-04-03 13:24:40.301 DEBUG cinder.openstack.common.service [-] volume_topic = volumes from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1913
2014-04-03 13:24:40.303 DEBUG cinder.openstack.common.service [-] volume_usage_audit_period = month from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1913
2014-04-03 13:24:40.308 DEBUG cinder.openstack.common.service [-] volumes_dir = /opt/stack/data/cinder/volumes from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1913
2014-04-03 13:24:40.311 DEBUG cinder.openstack.common.service [-] zoning_mode = none from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1913
2014-04-03 13:24:40.313 DEBUG cinder.openstack.common.service [-] fc-zone-manager.fc_fabric_names = None from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1921
2014-04-03 13:24:40.314 DEBUG cinder.openstack.common.service [-] fc-zone-manager.fc_san_lookup_service = cinder.zonemanager.drivers.brocade.brcd_fc_san_lookup_service.BrcdFCSanLookupService from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1921
2014-04-03 13:24:40.319 DEBUG cinder.openstack.common.service [-] fc-zone-manager.zone_driver = cinder.zonemanager.drivers.brocade.brcd_fc_zone_driver.BrcdFCZoneDriver from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1921
2014-04-03 13:24:40.320 DEBUG cinder.openstack.common.service [-] fc-zone-manager.zoning_policy = initiator-target from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1921
2014-04-03 13:24:40.321 DEBUG cinder.openstack.common.service [-] keymgr.api_class = cinder.keymgr.conf_key_mgr.ConfKeyManager from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1921
2014-04-03 13:24:40.322 DEBUG cinder.openstack.common.service [-] database.backend = sqlalchemy from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1921
2014-04-03 13:24:40.323 DEBUG cinder.openstack.common.service [-] database.use_tpool = False from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1921
2014-04-03 13:24:40.325 DEBUG cinder.openstack.common.service [-] ******************************************************************************** from (pid=4264) log_opt_values /opt/stack/oslo.config/oslo/config/cfg.py:1923
2014-04-03 13:24:40.450 DEBUG cinder.openstack.common.processutils [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C pvs --noheadings --unit=g -o vg_name,name,size,free --separator : --nosuffix from (pid=4269) execute /opt/stack/cinder/cinder/openstack/common/processutils.py:142
2014-04-03 13:24:40.631 DEBUG cinder.openstack.common.processutils [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings --unit=g -o name,size,free,lv_count,uuid --separator : --nosuffix stack-volumes from (pid=4269) execute /opt/stack/cinder/cinder/openstack/common/processutils.py:142
2014-04-03 13:24:40.821 DEBUG cinder.openstack.common.lockutils [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Got semaphore "dbapi_backend" for method "__get_backend"... from (pid=4269) inner /opt/stack/cinder/cinder/openstack/common/lockutils.py:191
sel2014-04-03 13:24:41.106 DEBUG cinder.volume.manager [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Re-exporting 0 volumes from (pid=4269) init_host /opt/stack/cinder/cinder/volume/manager.py:254
2014-04-03 13:24:41.107 DEBUG cinder.volume.manager [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Resuming any in progress delete operations from (pid=4269) init_host /opt/stack/cinder/cinder/volume/manager.py:293
2014-04-03 13:24:41.108 INFO cinder.volume.manager [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Updating volume status
2014-04-03 13:24:41.109 DEBUG cinder.volume.drivers.lvm [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Updating volume stats from (pid=4269) _update_volume_stats /opt/stack/cinder/cinder/volume/drivers/lvm.py:345
2014-04-03 13:24:41.110 DEBUG cinder.openstack.common.processutils [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C vgs --noheadings --unit=g -o name,size,free,lv_count,uuid --separator : --nosuffix stack-volumes from (pid=4269) execute /opt/stack/cinder/cinder/openstack/common/processutils.py:142
f.2014-04-03 13:24:41.300 DEBUG cinder.manager [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Notifying Schedulers of capabilities ... from (pid=4269) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:128
2014-04-03 13:24:41.301 DEBUG oslo.messaging._drivers.amqp [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] UNIQUE_ID is 0b9673838e00493ea3672bd6d451d52f. from (pid=4269) _add_unique_id /opt/stack/oslo.messaging/oslo/messaging/_drivers/amqp.py:338
2014-04-03 13:24:41.302 DEBUG oslo.messaging._drivers.amqp [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Pool creating new connection from (pid=4269) create /opt/stack/oslo.messaging/oslo/messaging/_drivers/amqp.py:70
2014-04-03 13:24:41.320 INFO oslo.messaging._drivers.impl_rabbit [req-bff25e2e-56a3-4eba-b335-095dc5ddc086 None None] Connected to AMQP server on 172.16.28.100:5672
2014-04-03 13:24:41.338 DEBUG cinder.service [-] Creating RPC server for service cinder-volume from (pid=4269) start /opt/stack/cinder/cinder/service.py:113

[Mike Perez (thingee)]

There are quite a few places the volume_topic from config is checked. Can you please double check the start up output from each service after changing the volume_topic in the config file. You should see where it lists the volume_topic it has found from config, otherwise it defaults to 'cinder-volume'. I just verified this myself.

[Sean McGinnis (sean-mcginnis)]

Closing stale bug. If this is still an issue please reopen.

[John Griffith (john-griffith)]

Appears this may be a valid issue. I've been trying to test some things with a different topic name and noted it results in an inability to send rpc messages. Looking through the code there are plenty of places that we reference the CONF variable, but few that we actually set anything from it.

We should get this triaged probably and see about fixing it up.

[Michal Dulko (michal-dulko-f)]

I believe the correct fix is starting to actually honor these configuration options by passing them into cinder.service.Service.create in each of the cinder.cmd.foo, where foo is all, scheduler, volume, backup.

I don't see any obvious blockers for that from my side. Question is - should be get back to using get_by_host_and_topic, that we're switching from in https://launchpad.net/bugs/1552006 ?

[Ivan Kolodyazhny (e0ne)]

Could it break RPC? We use it like https://github.com/openstack/cinder/blob/master/cinder/volume/rpcapi.py#L99

[Michal Dulko (michal-dulko-f)]

@Ivan: We've decided to remove these in the meeting. Anyway we were sending to CONF.volume_topic, but receiving from hardcoded 'cinder-volume', so any change of this config option would explode our RPC.

[Lisa Li (lisali)]

Hi vivekd, send the bug to you as you said you would do it in cinder meeting.

[Vivek Dhayaal (vivekdhayaal)]

Thank you Lisa Li.

Below is the meeting discussion for reference:
http://eavesdrop.openstack.org/irclogs/%23openstack-meeting/%23openstack-meeting.2016-03-02.log.html#t2016-03-02T16:29:24

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/288398

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/288398
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=f6c20ed00b9febaea42ac34d2988c1cf6c883b10
Submitter: Jenkins
Branch: master

commit f6c20ed00b9febaea42ac34d2988c1cf6c883b10
Author: Vivek Dhayaal <email address hidden>
Date: Wed Mar 2 22:14:15 2016 +0530

    Removed RPC topic config options

    Cinder had config options to customize the RPC topics on which the
    scheduler, volume and backup nodes listen. But this feature has been
    dysfunctional for quite a long time. This commit removed this feature.
    For more details: Refer the bug comments.

    DocImpact
    Change-Id: Ie76f070fe9a1222c209e8defd0d04fa7a7931b14
    Closes-Bug: #1301888

[Thierry Carrez (ttx)]

This issue was fixed in the openstack/cinder 9.0.0.0b3 development milestone.