conflicts between cinder-volume and zeromq driver

Bug #1440631 reported by Li Ma on 2015-04-06
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Vivek Dhayaal

Bug Description

Currently cinder-volume uses topic.hostname@backendname in rpc calls. It is working well with AMQP-based RPC methods, like rabbit.

However, as zeromq driver in oslo_messaging is introduced for large-scale deployment, this topic naming is totally not working, because zeromq driver strictly uses topic.hostname for message delivery.

In zeromq driver, topic string is used for routing and hostname string is the destination. This is also the common usage for oslo_messaging.

I suggest to change it from topic.hostname@backendname to <email address hidden> during rpc.

This change won't affect the current AMQP-based RPC methods, like rabbit, and zeromq driver can be also used for cinder.

By the way, currently, it seems that only cinder cannot take advantage of zeromq driver to scale out.

Tags: zmq Edit Tag help
Li Ma (nick-ma-z) on 2015-04-06
tags: added: zmq
description: updated
John Griffith (john-griffith) wrote :

Hmm... I had thought there was a workaround submitted upstream to zmq to allow an option to specify delims here. Sounds like maybe that's false though. Also there's another version of this bug logged against OSLO (not sure that's appropriate or not, but looks like there are some possibilities to work around it there).

Changed in cinder:
status: New → Confirmed
importance: Undecided → Medium
wuyuting (wytdahu) on 2015-04-08
Changed in cinder:
assignee: nobody → wuyuting (wytdahu)
Li Ma (nick-ma-z) wrote :

@John At first we try to fix it in the oslo.messaging, however, it is impossible to do it. After some discussion, we would like to try to propose a simple fix to cinder to make everything work.

Li Ma (nick-ma-z) wrote :

Hello, do you still work on it?

Sean McGinnis (sean-mcginnis) wrote :

Automatically unassigning due to inactivity.

Changed in cinder:
assignee: wuyuting (wytdahu) → nobody

Fix proposed to branch: master

Changed in cinder:
assignee: nobody → Vivek Dhayaal (vivekdhayaal)
status: Confirmed → In Progress

Submitter: Jenkins
Branch: master

commit 3320a8943827e82a1576d212b1ea076a9381ef63
Author: Vivek Dhayaal <email address hidden>
Date: Sun Jan 24 11:17:37 2016 +0530

    Support ZeroMQ messaging driver in cinder

    NOTE:This patch introduces support for ZeroMQ driver for cinder single
    backend case.
    Multi-backend will be addressed in the next patch as part of the

    CHANGES:ZeroMQ driver requires hostname for message delivery as there is no
    broker inbetween.
    So, extract the hostname and feed to the messaging client for zeromq

    For the record, ZeroMq is a very lightweight distributed messaging system
    specially designed for high throughput/low latency scenarios.
    Addition of support for ZeroMQ would help cinder scale out with high
    performance and be highly available as there is no centralised broker.

    Document the configurations for ZeroMQ driver for Cinder

    Change-Id: Ic4b4301e5d7ca1692fc91155ba53f2dd12f99311
    Closes-Bug: #1440631
    partially Implements bp cinder-zeromq-support

Changed in cinder:
status: In Progress → Fix Released

This issue was fixed in the openstack/cinder development milestone.

Submitter: Jenkins
Branch: master

commit df647d0ccd56b7a10b003b8c7372ed3b5b717cc1
Author: Michał Dulko <email address hidden>
Date: Wed Nov 16 15:34:59 2016 +0100

    Use oslo.messaging topics for multibackend

    We've built our multibackend support by abusing oslo.messaging's
    Target.server and appending '@backend-name' prefix to the hostname.
    This made implementation easier, as we're simply treating multibackends
    as totally separated services.

    While this worked in RabbitMQ, zmq is communicating explicitly using
    hostnames, so appending anything to hostname in Target.server breaks

    This commit modifies the messaging layer of cinder-volume to use
    Target.topic to distinguish backends. This is done by:

    * Making cinder-volume listen on new RPC server, with Target.server
      set to raw hostname, and topic is set to ''.
      'cinder-volume' prefix is added to keep compatibility with Newton's
      services (we're relying on how RabbitMQ transport is implemented in
    * Note that old RPC server listening on 'cinder-volume' topic is left
      there, as we need it to recieve fanout messages from scheduler.
    * When sending a message to cinder-volume, we're sending it using
      Target.topic to route it to correct host and backend. For backward
      compatibility it's controlled by conditional based on RPC version pin.

    Closes-Bug: 1630975
    Related-Bug: 1440631
    Implements: cinder-zeromq-support
    Change-Id: I22efbeb97e11838139e2b33226d1c10094d27c1d

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers