oslo_messaging.rpc.dispatcher UnicodeDecodeError: 'ascii' codec can't decode byte 0xf3 in position 8: ordinal not in range(128) When trying to attach cinder volume to instance

[[e-mail]]

Appeared suddenly in Openstack version Mitaka, when trying to attach volume to instance or launch instance with root fs based on cinder volume:

Steps to reproduce
=
Not defined

Expected result
=
Shoult be properly logged error from cinder/volume/targets/tgt.py" in line 55 when getting target list. Or another solution, also for next releases of openstack.

logs
=

Here is stacktrace:
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher [req-ff7f048f-4ee3-487f-9103-30798806b351 f5925f97ac2141a1917b385c6090f40b b29283bad4da4af885feb6247b90512f - - -] Exception during message handling: 'ascii' codec can't decode byte 0xf3 in position 8: ordinal not in range(128)
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher Traceback (most recent call last):
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 141, in _dispatch_and_reply
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher incoming.message))
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 193, in _dispatch
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher return self._do_dispatch(endpoint, method, ctxt, args)
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher result = func(ctxt, **new_args)
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1441, in initialize_connection
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher volume, connector)
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/lvm.py", line 760, in create_export
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher volume_path)
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/targets/iscsi.py", line 210, in create_export
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher **portals_config)
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/targets/tgt.py", line 220, in create_iscsi_target
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher tid = self._get_target(iqn)
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher File "/usr/lib/python2.7/dist-packages/cinder/volume/targets/tgt.py", line 55, in _get_target
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher if iqn in line:
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher UnicodeDecodeError: 'ascii' codec can't decode byte 0xf3 in position 8: ordinal not in range(128)
2019-04-05 09:32:45.646 5723 ERROR oslo_messaging.rpc.dispatcher
2019-04-05 09:32:45.650 5723 ERROR oslo_messaging._drivers.common [req-ff7f048f-4ee3-487f-9103-30798806b351 f5925f97ac2141a1917b385c6090f40b b29283bad4da4af885feb6247b90512f - - -] Returning exception 'ascii' codec can't decode byte 0xf3 in position 8: ordinal not in range(128) to caller
2019-04-05 09:32:45.650 5723 ERROR oslo_messaging._drivers.common [req-ff7f048f-4ee3-487f-9103-30798806b351 f5925f97ac2141a1917b385c6090f40b b29283bad4da4af885feb6247b90512f - - -] ['Traceback (most recent call last):\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 141, in _dispatch_and_reply\n incoming.message))\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 193, in _dispatch\n return self._do_dispatch(endpoint, method, ctxt, args)\n', ' File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 127, in _do_dispatch\n result = func(ctxt, **new_args)\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/manager.py", line 1441, in initialize_connection\n volume, connector)\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/drivers/lvm.py", line 760, in create_export\n volume_path)\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/targets/iscsi.py", line 210, in create_export\n **portals_config)\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/targets/tgt.py", line 220, in create_iscsi_target\n tid = self._get_target(iqn)\n', ' File "/usr/lib/python2.7/dist-packages/cinder/volume/targets/tgt.py", line 55, in _get_target\n if iqn in line:\n', "UnicodeDecodeError: 'ascii' codec can't decode byte 0xf3 in position 8: ordinal not in range(128)\n"]
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher [req-ff7f048f-4ee3-487f-9103-30798806b351 f5925f97ac2141a1917b385c6090f40b b29283bad4da4af885feb6247b90512f - - -] The dispatcher method must catches all exceptions
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher Traceback (most recent call last):
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/dispatcher.py", line 59, in run
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher self._result = self._dispatch(self._incoming)
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/rpc/dispatcher.py", line 154, in _dispatch_and_reply
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher incoming.reply(failure=exc_info)
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 104, in reply
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher log_failure=log_failure)
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/amqpdriver.py", line 64, in _send_reply
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher log_failure)
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_messaging/_drivers/common.py", line 201, in serialize_remote_exception
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher json_data = jsonutils.dumps(data)
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher File "/usr/lib/python2.7/dist-packages/oslo_serialization/jsonutils.py", line 178, in dumps
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher return json.dumps(obj, default=default, **kwargs)
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher File "/usr/lib/python2.7/json/__init__.py", line 251, in dumps
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher sort_keys=sort_keys, **kw).encode(obj)
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher File "/usr/lib/python2.7/json/encoder.py", line 207, in encode
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher chunks = self.iterencode(o, _one_shot=True)
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher File "/usr/lib/python2.7/json/encoder.py", line 270, in iterencode
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher return _iterencode(o, 0)
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher UnicodeDecodeError: 'utf8' codec can't decode byte 0xf3 in position 8: invalid continuation byte
2019-04-05 09:32:45.651 5723 ERROR oslo_messaging.dispatcher

And the key of the problem is here:

When openstack get info for tgt targets it recieves broken output of command tgt-admin --show
In my case there is two lines with strange symbols, examples (highlited with a lot of !):

Target 60: iqn.2010-10.org.openstack:volume-*
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
        I_T nexus: 3096
            Initiator: iqn.***** alias: compute-*
            Connection: 0
      P/<F3> LUN information: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
        LUN: 0
            Type: controller
            SCSI ID: IET 003c0000
            SCSI SN: beaf600
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET 003c0001
            SCSI SN: beaf601
            Size: 214748 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: rdwr
            Backing store path: /dev/cinder-volumes/volume-id
            Backing store flags:
    Account information:
        9ng7BANwyTq5CLhUwUQY
    ACL information:
        ALL

The second one:

Target 80: iqn.2010-10.org.openstack:volume-id
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
        I_T nexus: 3095
            Initiator: iqn.1993-08.org.debian:01:b7a02622c68c alias: compute-*
            Connection: 0
      <E0><8E>"^A LUN information: (!!!!!!!!!!!!!!!!!!!!!!!!!!)
        LUN: 0
            Type: controller
            SCSI ID: IET 00500000
            SCSI SN: beaf800
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            SWP: No
            Thin-provisioning: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
All other volumes are fine.
Can't get any idea how it could happen and if this needs some workarounds.