The oslo.log JSONFormatter gets stuck when passed an RBDVolumeIOWrapper

[Sofia Enriquez]

Currently volume backup gets stuck in creating status when debug and use_json are True. Only using Ceph for backups.

How reproducible:
Always

Steps to Reproduce:
1. Enable Ceph for backups.

2. Enable debug and json format log:

/etc/cinder/cinder.conf
~~~
[DEFAULT]
...
debug = True
...
use_json = True
~~~

3. Create volume backup
$ cinder backup-create <voume id> --name <backup name>

Actual results:
The created backup gets stuck in creating status

Expected results:
The backup is created and become active

[Eric Harney (eharney)]

Some notes on what is failing here:

The oslo.log JSONFormatter gets stuck when passed an RBDVolumeIOWrapper from os-brick. This happens via os-brick's utils.trace() method which passes a connector containing {'path': RBDVolumeIOWrapper}.

The oslo.log JSONFormatter format() method calls oslo_serialization's jsonutils.to_primitive and passes in this RBDVolumeIOWrapper object.

The to_primitive method eventually calls RBDVolumeIOWrapper.read() which I suspect is not intended.

This can be replicated without the oslo.log layers by adding the following into os-brick's utils.trace method just before the '<== %(func)s' line:
     jsonutils.to_primitive(mask_result, fallback=repr)

[Sofia Enriquez (lsofia-enriquez)]

Master/Wallaby patch merged https://review.opendev.org/c/openstack/oslo.serialization/+/769808/

Backports:
- Victoria https://review.opendev.org/c/openstack/oslo.serialization/+/771475
- Ussuri https://review.opendev.org/c/openstack/oslo.serialization/+/771476

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.serialization 3.1.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.serialization 4.0.2 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.serialization 4.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/oslo.serialization 2.29.3 release.