The oslo.log JSONFormatter gets stuck when passed an RBDVolumeIOWrapper

Bug #1908607 reported by Sofia Enriquez on 2020-12-17
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
os-brick
Undecided
Sofia Enriquez
oslo.serialization
Undecided
Sofia Enriquez

Bug Description

Currently volume backup gets stuck in creating status when debug and use_json are True. Only using Ceph for backups.

How reproducible:
Always

Steps to Reproduce:
1. Enable Ceph for backups.

2. Enable debug and json format log:

/etc/cinder/cinder.conf
~~~
[DEFAULT]
...
debug = True
...
use_json = True
~~~

3. Create volume backup
$ cinder backup-create <voume id> --name <backup name>

Actual results:
The created backup gets stuck in creating status

Expected results:
The backup is created and become active

Changed in os-brick:
assignee: nobody → Sofia Enriquez (lsofia-enriquez)
tags: added: backup error volume
Eric Harney (eharney) wrote :

Some notes on what is failing here:

The oslo.log JSONFormatter gets stuck when passed an RBDVolumeIOWrapper from os-brick. This happens via os-brick's utils.trace() method which passes a connector containing {'path': RBDVolumeIOWrapper}.

The oslo.log JSONFormatter format() method calls oslo_serialization's jsonutils.to_primitive and passes in this RBDVolumeIOWrapper object.

The to_primitive method eventually calls RBDVolumeIOWrapper.read() which I suspect is not intended.

This can be replicated without the oslo.log layers by adding the following into os-brick's utils.trace method just before the '<== %(func)s' line:
     jsonutils.to_primitive(mask_result, fallback=repr)

Changed in os-brick:
status: New → Invalid
summary: - [Ceph] utils.py don't display well the json logs
+ The oslo.log JSONFormatter gets stuck when passed an RBDVolumeIOWrapper
Changed in oslo.serialization:
assignee: nobody → Sofia Enriquez (lsofia-enriquez)

This issue was fixed in the openstack/oslo.serialization 3.1.2 release.

This issue was fixed in the openstack/oslo.serialization 4.0.2 release.

This issue was fixed in the openstack/oslo.serialization 4.1.0 release.

This issue was fixed in the openstack/oslo.serialization 2.29.3 release.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers