Cinder backup export broken

[Lucian Petrut]

One relatively recent patch[1] introduced a regression, breaking the volume backup export functionality, as shown by the related tempest test[2][3].

Apparently, the newly introduced "parent" db field isn't serialized properly. Most probably it will have to be added to the "OPTIONAL_FIELDS" list.

[1] I516b7c82b05b26e81195f7f106d43a9e0804082d
[2] tempest.api.volume.admin.test_volumes_backup.VolumesBackupsAdminTest.test_volume_backup_export_import
[3] http://paste.openstack.org/raw/789372/

[Sofia Enriquez (lsofia-enriquez)]

Hi @Lucian,

I've been testing different combinations of things and I'm not able to reproduce the error: 'oslo_versionedobjects.exception.ObjectActionError: Object action obj_load_attr failed because: attribute parent not lazy-loadable'

You can see my output here: http://paste.openstack.org/raw/789482/

[Gorka Eguileor (gorka)]

This is the traceback from the paste:

   File "C:\Program Files\Cloudbase Solutions\OpenStack\Cinder\Python\lib\site-packages\cinder\backup\manager.py", line 840, in export_record
     backup_url = backup.encode_record(driver_info=driver_info)
   File "C:\Program Files\Cloudbase Solutions\OpenStack\Cinder\Python\lib\site-packages\cinder\objects\backup.py", line 215, in encode_record
     for name, field in self.fields.items()}
   File "C:\Program Files\Cloudbase Solutions\OpenStack\Cinder\Python\lib\site-packages\cinder\objects\backup.py", line 215, in <dictcomp>
     for name, field in self.fields.items()}
   File "C:\Program Files\Cloudbase Solutions\OpenStack\Cinder\Python\lib\site-packages\oslo_versionedobjects\base.py", line 67, in getter
     self.obj_load_attr(name)
   File "C:\Program Files\Cloudbase Solutions\OpenStack\Cinder\Python\lib\site-packages\cinder\objects\backup.py", line 155, in obj_load_attr
     reason=_('attribute %s not lazy-loadable') % attrname)
oslo_versionedobjects.exception.ObjectActionError: Object action obj_load_attr failed because: attribute parent not lazy-loadable

Looking at the traceback, and the Backup OVO code it seems like we have 2 problems:
- We don't support lazy loading the parent
- We try to serialize the parent

Solution:
- Add 'parent' to OPTIONAL_FIELDS
      OPTIONAL_FIELDS = ('metadata', 'parent')

- Add support to lazy load the parent in `obj_load_attr`
    def obj_load_attr(self, attrname):
        if attrname not in self.OPTIONAL_FIELDS:
            raise exception.ObjectActionError(
                action='obj_load_attr',
                reason=_('attribute %s not lazy-loadable') % attrname)
        if not self._context:
            raise exception.OrphanedObjectError(method='obj_load_attr',
                                                objtype=self.obj_name())
        if attrname == 'parent':
            if self.parent_id:
                self.parent = self.get_by_id(self._context, self.parent_id)
            else:
                self.parent = None
        self.obj_reset_changes(fields=[attrname])

- Exclude the 'parent' in the dict comprehension in `encode_record`
        record = {name: field.to_primitive(self, name, getattr(self, name))
                  for name, field in self.fields.items()
                  if name != 'parent'}
  Here we could code this to check the field is not an `ObjectField` instead of checking the name.

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.opendev.org/708908

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/708908
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=a98969380ac2172766995cf02b889e3c863511ba
Submitter: Zuul
Branch: master

commit a98969380ac2172766995cf02b889e3c863511ba
Author: Sofia Enriquez <email address hidden>
Date: Thu Feb 20 16:25:50 2020 +0000

    Cinder backup export broken

    Incremental patch I516b7c82b05b26e81195f7f106d43a9e0804082d
    introduced a regression, breaking the volume backup export
    functionality.

    Apparently, the newly introduced "parent" db field isn't
    serialized properly.

    This is the traceback from the launchpad bug:
    oslo_versionedobjects.exception.ObjectActionError:
    Object action obj_load_attr failed because: attribute
    parent not lazy-loadable

    Looking at the traceback, and the Backup OVO code it seems
    like we have 2 problems:

    - We don't support lazy loading the parent
    - We try to serialize the parent

    Co-Authored-By: Gorka Eguileor <email address hidden>
    Change-Id: I017602353e96cf9f0922074f94276002b17d1359
    Closes-Bug: #1862635

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/train
Review: https://review.opendev.org/709654

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/stein
Review: https://review.opendev.org/709833

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/rocky
Review: https://review.opendev.org/709856

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/709654
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=6dfbec7f7403bbbe498c5eaba16b5b68bd7d698c
Submitter: Zuul
Branch: stable/train

commit 6dfbec7f7403bbbe498c5eaba16b5b68bd7d698c
Author: Sofia Enriquez <email address hidden>
Date: Thu Feb 20 16:25:50 2020 +0000

    Cinder backup export broken

    Incremental patch I516b7c82b05b26e81195f7f106d43a9e0804082d
    introduced a regression, breaking the volume backup export
    functionality.

    Apparently, the newly introduced "parent" db field isn't
    serialized properly.

    This is the traceback from the launchpad bug:
    oslo_versionedobjects.exception.ObjectActionError:
    Object action obj_load_attr failed because: attribute
    parent not lazy-loadable

    Looking at the traceback, and the Backup OVO code it seems
    like we have 2 problems:

    - We don't support lazy loading the parent
    - We try to serialize the parent

    Co-Authored-By: Gorka Eguileor <email address hidden>
    Change-Id: I017602353e96cf9f0922074f94276002b17d1359
    Closes-Bug: #1862635
    (cherry picked from commit a98969380ac2172766995cf02b889e3c863511ba)
    modified:
    cinder/tests/unit/objects/test_backup.py - pyflake issue #202

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/709833
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=d906d811d738223a5f934e9930b51af518507cc1
Submitter: Zuul
Branch: stable/stein

commit d906d811d738223a5f934e9930b51af518507cc1
Author: Sofia Enriquez <email address hidden>
Date: Thu Feb 20 16:25:50 2020 +0000

    Cinder backup export broken

    Incremental patch I516b7c82b05b26e81195f7f106d43a9e0804082d
    introduced a regression, breaking the volume backup export
    functionality.

    Apparently, the newly introduced "parent" db field isn't
    serialized properly.

    This is the traceback from the launchpad bug:
    oslo_versionedobjects.exception.ObjectActionError:
    Object action obj_load_attr failed because: attribute
    parent not lazy-loadable

    Looking at the traceback, and the Backup OVO code it seems
    like we have 2 problems:

    - We don't support lazy loading the parent
    - We try to serialize the parent

    Co-Authored-By: Gorka Eguileor <email address hidden>
    Change-Id: I017602353e96cf9f0922074f94276002b17d1359
    Closes-Bug: #1862635
    (cherry picked from commit a98969380ac2172766995cf02b889e3c863511ba)
    modified:
    cinder/tests/unit/objects/test_backup.py - pyflake issue #202
    (cherry picked from commit 6dfbec7f7403bbbe498c5eaba16b5b68bd7d698c)

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.opendev.org/709856
Committed: https://git.openstack.org/cgit/openstack/cinder/commit/?id=bcea359d54fcaa5a013204f7981f952e4329cf93
Submitter: Zuul
Branch: stable/rocky

commit bcea359d54fcaa5a013204f7981f952e4329cf93
Author: Sofia Enriquez <email address hidden>
Date: Thu Feb 20 16:25:50 2020 +0000

    Cinder backup export broken

    Incremental patch I516b7c82b05b26e81195f7f106d43a9e0804082d
    introduced a regression, breaking the volume backup export
    functionality.

    Apparently, the newly introduced "parent" db field isn't
    serialized properly.

    This is the traceback from the launchpad bug:
    oslo_versionedobjects.exception.ObjectActionError:
    Object action obj_load_attr failed because: attribute
    parent not lazy-loadable

    Looking at the traceback, and the Backup OVO code it seems
    like we have 2 problems:

    - We don't support lazy loading the parent
    - We try to serialize the parent

    Co-Authored-By: Gorka Eguileor <email address hidden>
    Change-Id: I017602353e96cf9f0922074f94276002b17d1359
    Closes-Bug: #1862635
    (cherry picked from commit a98969380ac2172766995cf02b889e3c863511ba)
    modified:
    cinder/tests/unit/objects/test_backup.py - pyflake issue #202
    (cherry picked from commit 6dfbec7f7403bbbe498c5eaba16b5b68bd7d698c)
    (cherry picked from commit 196b6786db82de5afcaed2790395d869a7c2c183)

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 13.0.9 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 15.1.0 release.

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/cinder 14.0.4 release.