[RBD -> SolidFire] Cache mode was not updated after volume retype

[mikhail]

Steps to reproduce
1. Set nova.conf
[libvirt]
disk_cachemodes = network=writeback

2. Create RBD volume and attach it to an instance as additional volume.

    <disk type='network' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
      <auth username='cinder'>
        <secret type='ceph' uuid='<secret uuid>'/>
      </auth>
      <source protocol='rbd' name='volumes/volume-<volume id>'>
        <host name='192.168.1.37' port='6789'/>
        <host name='192.168.1.38' port='6789'/>
        <host name='192.168.1.55' port='6789'/>
      </source>
      <mirror type='block' job='copy'>
        <format type='raw'/>
        <source dev='/dev/sdd'/>
        <backingStore/>
      </mirror>
      <target dev='vdg' bus='virtio'/>
      <serial>5a71c185-1a17-48da-9d70-c73cdbf73681</serial>
      <alias name='virtio-disk6'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
    </disk>

3. Retype volume in in-use state to SolidFire or NetApp ONTAP backend. Retype mode is On-demand.

4. Volume was moved to another backend, but the cache mode was not changed

    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='writeback' discard='unmap'/>
      <source dev='/dev/sdd'/>
      <backingStore/>
      <target dev='vdg' bus='virtio'/>
      <serial>5a71c185-1a17-48da-9d70-c73cdbf73681</serial>
      <alias name='virtio-disk6'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x0b' function='0x0'/>
    </disk>

In the end of those steps we are not able to live-migrate this instance due to the following error:

Live Migration failure: Unsafe migration: Migration may lead to data corruption if disks use cache != none or cache != directsync: libvirt.libvirtError: Unsafe migration: Migration may lead to data corruption if disks use cache != none or cache != directsync

[Sofia Enriquez (lsofia-enriquez)]

Hi mikhail (kraynovmv), hope this message finds you well.
Please share the cinder version you are referring to.
Is the error message reported by c-vol or by nova side?
Thanks,
Sofia

[mikhail (kraynovmv)]

Hello Sofia,
I'm using cinder from Train version - 15.4.0
The error comes from nova side.