Hi, I patch this https://review.openstack.org/319554 get this: [root@controller ~]# cinder backup-create --incremental --force 64ad570c-9556-43f1-91eb-16f506efe4ea +-----------+--------------------------------------+ | Property | Value | +-----------+--------------------------------------+ | id | 3df9976c-0f01-4548-9e45-ed9ad672775b | | name | None | | volume_id | 64ad570c-9556-43f1-91eb-16f506efe4ea | +-----------+--------------------------------------+ 2016-10-21 10:33:40.945 2661171 DEBUG oslo_messaging._drivers.amqpdriver [req-c32eb48c-3109-4500-ae17-228bc69d533e 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] received message msg_id: None reply to None __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:201 2016-10-21 10:33:41.033 2661171 INFO cinder.backup.manager [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Create backup started, backup: 3df9976c-0f01-4548-9e45-ed9ad672775b volume: 64ad570c-9556-43f1-91eb-16f506efe4ea. 2016-10-21 10:33:41.049 2661171 DEBUG oslo_concurrency.processutils [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf cat /etc/iscsi/initiatorname.iscsi execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:344 2016-10-21 10:33:41.187 2661171 DEBUG oslo_concurrency.processutils [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf cat /etc/iscsi/initiatorname.iscsi" returned: 0 in 0.138s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:374 2016-10-21 10:33:41.189 2661171 DEBUG oslo_concurrency.processutils [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf systool -c fc_host -v execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:344 2016-10-21 10:33:41.329 2661171 DEBUG oslo_concurrency.processutils [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf systool -c fc_host -v" returned: 1 in 0.140s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:374 2016-10-21 10:33:41.331 2661171 DEBUG oslo_concurrency.processutils [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] u'sudo cinder-rootwrap /etc/cinder/rootwrap.conf systool -c fc_host -v' failed. Not Retrying. execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:422 2016-10-21 10:33:41.333 2661171 DEBUG oslo_concurrency.processutils [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf systool -c fc_host -v execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:344 2016-10-21 10:33:41.469 2661171 DEBUG oslo_concurrency.processutils [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf systool -c fc_host -v" returned: 1 in 0.137s execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:374 2016-10-21 10:33:41.471 2661171 DEBUG oslo_concurrency.processutils [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] u'sudo cinder-rootwrap /etc/cinder/rootwrap.conf systool -c fc_host -v' failed. Not Retrying. execute /usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py:422 2016-10-21 10:33:41.528 2661171 DEBUG oslo_messaging._drivers.amqpdriver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] CALL msg_id: b23b0a8a96fe438b9f2698fb96c7f8b0 exchange 'openstack' topic 'cinder-volume' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:454 2016-10-21 10:33:41.535 2661171 WARNING py.warnings [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] /usr/lib/python2.7/site-packages/pkg_resources/__init__.py:187: RuntimeWarning: You have iterated over the result of pkg_resources.parse_version. This is a legacy behavior which is inconsistent with the new version class introduced in setuptools 8.0. In most cases, conversion to a tuple is unnecessary. For comparison of versions, sort the Version instances directly. If you have another use case requiring the tuple, please file a bug with the setuptools project describing that need. stacklevel=1, 2016-10-21 10:33:41.624 2661171 DEBUG oslo_messaging._drivers.amqpdriver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] received reply msg_id: b23b0a8a96fe438b9f2698fb96c7f8b0 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:302 2016-10-21 10:33:41.629 2661171 DEBUG oslo_messaging._drivers.amqpdriver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] CALL msg_id: 91f313755a274dffb25089dea0fd501a exchange 'openstack' topic 'cinder-volume' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:454 2016-10-21 10:33:42.321 2661171 DEBUG oslo_messaging._drivers.amqpdriver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] received reply msg_id: 91f313755a274dffb25089dea0fd501a __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:302 2016-10-21 10:33:42.323 2661171 DEBUG os_brick.initiator.connector [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Factory for rbd on x86_64 factory /usr/lib/python2.7/site-packages/os_brick/initiator/connector.py:180 2016-10-21 10:33:42.469 2661171 DEBUG oslo_db.sqlalchemy.engines [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION _check_effective_sql_mode /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/engines.py:256 2016-10-21 10:33:42.502 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Starting backup of volume='64ad570c-9556-43f1-91eb-16f506efe4ea'. backup /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:852 2016-10-21 10:33:42.503 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Volume file is NOT RBD: will do full backup. backup /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:869 2016-10-21 10:33:42.524 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Creating backup base image='volume-64ad570c-9556-43f1-91eb-16f506efe4ea.backup.3df9976c-0f01-4548-9e45-ed9ad672775b' for volume 64ad570c-9556-43f1-91eb-16f506efe4ea. _full_backup /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:701 2016-10-21 10:33:42.580 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Copying data from volume 64ad570c-9556-43f1-91eb-16f506efe4ea. _full_backup /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:710 2016-10-21 10:33:42.603 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Transferring data between 'volume-64ad570c-9556-43f1-91eb-16f506efe4ea' and 'volume-64ad570c-9556-43f1-91eb-16f506efe4ea.backup.3df9976c-0f01-4548-9e45-ed9ad672775b' _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:302 2016-10-21 10:33:42.604 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] 8 chunks of 134217728 bytes to be transferred _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:306 2016-10-21 10:33:43.759 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Transferred chunk 1 of 8 (113627K/s) _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:328 2016-10-21 10:33:44.921 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Transferred chunk 2 of 8 (113255K/s) _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:328 2016-10-21 10:33:46.202 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Transferred chunk 3 of 8 (102398K/s) _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:328 2016-10-21 10:33:47.530 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Transferred chunk 4 of 8 (99616K/s) _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:328 2016-10-21 10:33:48.753 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Transferred chunk 5 of 8 (108966K/s) _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:328 2016-10-21 10:33:50.101 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Transferred chunk 6 of 8 (97330K/s) _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:328 2016-10-21 10:33:51.137 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Transferred chunk 7 of 8 (126745K/s) _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:328 2016-10-21 10:33:52.538 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Transferred chunk 8 of 8 (93650K/s) _transfer_data /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:328 2016-10-21 10:33:52.576 2661171 DEBUG cinder.backup.driver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Getting metadata type 'volume-base-metadata' _save_vol_base_meta /usr/lib/python2.7/site-packages/cinder/backup/driver.py:80 2016-10-21 10:33:52.648 2661171 DEBUG cinder.backup.driver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Completed fetching metadata type 'volume-base-metadata' _save_vol_base_meta /usr/lib/python2.7/site-packages/cinder/backup/driver.py:96 2016-10-21 10:33:52.649 2661171 DEBUG cinder.backup.driver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Getting metadata type 'volume-metadata' _save_vol_meta /usr/lib/python2.7/site-packages/cinder/backup/driver.py:107 2016-10-21 10:33:52.791 2661171 DEBUG cinder.backup.driver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] No metadata type 'volume-metadata' available _save_vol_meta /usr/lib/python2.7/site-packages/cinder/backup/driver.py:121 2016-10-21 10:33:52.792 2661171 DEBUG cinder.backup.driver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Getting metadata type 'volume-glance-metadata' _save_vol_glance_meta /usr/lib/python2.7/site-packages/cinder/backup/driver.py:130 2016-10-21 10:33:52.934 2661171 DEBUG cinder.backup.driver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] No metadata type 'volume-glance-metadata' available _save_vol_glance_meta /usr/lib/python2.7/site-packages/cinder/backup/driver.py:145 2016-10-21 10:33:52.935 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Backing up metadata for volume 64ad570c-9556-43f1-91eb-16f506efe4ea. _backup_metadata /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:831 2016-10-21 10:33:52.968 2661171 DEBUG cinder.backup.drivers.ceph [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Backup '3df9976c-0f01-4548-9e45-ed9ad672775b' of volume 64ad570c-9556-43f1-91eb-16f506efe4ea finished. backup /usr/lib/python2.7/site-packages/cinder/backup/drivers/ceph.py:888 2016-10-21 10:33:52.971 2661171 DEBUG oslo_messaging._drivers.amqpdriver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] CALL msg_id: e75912851380409aa51b96f40d15f7c9 exchange 'openstack' topic 'cinder-volume' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:454 2016-10-21 10:33:53.190 2661171 DEBUG oslo_messaging._drivers.amqpdriver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] received reply msg_id: e75912851380409aa51b96f40d15f7c9 __call__ /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:302 2016-10-21 10:33:53.192 2661171 DEBUG oslo_messaging._drivers.amqpdriver [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] CAST unique_id: c0a5a4439cbb48c1a0fc8e7b032cc07b exchange 'openstack' topic 'cinder-volume' _send /usr/lib/python2.7/site-packages/oslo_messaging/_drivers/amqpdriver.py:454 2016-10-21 10:33:53.310 2661171 INFO cinder.backup.manager [req-8437d817-9d29-4954-8836-8c609cdf1e17 597e55d40136460680b3039b7087eb43 98189fbdeb8d4db998a4d2eb3afa4d68 - - -] Create backup finished. backup: 3df9976c-0f01-4548-9e45-ed9ad672775b. why it say: Volume file is NOT RBD: will do full backup.