An instance RBD snapshots don't work

Bug #1695061 reported by Denis Kostryukov
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Mirantis OpenStack
Fix Released
High
Max Yatsenko

Bug Description

When I try to create a snapshot from an instance it creates standard snapshot instead of RBD snapshot.
I get the following error in nova-compute.log:

2017-06-01 18:54:21.501 5940 INFO nova.compute.manager [req-7603f0a8-1178-466c-9b5e-903c7ff7bbcb ec0ffcafc20244029d090e7e98aa0461 9b05738c81444d5e8ba74669c058f53a - - -] [instance: a5eeeaf7-7d0b-4348-b84f-f009d57703b9] instance snapshotting
2017-06-01 18:54:21.746 5940 INFO nova.compute.manager [req-5bb61737-13af-4c66-af8f-7072b6611d71 - - - - -] [instance: a5eeeaf7-7d0b-4348-b84f-f009d57703b9] VM Paused (Lifecycle Event)
2017-06-01 18:54:22.287 5940 INFO nova.compute.manager [req-5bb61737-13af-4c66-af8f-7072b6611d71 - - - - -] [instance: a5eeeaf7-7d0b-4348-b84f-f009d57703b9] During sync_power_state the instance has a pending task (image_snapshot). Skip.
2017-06-01 18:54:23.287 5940 INFO nova.virt.libvirt.driver [req-7603f0a8-1178-466c-9b5e-903c7ff7bbcb ec0ffcafc20244029d090e7e98aa0461 9b05738c81444d5e8ba74669c058f53a - - -] [instance: a5eeeaf7-7d0b-
4348-b84f-f009d57703b9] Beginning cold snapshot process

2017-06-01 18:54:28.283 5940 WARNING nova.virt.libvirt.driver [req-7603f0a8-1178-466c-9b5e-903c7ff7bbcb ec0ffcafc20244029d090e7e98aa0461 9b05738c81444d5e8ba74669c058f53a - - -] Performing standard snapshot because direct snapshot failed: no write permission on storage pool images

2017-06-01 18:54:31.380 5940 INFO nova.compute.manager [req-5bb61737-13af-4c66-af8f-7072b6611d71 - - - - -] [instance: a5eeeaf7-7d0b-4348-b84f-f009d57703b9] VM Started (Lifecycle Event)
2017-06-01 18:54:32.080 5940 INFO nova.virt.libvirt.driver [req-7603f0a8-1178-466c-9b5e-903c7ff7bbcb ec0ffcafc20244029d090e7e98aa0461 9b05738c81444d5e8ba74669c058f53a - - -] [instance: a5eeeaf7-7d0b-4348-b84f-f009d57703b9] Snapshot extracted, beginning image upload
2017-06-01 18:54:32.157 5940 INFO nova.compute.manager [req-5bb61737-13af-4c66-af8f-7072b6611d71 - - - - -] [instance: a5eeeaf7-7d0b-4348-b84f-f009d57703b9] During sync_power_state the instance has a pending task (image_pending_upload). Skip.
2017-06-01 18:54:32.158 5940 INFO nova.compute.manager [req-5bb61737-13af-4c66-af8f-7072b6611d71 - - - - -] [instance: a5eeeaf7-7d0b-4348-b84f-f009d57703b9] VM Resumed (Lifecycle Event)
2017-06-01 18:54:32.340 5940 INFO nova.compute.manager [req-5bb61737-13af-4c66-af8f-7072b6611d71 - - - - -] [instance: a5eeeaf7-7d0b-4348-b84f-f009d57703b9] During sync_power_state the instance has a pending task (image_uploading). Skip.
2017-06-01 18:54:35.973 5940 INFO nova.virt.libvirt.driver [req-7603f0a8-1178-466c-9b5e-903c7ff7bbcb ec0ffcafc20244029d090e7e98aa0461 9b05738c81444d5e8ba74669c058f53a - - -] [instance: a5eeeaf7-7d0b-4348-b84f-f009d57703b9] Snapshot image upload complete

MOS 9.0 Mitaka

Possible related issues:
https://bugs.launchpad.net/openstack-manuals/+bug/1538244
https://disqus.com/home/discussion/sebastienhan/openstack_nova_snapshots_on_ceph_rbd_sebastien_han/#comment-3025636092

Changed in mos:
milestone: none → 9.x-updates
assignee: nobody → MOS Maintenance (mos-maintenance)
status: New → Confirmed
Changed in mos:
importance: Medium → High
Changed in mos:
milestone: 9.x-updates → 9.2-mu-3
Max Yatsenko (myatsenko)
Changed in mos:
assignee: MOS Maintenance (mos-maintenance) → Max Yatsenko (myatsenko)
Revision history for this message
Max Yatsenko (myatsenko) wrote :

The problem was reproduced only on 'clean' MOS9.0 environment, when an environment was deployed with all MU - the issue was not reproduced.

In upstream similar problem was solved by this patch:
https://review.openstack.org/#/c/407410/

This one was backported to fuel-infra:
https://review.fuel-infra.org/gitweb?p=openstack/fuel-library.git;a=commit;h=7cb1801e005472fe4dc33fd45bdfd639d352b5b4
https://review.fuel-infra.org/gitweb?p=openstack/fuel-library.git;a=commit;h=d891b4fd30677acb112e7e6b26e36e5c267028cf

Revision history for this message
Denis Kostryukov (dkostryukov) wrote :

After upgrade an existing environment to 9.2 the error still persists.
The error has gone after reset and redeploy the environment.

Revision history for this message
Max Yatsenko (myatsenko) wrote :

~dkostryukov
As for MOS9.2 with MUs this problem is not confirmed - can you please provide your steps to reproduce upgrade to 9.2? Its needed/helpfull for further issue investigation/solving.

Changed in mos:
status: Confirmed → Incomplete
Changed in mos:
assignee: Max Yatsenko (myatsenko) → Denis Kostryukov (dkostryukov)
Revision history for this message
Denis Kostryukov (dkostryukov) wrote :
Changed in mos:
assignee: Denis Kostryukov (dkostryukov) → Max Yatsenko (myatsenko)
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix proposed to openstack/mos-updates (9.0/mitaka)

Fix proposed to branch: 9.0/mitaka
Change author: Max Yatsenko <email address hidden>
Review: https://review.fuel-infra.org/36018

Changed in mos:
status: Incomplete → In Progress
Max Yatsenko (myatsenko)
Changed in mos:
status: In Progress → Fix Committed
Revision history for this message
Fuel Devops McRobotson (fuel-devops-robot) wrote : Fix merged to openstack/mos-updates (9.0/mitaka)

Reviewed: https://review.fuel-infra.org/36018
Submitter: Pkgs Jenkins <email address hidden>
Branch: 9.0/mitaka

Commit: 0dba9f2f3787b00782e7ea14904d4e38192b746d
Author: Max Yatsenko <email address hidden>
Date: Fri Jul 21 09:50:15 2017

Add write permission for ceph 'image' pool

It adds write permission for ceph 'image' pool
if it is missing.

Change-Id: Ifefbc5fedc3bba89de8ea103cb8e6f25cab71f4e
Closes-Bug: #1695061

Revision history for this message
Vladimir Jigulin (vjigulin) wrote :

Verified on 9.2 mu3-proposed

With proposed:
No warnings in logs
RBD snapshot instead of standard

Changed in mos:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.