Comment 9 for bug 1670055

I found the root cause is that my glance storage backend is file and glance services are running in 3 controllers.

[glance_store]
default_store = file
filesystem_store_datadir = /var/lib/glance/images/

when I upload the image to one of 3 controllers successfully, it met failure when retrieving the image next time from another one controller, so It's necessary to use shared storage such as ceph when running glance service on multi-nodes.

workaround:
1.copy the image to anthother 2 controllers

[root@kode2 ~]# ls /var/lib/docker/volumes/glance/_data/images
2c02d662-0717-4c9c-bf56-00a447040f0a
[root@kode2 ~]# cd /var/lib/docker/volumes/glance/_data/images;ls -lah
total 13M
drwxr-x---. 1 42415 42415 72 Mar 5 08:41 .
drwxr-xr-x. 1 42415 42415 12 Mar 4 00:00 ..
-rw-r-----. 1 42415 42415 13M Mar 5 08:41 2c02d662-0717-4c9c-bf56-00a447040f0a

[root@kode1 ~]# ls /var/lib/docker/volumes/glance/_data/images
[root@kode1 ~]#

[root@kode0 ~]# ls /var/lib/docker/volumes/glance/_data/images
[root@kode0 ~]#

[root@kode2 images]# scp 2c02d662-0717-4c9c-bf56-00a447040f0a kode0:/var/lib/docker/volumes/glance/_data/images/

[root@kode2 images]# scp 2c02d662-0717-4c9c-bf56-00a447040f0a kode1:/var/lib/docker/volumes/glance/_data/images/

2.modify the ownership of image.

[root@kode1 images]# chown 42415:42415 2c02d662-0717-4c9c-bf56-00a447040f0a
[root@kode0 images]# chown 42415:42415 2c02d662-0717-4c9c-bf56-00a447040f0a

3. boot a vm
[root@kode0 ~]# openstack server create --image cirros --flavor m1.tiny --key-name mykey --nic net-id=c53cbfe2-f210-4f0e-b62c-cdab995a3f63 demo1

[root@kode0 ~]# openstack server list
+--------------------------------------+-------+--------+-------------------+------------+
| ID | Name | Status | Networks | Image Name |
+--------------------------------------+-------+--------+-------------------+------------+
| d1544927-291c-4a34-9634-dcc23880d1b7 | demo1 | ACTIVE | demo-net=10.0.0.8 | cirros |
+--------------------------------------+-------+--------+-------------------+------------+