/var/lib/nova/instances/_base has wrong permissions

Bug #973377 reported by David Kranz
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
High
Chuck Short
nova (Ubuntu)
Fix Released
Undecided
Unassigned

Bug Description

The nova-compute process running as "nova" needs to modify this directory but:

root@xg12:~# ls -l /var/lib/nova/instances/_base/
total 1805552
-rw-r--r-- 1 libvirt-qemu kvm 2147483648 Apr 2 17:07 1463a880811561c92d617f1b011763447e65f76f
-rw-r--r-- 1 libvirt-qemu kvm 10737418240 Apr 2 17:14 1463a880811561c92d617f1b011763447e65f76f_10
-rw-r--r-- 1 libvirt-qemu kvm 21474836480 Apr 2 17:14 ephemeral_0_20_None

You can see this error in the compute log:

2012-04-04 07:05:03 ERROR nova.manager [-] Error during ComputeManager._run_image_cache_manager_pass: [Errno 13] Permissio\
n denied: '/var/lib/nova/instances/_base/1463a880811561c92d617f1b011763447e65f76f'
2012-04-04 07:05:03 TRACE nova.manager Traceback (most recent call last):
2012-04-04 07:05:03 TRACE nova.manager File "/usr/lib/python2.7/dist-packages/nova/manager.py", line 155, in periodic_ta\
sks
2012-04-04 07:05:03 TRACE nova.manager task(self, context)
2012-04-04 07:05:03 TRACE nova.manager File "/usr/lib/python2.7/dist-packages/nova/compute/manager.py", line 2579, in _r\
un_image_cache_manager_pass
2012-04-04 07:05:03 TRACE nova.manager self.driver.manage_image_cache(context)
2012-04-04 07:05:03 TRACE nova.manager File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/connection.py", line 228\
9, in manage_image_cache
2012-04-04 07:05:03 TRACE nova.manager self.image_cache_manager.verify_base_images(context)
2012-04-04 07:05:03 TRACE nova.manager File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagecache.py", line 392\
, in verify_base_images
2012-04-04 07:05:03 TRACE nova.manager self._handle_base_image(img, base_file)
2012-04-04 07:05:03 TRACE nova.manager File "/usr/lib/python2.7/dist-packages/nova/virt/libvirt/imagecache.py", line 358\
, in _handle_base_image
2012-04-04 07:05:03 TRACE nova.manager os.utime(base_file, None)
2012-04-04 07:05:03 TRACE nova.manager OSError: [Errno 13] Permission denied: '/var/lib/nova/instances/_base/1463a88081156\
1c92d617f1b011763447e65f76f'
2012-04-04 07:05:03 TRACE nova.manager

Revision history for this message
David Kranz (david-kranz) wrote :

This is nova-compute version 2012.1~rc2-0ubuntu1

Revision history for this message
Chuck Short (zulcss) wrote :

Please attach your nova.conf.
Thanks
chuck

Changed in nova (Ubuntu):
status: New → Incomplete
Revision history for this message
David Kranz (david-kranz) wrote :

--flagfile=/etc/nova/nova-compute.conf
--allow_admin_api
--dhcpbridge_flagfile=/etc/nova/nova.conf
--dhcpbridge=/usr/bin/nova-dhcpbridge
--sql_connection=mysql://nova:notnova@172.18.0.146/nova
--s3_host=172.18.0.146
--rabbit_host=172.18.0.146
--glance_api_servers=172.18.0.146:9292
--logdir=/var/log/nova
--state_path=/var/lib/nova
--lock_path=/var/lock/nova
--verbose
--ec2_url=http://172.18.0.146:8773/services/Cloud
--dmz_cidr=10.0.2.0/8
--fixed_range=10.0.2.0/24
--network_size=256
--image_service=nova.image.glance.GlanceImageService
--my_ip=172.18.0.176
--bridge_interface=eth1
--flat_network_bridge=br100
--flat_interface=eth1
--instance_name_template=instance-%08x
--network_manager=nova.network.manager.FlatDHCPManager
--force_dhcp_release
--public_interface=eth0
--multi_host
--send_arp_for_ha
--osapi_host=172.18.0.146
--quota_instances=1000000
--quota_ram=1000000
--quota_cores=1000000
--iscsi_ip_prefix=172.18.0
--connection_type=libvirt
--compute_scheduler_driver=nova.scheduler.chance.ChanceScheduler
--osapi_compute_extension=nova.api.openstack.compute.contrib.standard_extensions
--root_helper=sudo nova-rootwrap
--api_rate_limit=False
--auth_strategy=keystone
--allow_resize_to_same_host
--enabled_apis=metadata
--novncproxy_base_url=http://172.18.0.146:6080/vnc_auto.html
--vncserver_proxyclient_address=172.18.0.176
--vncserver_listen=172.18.0.176
--iscsi_helper=tgtadm

Revision history for this message
Chuck Short (zulcss) wrote :

I am able to reproduce this, thanks for the bug report.

Changed in nova (Ubuntu):
status: Incomplete → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/6231

Changed in nova:
assignee: nobody → Chuck Short (zulcss)
status: New → In Progress
Chuck Short (zulcss)
tags: added: essex-rc-potential
Changed in nova:
importance: Undecided → High
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (milestone-proposed)

Fix proposed to branch: milestone-proposed
Review: https://review.openstack.org/6235

Revision history for this message
Vish Ishaya (vishvananda) wrote :

Verified this bug exists in devstack as well:

2012-04-04 17:10:34 ERROR nova.manager [-] Error during ComputeManager._run_image_cache_manager_pass: [Errno 13] Permission denied: '/home/vagrant/_base/87ad83b59bbed6a3d3cea9afdf3e1fd36c694ac2'
2012-04-04 17:10:34 TRACE nova.manager Traceback (most recent call last):
2012-04-04 17:10:34 TRACE nova.manager File "/opt/stack/nova/nova/manager.py", line 155, in periodic_tasks
2012-04-04 17:10:34 TRACE nova.manager task(self, context)
2012-04-04 17:10:34 TRACE nova.manager File "/opt/stack/nova/nova/compute/manager.py", line 2554, in _run_image_cache_manager_pass
2012-04-04 17:10:34 TRACE nova.manager self.driver.manage_image_cache(context)
2012-04-04 17:10:34 TRACE nova.manager File "/opt/stack/nova/nova/virt/libvirt/connection.py", line 2435, in manage_image_cache
2012-04-04 17:10:34 TRACE nova.manager self.image_cache_manager.verify_base_images(context)
2012-04-04 17:10:34 TRACE nova.manager File "/opt/stack/nova/nova/virt/libvirt/imagecache.py", line 392, in verify_base_images
2012-04-04 17:10:34 TRACE nova.manager self._handle_base_image(img, base_file)
2012-04-04 17:10:34 TRACE nova.manager File "/opt/stack/nova/nova/virt/libvirt/imagecache.py", line 358, in _handle_base_image
2012-04-04 17:10:34 TRACE nova.manager os.utime(base_file, None)
2012-04-04 17:10:34 TRACE nova.manager OSError: [Errno 13] Permission denied: '/home/vagrant/_base/87ad83b59bbed6a3d3cea9afdf3e1fd36c694ac2'

Changed in nova:
assignee: Chuck Short (zulcss) → Vish Ishaya (vishvananda)
Revision history for this message
Vish Ishaya (vishvananda) wrote :

fyi, regression was introduced in 2c5112e5938e3d567504748f923fb198fad3117e

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/6231
Committed: http://github.com/openstack/nova/commit/f698eba45752fddddd914892ed2bf1d5ef7e484e
Submitter: Jenkins
Branch: master

commit f698eba45752fddddd914892ed2bf1d5ef7e484e
Author: Chuck Short <email address hidden>
Date: Wed Apr 4 12:36:42 2012 -0400

    Fix traceback in image cache manager.

    When nova is running as a different user, the image
    cache manager can cause tracebacks because it is not
    able to access the file. This fixes bug: 973377

    Change-Id: I7902b024e7d564fbda97ce8532c1de84ab1a4206
    Signed-off-by: Chuck Short <email address hidden>

Changed in nova:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (milestone-proposed)

Reviewed: https://review.openstack.org/6235
Committed: http://github.com/openstack/nova/commit/2c7365c3b5d2a560d0709a7ad9dbf491000cdd9e
Submitter: Jenkins
Branch: milestone-proposed

commit 2c7365c3b5d2a560d0709a7ad9dbf491000cdd9e
Author: Chuck Short <email address hidden>
Date: Wed Apr 4 12:36:42 2012 -0400

    Fix traceback in image cache manager.

    When nova is running as a different user, the image
    cache manager can cause tracebacks because it is not
    able to access the file. This fixes bug: 973377

    Change-Id: I7902b024e7d564fbda97ce8532c1de84ab1a4206
    Signed-off-by: Chuck Short <email address hidden>

Changed in nova:
status: Fix Committed → Fix Released
Changed in nova:
milestone: none → essex-rc4
Changed in nova:
assignee: Vish Ishaya (vishvananda) → Chuck Short (zulcss)
Thierry Carrez (ttx)
Changed in nova:
milestone: essex-rc4 → 2012.1
Chuck Short (zulcss)
Changed in nova (Ubuntu):
status: In Progress → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.