OpenStack Compute (nova)

No rootwrap filter for chmod in libvirt/utils

Bug #1717533 reported by Evgeny Antyshev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
OpenStack Compute (nova)
Fix Released
Medium
Michael Still

Bug Description

After https://review.openstack.org/459166 was applied, Virtuozzo-specific code became broken,
which was noticed when we started running Tempest tests
for ephemeral disk.

n-cpu.service log:
Sep 15 10:15:09.633992 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [None req-ff184083-1ba2-44ec-a961-111adafb4cbe service nova] [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] Instance failed to spawn: ProcessExecutionError: Unexpected error while running command.
Sep 15 10:15:09.634505 localhost.localdomain nova-compute[67509]: Command: sudo nova-rootwrap /etc/nova/rootwrap.conf chmod -R a+r /opt/stack/data/nova/instances/c9d08a85-4a46-4b34-b919-8c2cb283ecfc/disk.eph0
Sep 15 10:15:09.634683 localhost.localdomain nova-compute[67509]: Exit code: 99
Sep 15 10:15:09.634852 localhost.localdomain nova-compute[67509]: Stdout: u''
Sep 15 10:15:09.635244 localhost.localdomain nova-compute[67509]: Stderr: u'/usr/bin/nova-rootwrap: Unauthorized command: chmod -R a+r /opt/stack/data/nova/instances/c9d08a85-4a46-4b34-b919-8c2cb283ecfc/disk.eph0 (no filter matched)\n'
Sep 15 10:15:09.635435 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] Traceback (most recent call last):
Sep 15 10:15:09.635601 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/compute/manager.py", line 2162, in _build_resources
Sep 15 10:15:09.635772 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] yield resources
Sep 15 10:15:09.636252 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/compute/manager.py", line 1977, in _build_and_run_instance
Sep 15 10:15:09.636523 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] block_device_info=block_device_info)
Sep 15 10:15:09.636965 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 2797, in spawn
Sep 15 10:15:09.637339 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] block_device_info=block_device_info)
Sep 15 10:15:09.637582 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3273, in _create_image
Sep 15 10:15:09.637833 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] specified_fs=specified_fs)
Sep 15 10:15:09.638079 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 242, in cache
Sep 15 10:15:09.638483 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] *args, **kwargs)
Sep 15 10:15:09.638733 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 1087, in create_image
Sep 15 10:15:09.638973 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] prepare_template(target=self.path, *args, **kwargs)
Sep 15 10:15:09.639245 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/usr/lib/python2.7/site-packages/oslo_concurrency/lockutils.py", line 274, in inner
Sep 15 10:15:09.639494 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] return f(*args, **kwargs)
Sep 15 10:15:09.639732 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/virt/libvirt/imagebackend.py", line 238, in fetch_func_sync
Sep 15 10:15:09.640069 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] fetch_func(target=target, *args, **kwargs)
Sep 15 10:15:09.640367 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/virt/libvirt/driver.py", line 3017, in _create_ephemeral
Sep 15 10:15:09.640615 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] specified_fs)
Sep 15 10:15:09.640852 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/virt/libvirt/utils.py", line 119, in create_ploop_image
Sep 15 10:15:09.641093 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] run_as_root=True, check_exit_code=True)
Sep 15 10:15:09.641367 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/utils.py", line 223, in execute
Sep 15 10:15:09.641616 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] return RootwrapProcessHelper().execute(*cmd, **kwargs)
Sep 15 10:15:09.641862 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/opt/stack/new/nova/nova/utils.py", line 106, in execute
Sep 15 10:15:09.642104 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] return processutils.execute(*cmd, **kwargs)
Sep 15 10:15:09.642382 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] File "/usr/lib/python2.7/site-packages/oslo_concurrency/processutils.py", line 419, in execute
Sep 15 10:15:09.642726 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] cmd=sanitized_cmd)
Sep 15 10:15:09.642965 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] ProcessExecutionError: Unexpected error while running command.
Sep 15 10:15:09.643238 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] Command: sudo nova-rootwrap /etc/nova/rootwrap.conf chmod -R a+r /opt/stack/data/nova/instances/c9d08a85-4a46-4b34-b919-8c2cb283ecfc/disk.eph0
Sep 15 10:15:09.643486 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] Exit code: 99
Sep 15 10:15:09.643724 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] Stdout: u''
Sep 15 10:15:09.643970 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc] Stderr: u'/usr/bin/nova-rootwrap: Unauthorized command: chmod -R a+r /opt/stack/data/nova/instances/c9d08a85-4a46-4b34-b919-8c2cb283ecfc/disk.eph0 (no filter matched)\n'
Sep 15 10:15:09.644248 localhost.localdomain nova-compute[67509]: ERROR nova.compute.manager [instance: c9d08a85-4a46-4b34-b919-8c2cb283ecfc]

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to nova (master)

Fix proposed to branch: master
Review: https://review.openstack.org/504429

Changed in nova:
assignee: nobody → Evgeny Antyshev (eantyshev)
status: New → In Progress
OpenStack Infra (hudson-openstack)
Changed in nova:
assignee: Evgeny Antyshev (eantyshev) → Michael Still (mikal)
melanie witt (melwitt)
Changed in nova:
importance: Undecided → Medium
tags: added: libvirt privsep
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Change abandoned on nova (master)

Change abandoned by Evgeny Antyshev (<email address hidden>) on branch: master
Review: https://review.openstack.org/504429
Reason: Let's abandon in favour of https://review.openstack.org/#/c/492325/
as it is more elaborated and solves the problem

OpenStack Infra (hudson-openstack)
Changed in nova:
assignee: Michael Still (mikal) → Sean Dague (sdague)
OpenStack Infra (hudson-openstack)
Changed in nova:
assignee: Sean Dague (sdague) → Michael Still (mikal)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to nova (master)

Reviewed: https://review.openstack.org/492325
Committed: https://git.openstack.org/cgit/openstack/nova/commit/?id=c1eb6f0e5078051ff03e4592e5aaff7cf04aa449
Submitter: Jenkins
Branch: master

commit c1eb6f0e5078051ff03e4592e5aaff7cf04aa449
Author: Michael Still <email address hidden>
Date: Wed Sep 27 06:30:14 2017 +1000

    Move ploop commands to privsep.

    The same pattern as the others, but with an added security concern.

    Co-Authored-By: Evgeny Antyshev <email address hidden>

    Closes-Bug: #1717533

    Change-Id: I1ac3a0ea4756ec68884866435c3da69171bbeb13
    blueprint: hurrah-for-privsep

Changed in nova:
status: In Progress → Fix Released
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix included in openstack/nova 17.0.0.0b1

This issue was fixed in the openstack/nova 17.0.0.0b1 development milestone.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.