Cloudfiles backend does not handle container ACLs

Bug #1715650 reported by Matthew S on 2017-09-07
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Duplicity
Medium
Unassigned

Bug Description

duplicity --version - duplicity 0.7.14
python2 - Python 2.7.6
Ubuntu 14.04.5
Using a Rackspace Cloudfiles container (with ACLs enabled)
Accessed via the Pyrax backend (also tested with Cloudfiles)

With Duplicity, both the old Cloudfiles and newer Pyrax backend (both connect to Rackspace Cloudfiles-the-service) do not work with container ACLs(1). The backends try to create a container, which with a normal privileged user either creates a container, or returns an existing container of the same name. However, to take advantage of Rackspace container ACLs means using an unprivileged user, which only has read/write access to a single container, and no other privileges. Consequently a Duplicity backup results in a 403 Forbidden permissions error, even when the user being used is able to read/write just fine.

1) https://developer.rackspace.com/docs/cloud-files/v1/use-cases/additional-container-services-information/#container-access-control-lists

Traceback of Pyrax error:

Traceback (most recent call last):
  File "/usr/bin/duplicity", line 1532, in <module>
    with_tempdir(main)
  File "/usr/bin/duplicity", line 1526, in with_tempdir
    fn()
  File "/usr/bin/duplicity", line 1364, in main
    action = commandline.ProcessCommandLine(sys.argv[1:])
  File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", line 1116, in ProcessCommandLine
    backup, local_pathname = set_backend(args[0], args[1])
  File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py", line 1005, in set_backend
    globals.backend = backend.get_backend(bend)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 223, in get_backend
    obj = get_backend_object(url_string)
  File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line 209, in get_backend_object
    return factory(pu)
  File "/usr/lib/python2.7/dist-packages/duplicity/backends/_cf_pyrax.py", line 72, in __init__
    self.container = pyrax.cloudfiles.create_container(container)
  File "/usr/local/lib/python2.7/dist-packages/pyrax/client.py", line 103, in create
    return self._manager.create(*args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pyrax/object_storage.py", line 833, in create
    resp, resp_body = self.api.method_put(uri, headers=headers)
  File "/usr/local/lib/python2.7/dist-packages/pyrax/client.py", line 260, in method_put
    return self._api_request(uri, "PUT", **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pyrax/client.py", line 231, in _api_request
    resp, body = self._time_request(safe_uri, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pyrax/client.py", line 193, in _time_request
    resp, body = self.request(uri, method, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pyrax/client.py", line 184, in request
    resp, body = pyrax.http.request(method, uri, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pyrax/http.py", line 76, in request
    raise exc.from_response(resp, body)
Forbidden: <html><h1>Forbidden</h1><p>Access was denied to this resource.</p></html> (HTTP 403)

Matthew S (kineomatts) wrote :

I have written a patch for the Pyrax backend to fix this bug, which uses get_container() initially, and only uses create_container() if necessary. It handles both completely unprivileged users, and read-only privileged users.
I can provide a diff in a separate comment if necessary?

Download full text (4.0 KiB)

Please do. Thanks!

On Thu, Sep 7, 2017 at 9:15 AM, Matthew S <email address hidden>
wrote:

> I have written a patch for the Pyrax backend to fix this bug, which uses
> get_container() initially, and only uses create_container() if necessary.
> It handles both completely unprivileged users, and read-only privileged
> users.
> I can provide a diff in a separate comment if necessary?
>
> --
> You received this bug notification because you are subscribed to
> Duplicity.
> https://bugs.launchpad.net/bugs/1715650
>
> Title:
> Cloudfiles backend does not handle container ACLs
>
> Status in Duplicity:
> New
>
> Bug description:
> duplicity --version - duplicity 0.7.14
> python2 - Python 2.7.6
> Ubuntu 14.04.5
> Using a Rackspace Cloudfiles container (with ACLs enabled)
> Accessed via the Pyrax backend (also tested with Cloudfiles)
>
> With Duplicity, both the old Cloudfiles and newer Pyrax backend (both
> connect to Rackspace Cloudfiles-the-service) do not work with
> container ACLs(1). The backends try to create a container, which with
> a normal privileged user either creates a container, or returns an
> existing container of the same name. However, to take advantage of
> Rackspace container ACLs means using an unprivileged user, which only
> has read/write access to a single container, and no other privileges.
> Consequently a Duplicity backup results in a 403 Forbidden permissions
> error, even when the user being used is able to read/write just fine.
>
> 1) https://developer.rackspace.com/docs/cloud-files/v1/use-cases
> /additional-container-services-information/#container-access-control-
> lists
>
> Traceback of Pyrax error:
>
> Traceback (most recent call last):
> File "/usr/bin/duplicity", line 1532, in <module>
> with_tempdir(main)
> File "/usr/bin/duplicity", line 1526, in with_tempdir
> fn()
> File "/usr/bin/duplicity", line 1364, in main
> action = commandline.ProcessCommandLine(sys.argv[1:])
> File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py",
> line 1116, in ProcessCommandLine
> backup, local_pathname = set_backend(args[0], args[1])
> File "/usr/lib/python2.7/dist-packages/duplicity/commandline.py",
> line 1005, in set_backend
> globals.backend = backend.get_backend(bend)
> File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
> 223, in get_backend
> obj = get_backend_object(url_string)
> File "/usr/lib/python2.7/dist-packages/duplicity/backend.py", line
> 209, in get_backend_object
> return factory(pu)
> File "/usr/lib/python2.7/dist-packages/duplicity/backends/_cf_pyrax.py",
> line 72, in __init__
> self.container = pyrax.cloudfiles.create_container(container)
> File "/usr/local/lib/python2.7/dist-packages/pyrax/client.py", line
> 103, in create
> return self._manager.create(*args, **kwargs)
> File "/usr/local/lib/python2.7/dist-packages/pyrax/object_storage.py",
> line 833, in create
> resp, resp_body = self.api.method_put(uri, headers=headers)
> File "/usr/local/lib/python2.7/dist-packages/pyrax/client.py", line
> 260, in method_put
> return self...

Read more...

Matthew S (kineomatts) wrote :

The following patch has been tested in our work environment, and the code is provided “as is” without any guarantee or warranty of any kind, either expressed or implied. It is licensed GPLv2, to comply with the license of this project (http://bazaar.launchpad.net/~duplicity-team/duplicity/0.7-series/view/head:/COPYING).

/usr/lib/python2.7/dist-packages/duplicity/backends/_cf_pyrax.py

--- _cf_pyrax.py.orig 2017-09-06 16:35:26.528312165 +0100
+++ _cf_pyrax.py 2017-09-07 11:28:22.729342589 +0100
@@ -70,7 +70,25 @@

         self.client_exc = pyrax.exceptions.ClientException
         self.nso_exc = pyrax.exceptions.NoSuchObject
- self.container = pyrax.cloudfiles.create_container(container)
+
+ #query rackspace for the specified container name
+ try:
+ self.container = pyrax.cloudfiles.get_container(container)
+ except pyrax.exceptions.Forbidden as e:
+ log.FatalError("%s : %s \n" % (e.__class__.__name__, util.uexc(e))+
+ "Container may exist, but access was denied.\n"+
+ "If this container exists, please check its X-Container-Read/Write headers.\n"+
+ "Otherwise, please check your credentials and permissions.",
+ log.ErrorCode.backend_permission_denied)
+ except pyrax.exceptions.NoSuchContainer as e:
+ try:
+ self.container = pyrax.cloudfiles.create_container(container)
+ except pyrax.exceptions.Forbidden as e:
+ log.FatalError("%s : %s \n" % (e.__class__.__name__, util.uexc(e))+
+ "Container does not exist, but creation was denied.\n"+
+ "You may be using a read-only user that can view but not create containers.\n"+
+ "Please check your credentials and permissions.",
+ log.ErrorCode.backend_permission_denied)

     def _error_code(self, operation, e):
         if isinstance(e, self.nso_exc):

Matthew S (kineomatts) wrote :

Please note, I have observed that this patch will not display its warnings if working credentials are used for the first backup, and then changed to non-working credentials for a second backup.
Duplicity only displays "Attempt x failed. Forbidden: <html><h1>Forbidden</h1><p>Access was denied to this resource.</p></html> (HTTP 403)" at that point, with no further context. Unfortunately I do not have the time to debug this further.

Matthew S (kineomatts) wrote :

It appears my patch post has been malformed when posted as a comment, as the whitespace has been taken out. Do you have any advice on a better place to submit this?

Matthew S (kineomatts) wrote :

Apologies, I have found the relevant button. :-) Patch is attached.

Changed in duplicity:
importance: Undecided → Medium
milestone: none → 0.7.15
status: New → Fix Committed
Changed in duplicity:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers