IOError when downloading a directory

[Ana Krivokapić]

When trying to download a container which has a folder inside it, the client errors out with an IOError:

[akrivoka@localhost ~]$ swift -V 2 --os-username admin --os-tenant-name admin --os-password blablabla --os-auth-url http://192.168.100.136:5000/v2.0/ list picture
pf/
pf/rh pdf p
pf/rh_pdf_p.pdf
rh pic
[akrivoka@localhost ~]$ swift -V 2 --os-username admin --os-tenant-name admin --os-password blablabla --os-auth-url http://192.168.100.136:5000/v2.0/ download picture
rh pic [auth 0.442s, headers 0.517s, total 0.525s, 2.512 MB/s]
pf/rh pdf p [auth 0.412s, headers 0.461s, total 0.562s, 21.203 MB/s]
pf/rh_pdf_p.pdf [auth 0.435s, headers 0.510s, total 0.570s, 23.617 MB/s]
Traceback (most recent call last):
  File "/opt/stack/python-swiftclient/swiftclient/multithreading.py", line 70, in run
    result = self.func(item, *self.args, **self.kwargs)
  File "/opt/stack/python-swiftclient/bin/swift", line 367, in _download_object
    fp = open(path, 'wb')
IOError: [Errno 21] Is a directory: 'pf/'

[akrivoka@localhost ~]$

[Fabien Boucher (fabien-boucher)]

Hello Ana,

I've try to reproduce without any success. Below is the details of what I've done. Could you give us the detail to reach this
IOError ? The Swift client I use is 1.8.0.11.g04e0cb2.

fabien@saio:/tmp/picture$ find .
.
./rh pic
./pf
./pf/rh_pdf_p.pdf
./pf/rh pdf p
fabien@saio:/tmp/picture$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list picture
Container 'picture' not found
fabien@saio:/tmp/picture$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload picture .
pf/rh_pdf_p.pdf
rh pic
pf/rh pdf p
fabien@saio:/tmp/picture$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list picture
pf/rh pdf p
pf/rh_pdf_p.pdf
rh pic
fabien@saio:/tmp/picture$ mkdir ../_picture
fabien@saio:/tmp/picture$ cd !$
cd ../_picture
fabien@saio:/tmp/_picture$ swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download picture
rh pic [headers 0.021s, total 0.022s, 0.000 MB/s]
pf/rh_pdf_p.pdf [headers 0.024s, total 0.025s, 0.000 MB/s]
pf/rh pdf p [headers 0.027s, total 0.028s, 0.000 MB/s]
fabien@saio:/tmp/_picture$

Cheers,
Fabien

[Ana Krivokapić (akrivoka)]

The difference seems to be that I used Horizon to create a pseudo-folder and upload a file. The client correctly handles download of containers uploaded using the command line, but crashes when downloading containers uploaded using Horizon.

[JC (jcotton1123)]

I just encountered what appears to be the same bug. Like Ana, when I use the swift cmd-line util to upload content it does not create psuedo-folders. If I use another util, like Cloudfuse, it does create psuedo-folders and when the swift util encounters these psuedo-folders it blows up.

[Charles Hsu (charles0126)]

Hi Guys,

What's content-type of the object("/pf")?

 $swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat picture pf/

[Charles Hsu (charles0126)]

As I know, Swift CLI only handles the object has "Content-Type: text/directory". So if "/pf" object is a directory, please keep the "Content-Type" as text/directory as well.

[Charles Hsu (charles0126)]

Looks cloudfuse use "application/directory" for diretory.

 * Directory entries are created as empty files with the content-type
       "application/directory".

 [1] https://github.com/redbo/cloudfuse

[Tim Burke (1-tim-z)]

...and Horizon uses "application/pseudo-folder":

https://github.com/openstack/horizon/blob/9010d51/openstack_dashboard/api/swift.py#L74

How many variations are we willing to support?

[Charles Hsu (charles0126)]

@Tim,

I don't know; it should be more.
Do you think to get other content-types from ENV variable is a good idea?

[OpenStack Infra (hudson-openstack)]

Change abandoned by Charles Hsu (<email address hidden>) on branch: master
Review: https://review.openstack.org/258927
Reason: We have a similar patch in here, so drop this one.
https://review.openstack.org/#/c/254921/2

[Charles Hsu (charles0126)]

The patch is in here (https://review.openstack.org/#/c/254921/2)

[Charles Hsu (charles0126)]

New patch, https://review.openstack.org/#/c/260689/1

[Nileshseo206 Patwal (seo206packersmovers)]

Expert5th Packers and Movers India

Packing and Unpacking services in Chennai visit :
http://www.expert5th.in/packers-and-movers-chennai/
Packing and Unpacking services in Faridabad visit :-
http://www.expert5th.in/packers-and-movers-faridabad/
Packing and Unpacking services in Ghaziabad visit :-
http://www.expert5th.in/packers-and-movers-ghaziabad/
Packing and Unpacking services in Noida visit :
http://www.expert5th.in/packers-and-movers-noida/
Packing and Unpacking services in Thane visit :
http://www.expert5th.in/packers-and-movers-thane/
Packing and Unpacking services in Navi Mumbai visit :
http://www.expert5th.in/packers-and-movers-navimumbai/

[Nileshseo206 Patwal (seo206packersmovers)]

Expert5th Packers and Movers India

Packing and Unpacking services in Delhi visit :-
http://www.expert5th.in/packers-and-movers-delhi/
Packing and Unpacking services in Gurgaon visit :
http://www.expert5th.in/packers-and-movers-gurgaon/
Packing and Unpacking services in Mumbai visit :
http://www.expert5th.in/packers-and-movers-mumbai/

[Nileshseo206 Patwal (seo206packersmovers)]

Expert5th Packers and Movers India

Packing and Unpacking services in Pune visit :
http://www.expert5th.in/packers-and-movers-pune/
Packing and Unpacking services in Hyderabad visit :
http://www.expert5th.in/packers-and-movers-hyderabad/
Packing and Unpacking services in Bangalore visit :
http://www.expert5th.in/packers-and-movers-bangalore/

[Charles Hsu (charles0126)]

merged to upsteam https://review.openstack.org/#/c/254921/.

[Tim Burke (1-tim-z)]

I'm less and less convinced that Horizon actually creates objects with an 'application/pseudo-folder' content-type. Judging by the bit that does the creating it seems like Horizon doesn't send *any* content-type [1], in which case Swift would just guess 'application/octet-stream' [2]. I've never actually set up Horizon though (looks like it requires Nova, Neutron, and Glance? [3]), so I can't readily check.

I guess one more thing that we *could* do would be to set content-type to our expected directory marker instead of an empty string when no content-type was provided [4], provided that path[-1] == '/'.

[1] https://github.com/openstack/horizon/blob/9.1.0/openstack_dashboard/api/swift.py#L300
[2] https://github.com/openstack/swift/blob/2.9.0/swift/proxy/controllers/obj.py#L431-L432
[3] http://docs.openstack.org/developer/horizon/topics/install.html
[4] https://github.com/openstack/python-swiftclient/blob/3.0.0/swiftclient/client.py#L1243

[komal sharma (komalsharma)]

Packers and Movers Bangalore as a Services providing company can make all the difference to your ### home Relocation experience. Bangalore based Company which offers versatile solutions, Right team that easily reduce the stress associated with a ✔✔✔Household Shifting, ***Vehicle Transportation. we help things run smoothly and reduce breakages and offer you seamless, Affordable, Reliable Shifting Services, Compare Shifting Charges @ https://packers-and-movers-bangalore.in/