euca-upload-bundle fails when connecting to Eucalyptus and a path in bucket is specified

Bug #711534 reported by Marek Goldmann on 2011-02-01
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Eucalyptus
Invalid
Undecided
Unassigned
2.0
New
Medium
graziano obertelli
eucalyptus (Ubuntu)
Medium
Unassigned

Bug Description

# euca-upload-bundle -U http://URL:8773/services/Walrus -b goldmann-test/f14-basic/fedora/14/1.0/x86_64 -m build/appliances/x86_64/fedora/14/f14-basic/s3-plugin/ami/f14-basic.ec2.manifest.xml -a AK -s SAK
Checking bucket: goldmann-test/f14-basic/fedora/14/1.0/x86_64
Warning: failed to parse error message from AWS: <unknown>:1:0: syntax error
S3ResponseError: 400 Bad Request
Failure: 400 Bad Request
Could not determine operation name for /services/Walrus/goldmann-test/f14-basic/fedora/14/1.0/x86_64/

I'm using euca2ools-1.3.1-4.fc14.noarch RPM

Above command works perfectly with S3, it only happens when connecting to a Eucalyptus S3 service.

Related branches

affects: euca2ools → eucalyptus
Daniel Nurmi (nurmi) on 2011-02-21
Changed in eucalyptus:
assignee: nobody → Neil Soman (neilsoman)
Garrett Holmstrom (gholms) wrote :

I'm surprised it works in S3 at all. AWS's documentation [1] doesn't allow '/' characters in bucket names and further recommends naming buckets such that they are valid as part of DNS names.

[1] http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?BucketRestrictions.html

Neil Soman (neilsoman) wrote :

The command specified in the bug report doesn't work with AWS S3 unless the user has actually created the bucket somehow. I just tested this.

For instance, if you use the option "-b mybucket/abcd/efgh" S3 will correctly interpret this request (PUT to mybucket/abcd/efgh) as an empty put to the bucket "mybucket" with the keyname "abcd/efgh." Which is correct behavior.

The user must have already have a bucket, "mybucket." Otherwise the key "abcd/efgh" cannot be created because the bucket does not exist. Before the key can be created, the bucket needs to be created with a PUT to "mybucket"

"/"s are not allowed in the bucket name. The "/" is interpreted as part of the key. The bucket must exist for that key to be created.

So the bug report is partly right and partly wrong.

The real bug is that Walrus should be returning a 404 and not a 400 when someone tries to access "mybucket/abcd/efgh" if the bucket does not exist.

The command will not work on S3 or against Walrus if the bucket (the part of the string before the first "/") was never created, but the right error to send back is a 404 (no such bucket) and not a 400 (bad request).

This is fixed in revno 1257 in lp:eucalyptus.

Hope all this makes sense,
neil

Changed in eucalyptus:
status: New → Fix Committed
Scott Moser (smoser) on 2011-03-10
Changed in eucalyptus (Ubuntu):
importance: Undecided → Medium
status: New → Triaged
Marek Goldmann (mgoldman) wrote :

Neil,

This means that I need to create the required directory structure (f14-basic/fedora/14/1.0/x86_64) in my bucket (goldmann-test) first and after this upload the image?

--Marek

Neil Soman (neilsoman) wrote :

Marek, it means that the bucket ("goldmann-test") needs to be created first. The "directory" structure is part of the key.

Hope that helps.
neil

Marek Goldmann (mgoldman) wrote :

Neil,

I created the bucket, but I still get the same result:

$ ~/s3cmd-0.9.8.3$ ./s3cmd ls
2011-02-01 19:43 s3://goldmann-test

$ euca-upload-bundle -U http://URL:8773/services/Walrus -b goldmann-test/f14-basic/fedora/14/1.0/x86_64 -m build/appliances/x86_64/fedora/14/f14-basic/s3-plugin/ami/f14-basic.ec2.manifest.xml -a KEY -s SECRET_KEY
Checking bucket: goldmann-test/f14-basic/fedora/14/1.0/x86_64
Warning: failed to parse error message from AWS: <unknown>:1:0: syntax error
S3ResponseError: 400 Bad Request
Failure: 400 Bad Request
Could not determine operation name for /services/Walrus/goldmann-test/f14-basic/fedora/14/1.0/x86_64/

Using: euca2ools-1.3.1-6.fc14.noarch and partnercloud.

Can you confirm that my call shouldn't fail?

Neil Soman (neilsoman) wrote :

It shouldn't fail as of revno 1257 as noted above :)

Did you try that revno?

Marek Goldmann (mgoldman) wrote :

I'm not sure what's running on partner cloud :)

Neil Soman (neilsoman) wrote :

Definitely not that :)

This is fixed in the source but not released in package form yet.

This would be nice to have it in 2.0.4.

Changed in eucalyptus:
milestone: none → 2.0.4
assignee: Neil Soman (neilsoman) → graziano obertelli (graziano.obertelli)
Changed in eucalyptus:
status: Fix Committed → Invalid
assignee: graziano obertelli (graziano.obertelli) → nobody
milestone: 2.0.4 → none

after a chat on #eucalyptus-devel, this is not present (the mis-handling of /) in eucal3 (marking invalid for -devel). It is targeted for 2.0.4, but the IRC chat seems to imply that more work may be needed to understand the right behavior here.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers