swift3 doesn't recognize specific create bucket request with location

[Andrey Pavlov]

request body: <CreateBucketConstraint><LocationConstraint>RegionOne</LocationConstraint></CreateBucketConstraint>

response: 400
logs:
swift: Expecting element CreateBucketConfiguration, got CreateBucketConstraint, line 1 (txn: tx10fb5ecf89844099b81cc-0057ab16f1)

steps to reproduce:

1) download cirros image
wget http://download.cirros-cloud.net/0.3.4/cirros-0.3.4-x86_64-disk.img

2) download and unzip OFFICIAL tools
wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip
unzip ec2-ami-tools.zip
export TOOLS_DIR="ec2-ami-tools-1.5.7/bin"

3) source cloud credentials and download certificates
nova x509-get-root-cert
nova x509-create-cert

4) prepare bundle
mkdir images
$TOOLS_DIR/ec2-bundle-image --cert cert.pem --privatekey pk.pem --ec2cert cacert.pem --image cirros-0.3.4-x86_64-disk.img --user 424242424242 --destination images --arch x86_64

5) get s3 url
openstack endpoint list --service s3 --interface public --os-identity-api-version=3 -c URL -f value

6) list and create credentials if needed
openstack ec2 credentials list
openstack ec2 credentials create

6) upload bundle and get error
$TOOLS_DIR/ec2-upload-bundle --url http://192.168.137.21:8080 --region "RegionOne" --access-key a8f020880d4f4f2db370383bdcb8ddf1 --secret-key 37387e4e29c54588875610b67c0f6a02 --bucket "tmp.bucket" --manifest images/cirros-0.3.4-x86_64-disk.img.manifest.xml --acl "public-read" --sigv 2

->
ERROR: Error talking to S3: Server.MalformedXML(400): The XML you provided was not well-formed or did not validate against our published schema.

[Tim Burke (1-tim-z)]

This seems like a bug in ec2-ami-tools. CreateBucketConfiguration is what's documented for the API [1] as well as what other clients use [2] [3] [4].

And yet, sure enough, AWS accepts CreateBucketConstraint:

  > PUT / HTTP/1.1
  > Host: b349581c95cb0fa4.s3.amazonaws.com
  > Accept: application/json;q=1, text/*;q=.9, */*;q=.8
  > x-amz-date: Wed, 28 Dec 2016 16:12:43 -0800
  > Authorization: AWS AKIAINX36BBVRCISCYCA:lnKWPdY6Px8HmZ/Fl77AVkyXEpM=
  > Content-Length: 99
  >
  * upload completely sent off: 99 out of 99 bytes
  < HTTP/1.1 200 OK
  < x-amz-id-2: 5+5eNAdV8xJnbhPCFqZxSiSCsSnexQXyf7M8zcHTKXcGqEJbmrnY6eCGtiJF+x1h59c4BLxd+5Y=
  < x-amz-request-id: D5D78288796C428A
  < Date: Thu, 29 Dec 2016 00:12:45 GMT
  < Location: http://b349581c95cb0fa4.s3.amazonaws.com/
  < Content-Length: 0
  < Server: AmazonS3
  <

  > GET / HTTP/1.1
  > Host: b349581c95cb0fa4.s3.amazonaws.com
  > Accept: application/json;q=1, text/*;q=.9, */*;q=.8
  > x-amz-date: Wed, 28 Dec 2016 16:12:44 -0800
  > Authorization: AWS AKIAINX36BBVRCISCYCA:glfdAWag+2XC50DTE4Az8JwVeNs=
  > Content-Length: 99
  >
  * upload completely sent off: 99 out of 99 bytes
  < HTTP/1.1 307 Temporary Redirect
  < x-amz-bucket-region: us-west-1
  < x-amz-request-id: C04BDC83F73C85CA
  < x-amz-id-2: xJR/BGFKs6h+R0IJiK8pSxC9l6TBIXl345rfCz3k9eJc/EEubf9qijEn48gFY1WGKrJnXKaXRSA=
  < Location: https://b349581c95cb0fa4.s3-us-west-1.amazonaws.com/
  < Content-Type: application/xml
  < Transfer-Encoding: chunked
  < Date: Thu, 29 Dec 2016 00:12:44 GMT
  < Connection: close
  < Server: AmazonS3
  <

In fact, it even accepts junk data, like "<foo><LocationConstraint>us-west-1</LocationConstraint></foo>":

  > PUT / HTTP/1.1
  > Host: b349581c95cb0fa6.s3.amazonaws.com
  > Accept: application/json;q=1, text/*;q=.9, */*;q=.8
  > x-amz-date: Wed, 28 Dec 2016 16:22:25 -0800
  > Authorization: AWS AKIAINX36BBVRCISCYCA:jjyByg8+HCGGktvrgmrXcKEXzMg=
  > Content-Length: 61
  >
  * upload completely sent off: 61 out of 61 bytes
  < HTTP/1.1 200 OK
  < x-amz-id-2: V+K/bmU3we/NAckK4w6BMdh7VRdP8Mx0nEMLbLvHHzqcHyica8qPI20tGZ/qqb+5zQsVmxZcZ2A=
  < x-amz-request-id: B2AEBF2ECA183277
  < Date: Thu, 29 Dec 2016 00:22:27 GMT
  < Location: http://b349581c95cb0fa6.s3.amazonaws.com/
  < Content-Length: 0
  < Server: AmazonS3
  <

  > GET / HTTP/1.1
  > Host: b349581c95cb0fa6.s3.amazonaws.com
  > Accept: application/json;q=1, text/*;q=.9, */*;q=.8
  > x-amz-date: Wed, 28 Dec 2016 16:22:26 -0800
  > Authorization: AWS AKIAINX36BBVRCISCYCA:au7fUHeGythnpzCs81437xzamE8=
  >
  < HTTP/1.1 307 Temporary Redirect
  < x-amz-bucket-region: us-west-1
  < x-amz-request-id: 21B3CABC0401A6A0
  < x-amz-id-2: W5NjXbClam0N1p4S0S8M3XfWtL4nOkMV3tMsNOAgfrrbPTEiIhIMYW+NMHNQ0g9OpcmYFxYNmYg=
  < Location: https://b349581c95cb0fa6.s3-us-west-1.amazonaws.com/
  < Content-Type: application/xml
  < Transfer-Encoding: chunked
  < Date: Thu, 29 Dec 2016 00:22:26 GMT
  < Server: AmazonS3
  <

Going stranger than that, like "<foo><bar>us-west-1</bar></foo>", thankfully *is* rejected:

  > PUT / HTTP/1.1
  > Host: b349581c95cb0fa7.s3-us-west-1.amazonaws.com
  > Accept: application/json;q=1, text/*;q=.9, */*;q=.8
  > x-amz-date: Wed, 28 Dec 2016 16:24:46 -0800...

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/415559
Committed: https://git.openstack.org/cgit/openstack/swift3/commit/?id=0a2bfb8baeede93830893452a801778eabb4e2ae
Submitter: Jenkins
Branch: master

commit 0a2bfb8baeede93830893452a801778eabb4e2ae
Author: Tim Burke <email address hidden>
Date: Thu Dec 29 00:59:09 2016 +0000

    Relax bucket creation schema

    Apparently some AWS-supplied tools use undocumented root elements.

    Change-Id: I3d56f9560c2d96b08e6837d053fdff654c531062
    Closes-Bug: #1611749