rest-bench always fails on initial execution

[James Page]

prompt> rest-bench --access-key=xxx --secret=yyy --api-host=localhost write
host=localhost
ERROR: failed to create bucket: ConnectionFailed
failed initializing benchmark

Except from apache2 access.log;
127.0.0.1 - - [05/Sep/2012:12:40:38 +0100] "PUT /rest-bench-bucket/ HTTP/1.1" 200 138 "-" "Mozilla/4.0 (Compatible; rest-bench; libs3 2.0; Linux x86_64)"

Excerpt from radosgw log file:
2012-09-05 12:40:38.448014 7f5068fc9700 10 cache get: name=.users+B587ZPR3Z9TCQZY6D7M4 : hit
2012-09-05 12:40:38.448069 7f5068fc9700 10 get_canon_resource(): dest=/rest-bench-bucket/
2012-09-05 12:40:38.448075 7f5068fc9700 10 auth_hdr:
PUT

x-amz-date:Wed, 05 Sep 2012 11:40:38 GMT
/rest-bench-bucket/
2012-09-05 12:40:38.448187 7f5068fc9700 15 b64=skwbg2pHJMbceUantJa1OVut8lk=
2012-09-05 12:40:38.448193 7f5068fc9700 15 auth_sign=skwbg2pHJMbceUantJa1OVut8lk=
2012-09-05 12:40:38.448196 7f5068fc9700 15 compare=0
2012-09-05 12:40:38.448201 7f5068fc9700 2 req 1:0.001795:s3:PUT /rest-bench-bucket/:create_bucket:reading permissions
2012-09-05 12:40:38.448207 7f5068fc9700 2 req 1:0.001801:s3:PUT /rest-bench-bucket/:create_bucket:verifying op permissions
2012-09-05 12:40:38.448872 7f5068fc9700 2 req 1:0.002466:s3:PUT /rest-bench-bucket/:create_bucket:verifying op params
2012-09-05 12:40:38.448888 7f5068fc9700 10 --> Status: 100

2012-09-05 12:40:38.448927 7f5068fc9700 2 req 1:0.002521:s3:PUT /rest-bench-bucket/:create_bucket:executing
2012-09-05 12:40:38.448966 7f5068fc9700 20 get_obj_state: rctx=0x7f50a8002160 obj=.rgw:rest-bench-bucket state=0x7f50a8007d38 s->prefetch_data=0
2012-09-05 12:40:38.448975 7f5068fc9700 10 cache get: name=.rgw+rest-bench-bucket : miss
2012-09-05 12:40:38.449498 7f5068fc9700 10 cache put: name=.rgw+rest-bench-bucket
2012-09-05 12:40:38.449505 7f5068fc9700 10 adding .rgw+rest-bench-bucket to cache LRU end
2012-09-05 12:40:38.449538 7f5068fc9700 20 get_obj_state: rctx=0x7f50a8008fa0 obj=.rgw:.pools.avail state=0x7f50a8008598 s->prefetch_data=0
2012-09-05 12:40:38.449546 7f5068fc9700 10 cache get: name=.rgw+.pools.avail : miss
2012-09-05 12:40:38.450015 7f5068fc9700 10 cache put: name=.rgw+.pools.avail
2012-09-05 12:40:38.450020 7f5068fc9700 10 adding .rgw+.pools.avail to cache LRU end
2012-09-05 12:40:38.778331 7f5068fc9700 15 omap_set bucket=.rgw oid=.pools.avail key=.rgw.buckets
2012-09-05 12:40:39.223706 7f5068fc9700 10 cache put: name=.rgw+.pools.avail
2012-09-05 12:40:39.223718 7f5068fc9700 10 moving .rgw+.pools.avail to cache LRU end
2012-09-05 12:40:39.223744 7f5068fc9700 10 distributing notification oid=notify bl.length()=174
2012-09-05 12:40:39.224468 7f50c8887700 10 RGWWatcher::notify() opcode=1 ver=1 bl.length()=174
2012-09-05 12:40:39.224489 7f50c8887700 10 cache put: name=.rgw+.pools.avail
2012-09-05 12:40:39.224494 7f50c8887700 10 moving .rgw+.pools.avail to cache LRU end
2012-09-05 12:40:40.454025 7f5068fc9700 10 cache put: name=.rgw+rest-bench-bucket
2012-09-05 12:40:40.454034 7f5068fc9700 10 moving .rgw+rest-bench-bucket to cache LRU end
2012-09-05 12:40:40.454038 7f5068fc9700 10 updating xattr: name=user.rgw.acl bl.length()=143
2012-09-05 12:40:40.454067 7f5068fc9700 10 distributing notification oid=notify bl.length()=406
2012-09-05 12:40:40.454835 7f50c8887700 10 RGWWatcher::notify() opcode=1 ver=1 bl.length()=406
2012-09-05 12:40:40.454855 7f50c8887700 10 cache put: name=.rgw+rest-bench-bucket
2012-09-05 12:40:40.454859 7f50c8887700 10 moving .rgw+rest-bench-bucket to cache LRU end
2012-09-05 12:40:40.454863 7f50c8887700 10 updating xattr: name=user.rgw.acl bl.length()=143
2012-09-05 12:40:40.455346 7f5068fc9700 20 store_bucket_info: bucket=rest-bench-bucket(@.rgw.buckets[4100.1]) owner ubuntu
2012-09-05 12:40:40.455357 7f5068fc9700 20 rgw_create_bucket returned ret=0 bucket=rest-bench-bucket(@.rgw.buckets[4100.1])
2012-09-05 12:40:40.455372 7f5068fc9700 15 omap_set bucket=.users.uid oid=ubuntu.buckets key=rest-bench-bucket
2012-09-05 12:40:40.537346 7f5068fc9700 10 --> Status: 200

2012-09-05 12:40:40.537357 7f5068fc9700 10 --> Content-type: application/xml^M

2012-09-05 12:40:40.891583 7f5068fc9700 2 req 1:2.445176:s3:PUT /rest-bench-bucket/:create_bucket:http status=200
2012-09-05 12:40:40.891736 7f5068fc9700 1 ====== req done req=0x7f50d634fe20 http_status=200 ======

rest-bench-bucket also shows up in a 's3 list' output as well.

Subsequent runs of rest-bench work just fine - its just the initial one that fails.

ProblemType: Bug
DistroRelease: Ubuntu 12.10
Package: rest-bench 0.48.1-0ubuntu1
ProcVersionSignature: Ubuntu 3.5.0-12.12-generic 3.5.2
Uname: Linux 3.5.0-12-generic x86_64
ApportVersion: 2.5.1-0ubuntu3
Architecture: amd64
Date: Wed Sep 5 12:41:28 2012
SourcePackage: ceph
UpgradeStatus: No upgrade log present (probably fresh install)

[James Page (james-page)]

This is actually just a timeout on the initial hit on the radow gateway - it takes a few seconds to initialize the rados pools.

Marking 'Low' and 'Triaged'; increasing the timeout in rest-bench is probably the correct way to resolve this issue.

[James Page (james-page)]

rest bench went away a while back - marking 'Won't Fix'