duplicate entries in s3 bucket listing

Bug #1241699 reported by Dieter P
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
goamz
New
Undecided
Unassigned

Bug Description

I wrote a simple program that just writes all keys in a bucket to a file.
the golang-goamz output starts with:

.com/451/45158_146.png
.com/451/45158_146.png
.com/451/45160_1261.png
.com/451/45160_1261.png
.com/451/45162_1261.png
.com/451/45162_1261.png
.com/451/45164_1261.jpg
.com/451/45164_1261.jpg
.com/451/45166_576.jpg
.com/451/45166_576.jpg

as you can see, pretty much all duplicates.

a similar script in python using the boto library to get a list:
.com/451/45157_146.png
.com/451/45158_146.png
.com/451/45159_1261.png
.com/451/45160_1261.png
.com/451/45161_1261.png
.com/451/45162_1261.png
.com/451/45163_1261.png
.com/451/45164_1261.jpg
.com/451/45165_1261.jpg
.com/451/45166_576.jpg

no duplicates here, this is also in line with the s3 web dashboard.

my code is basically just (edited for brevity):

func main() {
    auth := aws.Auth{access_key, secret_key}
    s3 := s3.New(auth, aws.USEast)
    bucket := s3.Bucket(my_bucket)
    get_aws := GetFilesAws(bucket)
    for in_aws := range get_aws {
        fmt.Println(in_aws.Key)
    }
}

func GetFilesAws(bucket *s3.Bucket) (in_aws chan *s3.Key) {
    in_aws = make(chan *s3.Key)
    go func() {
        marker := ""
        for {
            listresp, err := bucket.List("", "", marker, 1000)
            if err != nil {
                panic(err)
            }
            if len(listresp.Contents) == 0 {
                close(in_aws)
            }
            for _, key := range listresp.Contents {
                marker = key.Key
                in_aws <- &key
            }
        }
    }()
    return in_aws
}

maybe my code is wrong but I believe i implemented correctly according to the spec @ http://godoc.org/launchpad.net/goamz/s3#Bucket.List

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.