I can't find anything outside the go HTTP library that is taking such a long time. My guess is that there is a tiny buffer being used instead of something sensible, but I haven't found it yet.
It isn't our hashing code slowing us down. On an 800M file it takes a couple of seconds while the http client Do function is taking two minutes.
I am testing on LXD so there shouldn't be any limitation on network speed.
Reading the stream just next to the http post does so ~instantly.
It is also CPU limited on the client.
I can't find anything outside the go HTTP library that is taking such a long time. My guess is that there is a tiny buffer being used instead of something sensible, but I haven't found it yet.
It isn't our hashing code slowing us down. On an 800M file it takes a couple of seconds while the http client Do function is taking two minutes.
I am testing on LXD so there shouldn't be any limitation on network speed.
Reading the stream just next to the http post does so ~instantly.