Make store downloads more robust

[Sergio Schvezov]

We constantly get reset by the peer which slows down our development and makes for an unhappy user as well:

======================================================================

ERROR: test_classic_confinement (test_stage.StageTestCase)

test_stage.StageTestCase.test_classic_confinement

----------------------------------------------------------------------

testtools.testresult.real._StringException: output: {{{

Starting snapcraft devel from /home/travis/build/snapcore/snapcraft/snapcraft.

Setting up part 'hello' with plugin 'make' and properties {'snap': [], 'prime': [], 'plugin': 'make', 'stage': []}.

Getting details for core

Starting new HTTPS connection (1): search.apps.ubuntu.com

"GET /api/v1/snaps/details/core?fields=status%2Canon_download_url%2Cdownload_url%2Cdownload_sha3_384%2Cdownload_sha512%2Csnap_id%2Crevision%2Crelease&channel=stable HTTP/1.1" 200 992

Downloading core

Starting new HTTPS connection (1): public.apps.ubuntu.com

"GET /anon/download-snap/99T7MUlRhtI3U0QFgl5mXXESAiSwt776_1337.snap HTTP/1.1" 302 0

Starting new HTTPS connection (1): 068ed04f23.site.internapcdn.net

"GET /download-snap/99T7MUlRhtI3U0QFgl5mXXESAiSwt776_1337.snap?t=2017-03-11T12:01:29Z&h=4ddf6b981dd18e366f0c8b1e2584565134b52679 HTTP/1.1" 200 78807040

Downloading 'core.snap' 0%

Downloading 'core.snap' 1%

Downloading 'core.snap' 2%

Downloading 'core.snap' 3%

Downloading 'core.snap' 4%

Downloading 'core.snap' 5%

Downloading 'core.snap' 6%

Downloading 'core.snap' 7%

Downloading 'core.snap' 8%

Downloading 'core.snap' 9%

Downloading 'core.snap' 10%

Downloading 'core.snap' 11%

Downloading 'core.snap' 12%

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/response.py", line 226, in _error_catcher

    yield

  File "/usr/local/lib/python3.5/dist-packages/requests/packages/urllib3/response.py", line 301, in read

    data = self._fp.read(amt)

  File "/usr/lib/python3.5/http/client.py", line 448, in read

    n = self.readinto(b)

  File "/usr/lib/python3.5/http/client.py", line 488, in readinto

    n = self.fp.readinto(b)

  File "/usr/lib/python3.5/socket.py", line 575, in readinto

    return self._sock.recv_into(b)

  File "/usr/lib/python3.5/ssl.py", line 929, in recv_into

    return self.read(nbytes, buffer)

  File "/usr/lib/python3.5/ssl.py", line 791, in read

    return self._sslobj.read(len, buffer)

  File "/usr/lib/python3.5/ssl.py", line 575, in read

    v = self._sslobj.read(len, buffer)

ConnectionResetError: [Errno 104] Connection reset by peer

[Seth Arnold (seth-arnold)]

Are you trying to debug the CDN? or the download tool?

I just downloaded this snap 110 times via the URLs above and got identical results each time regardless of which CDN IP I hit.

Thanks

[Sergio Schvezov (sergiusens)]

On Fri, 10 Mar 2017 23:50:42 +0000, Seth Arnold wrote:
> Are you trying to debug the CDN? or the download tool?
>
> I just downloaded this snap 110 times via the URLs above and got
> identical results each time regardless of which CDN IP I hit.

In our CI we have constant connection resets sent to us (it is also sometimes seen when installing snaps). We are just adding a retry mechanism provided by requests.

--
Sent using Dekko from my Ubuntu device

[Kyle Fazzari (kyrofa)]

I'm still seeing these on master. Not as often, but still there.

[Sergio Schvezov (sergiusens)]

https://github.com/snapcore/snapcraft/pull/1212