Value out of range when pulling from a server with content-encoding: gzip

[Simon Poirier]

Doing a snapcraft -d pull with a source served from a server doing gzip encoding, I get this error

  File "/usr/bin/snapcraft", line 31, in <module>
    snapcraft.main.main()
  File "/usr/lib/python3/dist-packages/snapcraft/main.py", line 210, in main
    return run(args, project_options)
  File "/usr/lib/python3/dist-packages/snapcraft/main.py", line 245, in run
    lifecycle_command, project_options, args['<part>'])
  File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle.py", line 98, in execute
    _Executor(config, project_options).run(step, part_names)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle.py", line 127, in run
    self._run_step(step, part, part_names, dirty, recursed)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/lifecycle.py", line 165, in _run_step
    getattr(part, step)()
  File "/usr/lib/python3/dist-packages/snapcraft/internal/pluginhandler.py", line 275, in pull
    self.code.pull()
  File "/usr/lib/python3/dist-packages/snapcraft/_baseplugin.py", line 126, in pull
    sources.get(self.sourcedir, self.build_basedir, self.options)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/sources.py", line 365, in get
    handler.pull()
  File "/usr/lib/python3/dist-packages/snapcraft/internal/sources.py", line 101, in pull
    self.download()
  File "/usr/lib/python3/dist-packages/snapcraft/internal/sources.py", line 113, in download
    download_requests_stream(request, file_path)
  File "/usr/lib/python3/dist-packages/snapcraft/internal/indicators.py", line 51, in download_requests_stream
    progress_bar.update(total_read)
  File "/usr/lib/python3/dist-packages/progressbar/__init__.py", line 271, in update
    raise ValueError('Value out of range')
ValueError: Value out of range

Peeking a bit, I see that snapcraft.internal.indicators uses the len(buf) to calculate the download position. However, with the encoding this will fail as the content-length header represents the transfer size and and request.iter_content() returns the decoded payload which most likely be larger.

[Leo Arias (elopio)]

Thanks for your report Simon. Can you please share your snapcraft.yaml so we can reproduce this issue and confirm when it's fixed?

[Sergio Schvezov (sergiusens)]

https://github.com/snapcore/snapcraft/pull/826

[Aboobacker MK (aboobackervyd)]

@Simon : Can you please provide the source you used for making this snap ?

[Simon Poirier (simpoir)]

Here is the snapcraft file I was fiddling with. The resulting snap doesn't work (yet) but it fails at the "pull" phase, downloading the part source and this is what this issue is about.

[Sergio Schvezov (sergiusens)]

https://github.com/snapcore/snapcraft/pull/896