CI: BlockingIOError about 50% of the time

Bug #1717921 reported by Cris Dywan
12
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Snapcraft
Confirmed
Undecided
Unassigned

Bug Description

On Travis, I've seen what seems to be flaky test failures during the download of Python modules before, never locally. I'm not sure that the test name is meaningful here but including the most recent one for completeness.

 integration_tests.plugins.test_python_plugin.PythonPluginTestCase.test_build_doesnt_get_bad_install_directory_lp1586546

Downloading 'PyYAML-3.11.tar.gz' 23% Traceback (most recent call last):
  File "/usr/lib/python3.5/runpy.py", line 184, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/lib/python3.5/unittest/__main__.py", line 18, in <module>
    main(module=None)
  File "/usr/lib/python3.5/unittest/main.py", line 94, in __init__
    self.runTests()
  File "/usr/lib/python3.5/unittest/main.py", line 255, in runTests
    self.result = testRunner.run(self.test)
  File "/usr/lib/python3.5/unittest/runner.py", line 183, in run
    result.printErrors()
  File "/usr/lib/python3.5/unittest/runner.py", line 109, in printErrors
    self.printErrorList('ERROR', self.errors)
  File "/usr/lib/python3.5/unittest/runner.py", line 117, in printErrorList
    self.stream.writeln("%s" % err)
  File "/usr/lib/python3.5/unittest/runner.py", line 25, in writeln
    self.write(arg)
BlockingIOError: [Errno 11] write could not complete without blocking

Revision history for this message
Kyle Fazzari (kyrofa) wrote :
Download full text (20.2 KiB)

I'm seeing this all over the place, in different tests, and unrelated PRs. Here's a full one:

======================================================================

ERROR: test_build (test_rust_plugin.RustPluginConfinementTestCase)

test_rust_plugin.RustPluginConfinementTestCase.test_build (classic)

----------------------------------------------------------------------

testtools.testresult.real._StringException: output: {{{

Starting snapcraft devel from /root/snapcraft/snapcraft/cli.

"grade" property not specified: defaulting to "stable"

Setting up part 'rust-hello' with plugin 'rust' and properties {'plugin': 'rust', 'source': '.', 'stage': [], 'prime': []}.

Marking 'gcc' (and its dependencies) to be fetched

Marking 'git' (and its dependencies) to be fetched

Marking 'file' (and its dependencies) to be fetched

Marking 'curl' (and its dependencies) to be fetched

Getting details for core

Starting new HTTPS connection (1): api.snapcraft.io

"GET /api/v1/snaps/details/core?channel=stable&fields=status%2Canon_download_url%2Cdownload_url%2Cdownload_sha3_384%2Cdownload_sha512%2Csnap_id%2Crevision%2Crelease HTTP/1.1" 200 589

Downloading core

Starting new HTTPS connection (1): api.snapcraft.io

"HEAD /api/v1/snaps/download/99T7MUlRhtI3U0QFgl5mXXESAiSwt776_3017.snap HTTP/1.1" 302 0

Starting new HTTPS connection (1): 068ed04f23.site.internapcdn.net

"GET /download-snap/99T7MUlRhtI3U0QFgl5mXXESAiSwt776_3017.snap?t=2017-10-19T21:00:00Z&h=6b7cc022a250434d9f0754c24f915b5970ea5803 HTTP/1.1" 200 87080960

Successfully downloaded core at /tmp/tmpcb4i_dqh/core.snap

b'Parallel unsquashfs: Using 2 processors\n1 inodes (1 blocks) to write\n\n\r[===================================================================|] 1/1 100%\n\ncreated 1 files\ncreated 2 directories\ncreated 0 symlinks\ncreated 0 devices\ncreated 0 fifos\n'

Setting up '/tmp/tmpdl8bye5v/.cache/snapcraft/projects/snapcraft-core/snap_hashes/amd64/09407c3b0197d5a3f9324b83a50b266840550ad9f15ff7fafe3be282d927901def755c6a6f64916acb41ebbcdcd9cdd5' in '/snap/core/current'

Parallel unsquashfs: Using 2 processors

12267 inodes (13603 blocks) to write

[===========================================================-] 13603/13603 100%

created 9635 files

created 1464 directories

created 2543 symlinks

created 79 devices

created 0 fifos

Preparing to pull rust-hello

Pulling rust-hello

Starting new HTTPS connection (1): static.rust-lang.org

"GET /rustup.sh HTTP/1.1" 200 61986

Downloading 'core.snap' 0%

Downloading 'core.snap' 1%

Downloading 'core.snap' 2%

Downloading 'core.snap' 3%

Downloading 'core.snap' 4%

Downloading 'core.snap' 5%

Downloading 'core.snap' 6%

Downloading 'core.snap' 7%

Downloading 'core.snap' 8%
...

summary: - BlockingIOError after "Downloading 'PyYAML-3.11.tar.gz' 23%"
+ CI: BlockingIOError about 50% of the time
Revision history for this message
Kyle Fazzari (kyrofa) wrote :
Download full text (31.0 KiB)

And another, different PR, different test:

======================================================================

ERROR: test_build_rewrites_shebangs (test_python_plugin.PythonPluginTestCase)

test_python_plugin.PythonPluginTestCase.test_build_rewrites_shebangs

----------------------------------------------------------------------

testtools.testresult.real._StringException: output: {{{

Starting snapcraft devel from /root/snapcraft/snapcraft/cli.

"grade" property not specified: defaulting to "stable"

DEPRECATED: The 'python3' plugin's functionality has been replaced by the 'python' plugin, and it will soon be removed.

Setting up part 'python3-test' with plugin 'python3' and properties {'source': 'python3/', 'plugin': 'python3', 'prime': [], 'stage': []}.

Setting up part 'python-test' with plugin 'python' and properties {'stage': ['-lib/python3.5/site-packages/pip*', '-lib/python3.5/site-packages/setuptools*', '-lib/python3.5/site-packages/wheel*', '-lib/python3.5/site-packages/test-0.1.dist-info/RECORD'], 'prime': [], 'source': 'python3/', 'plugin': 'python', 'organize': {'bin/python3_test': 'bin/python_test'}}.

DEPRECATED: The 'python2' plugin's functionality has been replaced by the 'python' plugin, and it will soon be removed.

Setting up part 'python2-test' with plugin 'python2' and properties {'source': 'python2/', 'plugin': 'python2', 'prime': [], 'stage': []}.

Marking 'python3-pip' (and its dependencies) to be fetched

Marking 'python-setuptools' (and its dependencies) to be fetched

Marking 'python-pip' (and its dependencies) to be fetched

Marking 'python-dev' (and its dependencies) to be fetched

Marking 'python3-pkg-resources' (and its dependencies) to be fetched

Marking 'python3-dev' (and its dependencies) to be fetched

Marking 'python-pkg-resources' (and its dependencies) to be fetched

Marking 'python3-setuptools' (and its dependencies) to be fetched

Installing build dependencies: libpython-all-dev libpython-dev libpython2.7 libpython2.7-dev python-all python-all-dev python-dev python-pip python-wheel python2.7-dev

Reading package lists...

Building dependency tree...

Reading state information...

The following NEW packages will be installed:

  libpython-all-dev libpython-dev libpython2.7 libpython2.7-dev python-all

  python-all-dev python-dev python-pip python-wheel python2.7-dev

debconf: unable to initialize frontend: Dialog

debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)

debconf: falling back to frontend: Readline

0 upgraded, 10 newly installed, 0 to remove and 0 not upgraded.

Need to get 0 B/29.4 MB of archives.

After this operation, 45.7 MB of additional disk space will be used.

Selecting previously unselected package libpython2.7:amd64.

(Reading database ...

(Reading database ... 5%

(Reading database ... 10%

(Reading database ... 15%

(Reading database ... 20%

(Reading database ... 25%

(Reading database ... 30%

(Reading database ... 35%

(Reading database ... 40%

(Reading database ... 45%

(Reading database ... 50%

(Reading database ... 55%

(Reading database ... 60%

(Reading database ... 65%

(Reading database ... 70%...

Revision history for this message
Kyle Fazzari (kyrofa) wrote :

It's getting hard to get tests pass because of this one. I know subprocess uses nonblocking IO to get stdout/stderr, I suspect our issues can be traced to that somehow.

Kyle Fazzari (kyrofa)
Changed in snapcraft:
status: New → Confirmed
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.