python2.7 test regression: Tunnel connection failed: 403 Forbidden

[Martin Pitt]

When running python2.7 tests on adtcloud, one test fails with

======================================================================
ERROR: test_socketserver (test.test_ssl.ThreadedTests)
Using a SocketServer to create and manage SSL connections.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/test/test_ssl.py", line 2409, in test_socketserver
    f = urllib2.urlopen(url, context=context)
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/usr/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
URLError: <urlopen error Tunnel connection failed: 403 Forbidden>

Test run: http://d-jenkins.ubuntu-ci:8080/job/vivid-adt-python2.7/?
test log on amd64: http://d-jenkins.ubuntu-ci:8080/job/vivid-adt-python2.7/ARCH=amd64,label=adt/41/artifact/results/testsuite-stdout

This runs fine on i386 and the same package version in vivid has already run successfully on amd64 on the old infrastructure, so this is clearly due to a changed environment. We need to reproduce/investigate this locally, perhaps it's some problem with $https_proxy vs. $no_proxy?

[Martin Pitt (pitti)]

Sorry, there are more tests that fail, not just that one. Most of the others look the same, but there's another one which looks a little clearer:

======================================================================
FAIL: test_basic_auth_success (test.test_urllib2_localnet.BasicAuthTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/test/test_urllib2_localnet.py", line 311, in test_basic_auth_success
    self.fail("Basic Auth Failed for url: %s" % self.server_url)
AssertionError: Basic Auth Failed for url: http://127.0.0.1:37672

For the record, this also affects wily (http://d-jenkins.ubuntu-ci:8080/job/wily-adt-python2.7/11/ARCH=amd64,label=adt/artifact/results/testsuite-stdout).

It also affects python3.4, e. g. in http://d-jenkins.ubuntu-ci:8080/job/wily-adt-python3.4/8/ARCH=amd64,label=adt/ (the latest run fails due to a different adtcloud problem). Investigating this on python2.7 only ought to suffice, it's not likely to be specific to the python version.

[Celso Providelo (cprov)]

Martin,

Do you have any results from bootstack or canonistack ?

[Martin Pitt (pitti)]

This failure doesn't happen in Bootstack (using uci-nova). However, bootstack doesn't use the $*_proxy environment by default (and we can't even access squid.internal), while we do have them in ProdStack, right?

[Martin Pitt (pitti)]

This could potentially be related:

http://d-jenkins.ubuntu-ci:8080/view/Wily/view/AutoPkgTest/job/wily-adt-update-manager/lastBuild/ARCH=amd64,label=adt/console

======================================================================
FAIL: test_url_downloadable (test_meta_release_core.TestMetaReleaseCore)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/adt-run.GHhWJW/build.bXb/update-manager-15.04.7/tests/test_meta_release_core.py", line 119, in test_url_downloadable
    environ["http_proxy"])
nose.proxy.AssertionError: False is not true : download with proxy http://localhost:48402/ failed
-------------------- >> begin captured logging << --------------------
root: DEBUG: proxy 1
root: DEBUG: url_downloadable: http://www.ubuntu.com/desktop
root: DEBUG: s='http' n='www.ubuntu.com' p='/desktop' q='' f=''
root: DEBUG: error from httplib: 'HTTP Error 405: Method Not Allowed'

[Martin Pitt (pitti)]

Obsolete now, we don't use adt-cloud-worker any more and http://autopkgtest.ubuntu.com/packages/p/python2.7/ runs fine.