Comment 0 for bug 1946520

During an impish build of ubuntu-cpc images with magic-proxy enabled, the installation of certain packages tried to reach an internet endpoint (specifically http://169.254.169.254/latest/meta-data/instance-id - which appears to be used for EC2 metadata)

The magic-proxy log indicated :

10.10.10.2 - - [01/Oct/2021 13:21:26] urlopen() failed for http://169.254.169.254/2009-04-04/meta-data/instance-id with [Errno 110] Connection timed out
10.10.10.2 - - [01/Oct/2021 13:21:26] code 501, message [Errno 110] Connection timed out
10.10.10.2 - - [01/Oct/2021 13:21:26] "GET /2009-04-04/meta-data/instance-id HTTP/1.1" 501 -
----------------------------------------
Exception occurred during processing of request from ('10.10.10.2', 49672)
Traceback (most recent call last):
  File "/usr/lib/python3.9/urllib/request.py", line 1346, in do_open
    h.request(req.get_method(), req.selector, req.data, headers,
  File "/usr/lib/python3.9/http/client.py", line 1279, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1325, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1274, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.9/http/client.py", line 1034, in _send_output
    self.send(msg)
  File "/usr/lib/python3.9/http/client.py", line 974, in send
    self.connect()
  File "/usr/lib/python3.9/http/client.py", line 945, in connect
    self.sock = self._create_connection(
  File "/usr/lib/python3.9/socket.py", line 844, in create_connection
    raise err
  File "/usr/lib/python3.9/socket.py", line 832, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/share/livecd-rootfs/magic-proxy", line 873, in __get_request
    with urllib.request.urlopen(
  File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.9/urllib/request.py", line 517, in open
    response = self._open(req, data)
  File "/usr/lib/python3.9/urllib/request.py", line 534, in _open
    result = self._call_chain(self.handle_open, protocol, protocol +
  File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.9/urllib/request.py", line 1375, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.9/urllib/request.py", line 1349, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 110] Connection timed out>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3.9/socketserver.py", line 683, in process_request_thread
    self.finish_request(request, client_address)
  File "/usr/lib/python3.9/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/usr/lib/python3.9/socketserver.py", line 747, in __init__
    self.handle()
  File "/usr/lib/python3.9/http/server.py", line 427, in handle
    self.handle_one_request()
  File "/usr/lib/python3.9/http/server.py", line 415, in handle_one_request
    method()
  File "/usr/share/livecd-rootfs/magic-proxy", line 787, in do_GET
    self.__get_request()
  File "/usr/share/livecd-rootfs/magic-proxy", line 887, in __get_request
    self.send_error(501, e.reason)
  File "/usr/lib/python3.9/http/server.py", line 473, in send_error
    'message': html.escape(message, quote=False),
  File "/usr/lib/python3.9/html/__init__.py", line 19, in escape
    s = s.replace("&", "&amp;") # Must be done first!
AttributeError: 'TimeoutError' object has no attribute 'replace'

Because magic-proxy is proxying all traffic to port 80, it will intercept these sort of messages. The TimeoutError ends up bubbling up to the serversocket handling code, which will then cause a broken pipe for all future communications, effectively breaking future apt calls from within the build environment.

The concrete problem here is that inside magic-proxy, self.send_error is called with the argument of e.reason, which can be a string (no error) or an exception (in case of a nested exception). If the nested exception is passed into self.send_error, html.replace will try to be called on the exception, hence the above exception.

In addition to fixing this specific problem, magic-proxy's request handling should catch all exceptions to not interfere with higher up socketserver handling.