Comment 1 for bug 1982391

Revision history for this message
Jeffrey Chang (modern911) wrote :

This is another run on openstack, and vault/1 failed with same error.
https://solutions.qa.canonical.com/testruns/6706f1f1-daf2-4183-a226-86fc4830067f

Error logs, which repeat for 5 hr.
2024-04-26 12:24:12 DEBUG unit.vault/1.juju-log server.go:325 certificates:244: Opening vault port
2024-04-26 12:24:12 INFO unit.vault/1.juju-log server.go:325 certificates:244: Invoking reactive handler: reactive/vault_handlers.py:360:mysql_setup
2024-04-26 12:24:12 INFO unit.vault/1.juju-log server.go:325 certificates:244: Invoking reactive handler: reactive/vault_handlers.py:391:database_not_ready
2024-04-26 12:24:12 INFO unit.vault/1.juju-log server.go:325 certificates:244: Invoking reactive handler: reactive/vault_handlers.py:481:cluster_connected
2024-04-26 12:24:12 INFO unit.vault/1.juju-log server.go:325 certificates:244: Invoking reactive handler: reactive/vault_handlers.py:697:send_vault_url_and_ca
2024-04-26 12:24:12 WARNING unit.vault/1.juju-log server.go:325 certificates:244: Use of remote_binding in publish_url is deprecated. See LP Bug #1895185
2024-04-26 12:24:12 INFO unit.vault/1.juju-log server.go:325 certificates:244: Invoking reactive handler: reactive/vault_handlers.py:743:prime_assess_status
2024-04-26 12:24:12 INFO unit.vault/1.juju-log server.go:325 certificates:244: Invoking reactive handler: reactive/vault_handlers.py:1120:sync_cert_from_cache
2024-04-26 12:24:43 ERROR unit.vault/1.juju-log server.go:325 certificates:244: Hook error:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 449, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 444, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.10/http/client.py", line 1375, in getresponse
    response.begin()
  File "/usr/lib/python3.10/http/client.py", line 318, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.10/http/client.py", line 279, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.10/socket.py", line 705, in readinto
    return self._sock.recv_into(b)
TimeoutError: timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/urllib3/packages/six.py", line 770, in reraise
    raise value
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 451, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/urllib3/connectionpool.py", line 340, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='127.0.0.1', port=8220): Read timed out. (read timeout=30)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/charms/reactive/__init__.py", line 74, in main
    bus.dispatch(restricted=restricted_mode)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 390, in dispatch
    _invoke(other_handlers)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 359, in _invoke
    handler.invoke()
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/charms/reactive/bus.py", line 181, in invoke
    self._action(*args)
  File "/var/lib/juju/agents/unit-vault-1/charm/reactive/vault_handlers.py", line 1142, in sync_cert_from_cache
    ca_chain = vault_pki.get_chain()
  File "/var/lib/juju/agents/unit-vault-1/charm/lib/charm/vault_pki.py", line 88, in get_chain
    client = vault.get_local_client()
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/tenacity/__init__.py", line 339, in wrapped_f
    return self(f, *args, **kw)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/tenacity/__init__.py", line 430, in __call__
    do = self.iter(retry_state=retry_state)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/tenacity/__init__.py", line 367, in iter
    return fut.result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 451, in result
    return self.__get_result()
  File "/usr/lib/python3.10/concurrent/futures/_base.py", line 403, in __get_result
    raise self._exception
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/tenacity/__init__.py", line 433, in __call__
    result = fn(*args, **kwargs)
  File "/var/lib/juju/agents/unit-vault-1/charm/lib/charm/vault.py", line 284, in get_local_client
    client.auth.approle.login(app_role_id)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/hvac/api/auth_methods/approle.py", line 494, in login
    return self._adapter.login(
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/hvac/adapters.py", line 197, in login
    response = self.post(url, **kwargs)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/hvac/adapters.py", line 126, in post
    return self.request("post", url, **kwargs)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/hvac/adapters.py", line 364, in request
    response = super(JSONAdapter, self).request(*args, **kwargs)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/hvac/adapters.py", line 313, in request
    response = self.session.request(
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/var/lib/juju/agents/unit-vault-1/.venv/lib/python3.10/site-packages/requests/adapters.py", line 529, in send
    raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='127.0.0.1', port=8220): Read timed out. (read timeout=30)