py3 run-test: commands[2] | pytest -v -s --junit-xml=/home/ubuntu/project/generated/kubernetes/k8s-suite/test_audit_default_config-junit.xml --controller=foundation_vsphere --model=kubernetes /home/ubuntu/k8s-validation/jobs/integration/validation.py::test_audit_default_config
============================= test session starts ==============================
platform linux -- Python 3.8.5, pytest-6.1.2, py-1.9.0, pluggy-0.13.1 -- /home/ubuntu/k8s-validation/.tox/py3/bin/python
cachedir: .tox/py3/.pytest_cache
metadata: {'Python': '3.8.5', 'Platform': 'Linux-5.4.0-54-generic-x86_64-with-glibc2.29', 'Packages': {'pytest': '6.1.2', 'py': '1.9.0', 'pluggy': '0.13.1'}, 'Plugins': {'metadata': '1.10.0', 'mock': '3.3.1', 'html': '2.1.1', 'asyncio': '0.14.0', 'flaky': '3.7.0'}}
rootdir: /home/ubuntu/k8s-validation, configfile: pytest.ini
plugins: metadata-1.10.0, mock-3.3.1, html-2.1.1, asyncio-0.14.0, flaky-3.7.0
collecting ... collected 1 item
jobs/integration/validation.py::test_audit_default_config
-------------------------------- live log setup --------------------------------
unknown facade CAASModelOperator
unexpected facade CAASModelOperator found, unable to decipher version to use
unknown delta type: id
set_config_and_wait: new config identical to current, skipping
FAILED
=================================== FAILURES ===================================
__________________________ test_audit_default_config ___________________________
Traceback (most recent call last):
File "/home/ubuntu/k8s-validation/jobs/integration/validation.py", line 1110, in test_audit_default_config
assert after_date > before_date
AssertionError: assert datetime.datetime(2021, 2, 4, 1, 48, 54, 16152) > datetime.datetime(2021, 2, 4, 1, 48, 56, 970805)
------------------------------ Captured log setup ------------------------------
WARNING juju.client.connection:connection.py:706 unknown facade CAASModelOperator
WARNING juju.client.connection:connection.py:730 unexpected facade CAASModelOperator found, unable to decipher version to use
WARNING juju.model:model.py:905 unknown delta type: id
- generated xml file: /home/ubuntu/project/generated/kubernetes/k8s-suite/test_audit_default_config-junit.xml -
----- generated html file: file:///home/ubuntu/k8s-validation/report.html ------
=========================== short test summary info ============================
FAILED jobs/integration/validation.py::test_audit_default_config - assert dat...
============================== 1 failed in 10.69s ==============================
ERROR: InvocationError for command /home/ubuntu/k8s-validation/.tox/py3/bin/pytest -v -s --junit-xml=/home/ubuntu/project/generated/kubernetes/k8s-suite/test_audit_default_config-junit.xml --controller=foundation_vsphere --model=kubernetes /home/ubuntu/k8s-validation/jobs/integration/validation.py::test_audit_default_config (exited with code 1)
___________________________________ summary ____________________________________
ERROR: py3: commands failed
This is ck 1.20 on vsphere.
Example test run:
https://solutions.qa.canonical.com/testruns/testRun/1a94386d-fd90-44af-ab68-b96e45dca50d
This test seems a bit flaky in general. It's not clear what happened here, but the failure implies one of the following. Either:
1. The kubectl command didn't run (unlikely)
2. The apiserver hadn't written the event to audit logs yet (possible if audit log writes are buffered)
3. The event had been logged with an older timestamp (possible if kubernetes-master clocks are >5s apart)
I think we need to improve the test output here. Both before_date and after_date should be logged, along with maybe the last ~10 lines of audit.log from each master.