`lxc` commands returning `Error: Failed to begin transaction: context deadline exceeded`
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
linux (Ubuntu) |
New
|
Undecided
|
Unassigned | ||
lxd (Ubuntu) |
New
|
Undecided
|
Unassigned |
Bug Description
Since around 2024-05-24, we've been experiencing an error in the autopkgtest.
The symptom is all `lxc` command invokation returning `Error: Failed to begin transaction: context deadline exceeded`, whether from the machine usually running the commands through an LXD remote (like `lxc list lxd-armhf-
This is happening quite randomly, sometime every few days, sometimes two times per day on the same machine. At one point, the 16 workers where in that same situation at the same time (around 2024-05-26, Sunday evening, when nobody took care of that).
Those workers are all `arm64` Jammy machines, and the containers running on them are all `armhf` of all the supported Ubuntu releases.
LXD version: 5.21.1 LTS, installed with snapd
Kernel version: 5.15.0-107.117
Here are logs from around the issue on three machines:
https:/
https:/
https:/
As we can see, the pattern is always the same, and has also been observed on other problematic machines:
* First `kernel: physZlw57F: renamed from eth0` and following network related lines.
These lines are common during normal operation, but also always happen before the kernel calltrace. That might still just be a coincidence.
* Then `kernel: Unable to handle kernel paging request at virtual address` with the calltrace
* Finally LXD starting to have issues with the `Failed to begin transaction: context deadline exceeded`.
Sometimes these lines start to appear half an hour after the kernel issue, but we've never seen them before.
One workaround we're experimenting right now is running the HWE kernel (version 6.5.0-35.
tags: | added: cuqa-manual-testing |
Upload the logs as attachment for longer lifespan.