zsysd uses more than half of CPU and disk IO
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
zsys (Ubuntu) |
Fix Released
|
High
|
Didier Roche-Tolomelli | ||
Focal |
Fix Released
|
High
|
Didier Roche-Tolomelli |
Bug Description
[Impact]
* Infinite loop during GC phase can trigger (high CPU usage and disk IO). This has a no side-effect, just looping withouth any modifications.
* The fix is to cover this case (a dataset that the GC can‘t destroy, put in a black list for the next loop state). The issue is that due to the order of if/else if, the else if "in blacklist" was never evaluated.
* The new code is covered by a specific test and coverage confirms this is now run.
[Test Case]
1. Create a dataset clone on an userdataset. This clone should have a snapshot or a snapshot/clone to not be automatically deletable.
2. Run zsysctl service gc
-> no more infinite loop (the commands exits after a while with no timeout) with the fixed package.
[Regression Potential]
* GC is a separate, timer-based process and don’t impact boot or other operations.
* The 30 existing tests on the GC didn’t regress and we now have some additional ones around those use case.
---------
I was installed Ubuntu 20.04 with zfs yesterday, it was OK until a few updates, after that the zsysd was started draining CPU and disk IO. How can I fix it to back to normal? Is it possible to rollback recent system state? If Yes, just how?
NOTE: I was stopped and disabled zsysd service and removed zsysd.socket file, after that it worked normally for now. And there is this error occured when I am installing or removing a software package via apt:
ERROR couldn't connect to zsys daemon: connection error: desc = "transport: Error while dialing dial unix /run/zsysd.sock: connect: no such file or directory"
I think this is not a good approach
ProblemType: Bug
DistroRelease: Ubuntu 20.04
Package: zsys 0.4.2
ProcVersionSign
Uname: Linux 5.4.0-21-generic x86_64
NonfreeKernelMo
ApportVersion: 2.20.11-0ubuntu22
Architecture: amd64
CurrentDesktop: ubuntu:GNOME
Date: Fri Apr 3 05:37:29 2020
InstallationDate: Installed on 2020-04-01 (1 days ago)
InstallationMedia: Ubuntu 20.04 LTS "Focal Fossa" - Alpha amd64 (20200330)
SourcePackage: zsys
UpgradeStatus: No upgrade log present (probably fresh install)
description: | updated |
description: | updated |
description: | updated |
Changed in zsys (Ubuntu): | |
importance: | Undecided → High |
assignee: | nobody → Didier Roche (didrocks) |
description: | updated |
Changed in zsys (Ubuntu Focal): | |
importance: | Undecided → High |
assignee: | nobody → Didier Roche (didrocks) |
description: | updated |
Thanks for reporting this bug and sorry you have an issue with zsys.
It seems that zsys is going crazy in an infinite loop, I would be interested to know why:
1. can you try running manually "sudo zsysd -vv" and see paste the logs here? Do you have the spinning loops in that way as well?
2. if you don’t have the CPU going crazy, keep zsysd running as above and please run in another terminal "zsysctl service gc". Is the CPU going crazy now? If so, please attach the logs to this report.
Can you paste as well the result of:
- zpool list
- zfs list -t all
Disabling the socket and service is fine for now to prevent you going into this loop (but you won’t be able to revert as it’s zsys handling this). The error message just tell that it can’t save automatically the state which is expected as it’s zsysd doing it. This doesn’t hurt apart from not saving and not breaking your apt state.
Note that we have important (not zsys but zfs layout related) fixes in the incoming beta image (due today), I doubt though this is what is causing your issue.
Thanks in advance!