Comment 0 for bug 1484080
Revision history for this message
| Dennis Dmitriev (ddmitriev) wrote qemu2.0.0 should be upgraded to 2.3 on Jenkins Product CI | #0 |
© 2004
Canonical Ltd.
•
Terms of use
•
Data privacy
•
Contact Launchpad Support
•
Blog
•
Careers
•
System status
•
bbfa235
(Get the code!)
There are several types of issues faced using qemu2.0.0:
1) Fuel admin node freezes after suspend / resume sequence (even without taking a snapshot) eating 100% CPU: [1], [2]
Reproduced in ~70% system tests when snapshot is taken after deploy of cluster.
We use workaround for this issue in system tests: destroy and start Fuel admin node if it inaccessible after revert. Duration of each job that perform system tests can be increased more than on an hour because of this issue.
2) Docker containers on Fuel master node can be completely broken after destroy Fuel admin node that freezes by the reason from the issue №1 : [3]
No workaround here because there is an unrecoverable data loss. Tests are failed.
3) Sometimes libvirt unable to take a snapshot because a volume can be busy for unknown reason at the moment of taking the snapshot: [4]
No workaround here. Tests are failed.
To investigate the issue, was performed the following actions:
- Stop docker containers before taking a snapshot. Issue №1 is reproduced.
- Reduce IO on Fuel admin node by stopping docker containers and rsyslogd. Issue №1 is reproduced.
- Snapshot slave nodes first in order to get Fuel admin node snapshoted when there is no other activity on the host. Issue №1 is reproduced.
- Force closing all SSH sessions on Fuel admin node to reduce the amount of ESTABLISHED connections before taking a snapshot. Issue №1 is reproduced.
- All the above methods at the same time. No success.
Issue №1 can be not reproduced or reproducing constantly on the same host, when no other activity on the host was performed.
--------------
As the possible solution, we can test on Jenkins qemu v2.3 , which doesn't show issues with snapshoting or suspending on local hosts, and upgrade to qemu2.3 in case if the tests did not detect new problems.
[1] https:/
[2] https:/
[3] https:/
[4] https:/