Comment 37 for bug 1572521

Revision history for this message
Alexander Kurenyshev (akurenyshev) wrote : Re: Time on nodes was not set with 'ntpdate'

@Maksim Malchuk, @Oleksiy Molchanov
Guys are you joking? Or you really think that this product fail could be fixed in tests?

1) Smart fail couldn't help us, because overall cluster resources are broken, and there is no way to wait for ntp daemon becomes ready. I've implemented it in tests, I tried to wait for that and I was waiting for 2 hours without success.
2) `Non-controller nodes should sync time with the same ntp-server as controllers` approach couldn't working by the same reason: the cluster IS BROKEN. There is no difference between servers for sync, because ntp is not running.

The main problem is mysql performance scheme. We add some amount of memory and it is quickly consumed by mysql. Other resources are in SWAP.
We have never faced this error on the baremetal lab with 64 GB of memory per controller.

@Ksenia Svechnikova
I couldn't get know what happened with that job [1] I need a live environment. Other, the latest fails were not related to this bug. There are problems with setup master node.

[1] https://product-ci.infra.mirantis.net/view/10.0_swarm/job/10.0.system_test.ubuntu.services_reconfiguration_thread_1/