Comment 41 for bug 1687027

Revision history for this message
Slawek Kaplonski (slaweq) wrote :

Today I checked one failed test, from http://logs.openstack.org/30/626830/9/check/neutron-functional/e66470c/logs/dsvm-functional-logs/neutron.tests.functional.db.test_migrations.TestModelsMigrationsMysql.test_check_mysql_engine.txt.gz and compared it with same test which passed: http://logs.openstack.org/85/624685/2/check/neutron-functional/fede00f/logs/dsvm-functional-logs/neutron.tests.functional.db.test_migrations.TestModelsMigrationsMysql.test_check_mysql_engine.txt.gz

There is no some single migration step which takes long time or hangs. It is just very slow all time in case of failed test. E.g. first step, called "Running upgrade -> kilo" took 6 minutes (!) in failed test and 2.5 second in good run.

Comparing dstat from those 2 runs I can see that in this failed test there was less than 100 IOPS for most of the time. In case of this passed test it was around 2k - 3k IOPS.
So I think that this issue is just because of some slow VMs which maybe are on some overloaded nodes.