OSTF test "Instance live migration" failed
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
Mirantis OpenStack |
Confirmed
|
High
|
MOS Nova |
Bug Description
Detailed bug description:
After deploy MOS 9.1 snapshot #323 OSFT test "Instance live migration" failed with "Creating instance using the new security group has failed. Please refer to OpenStack logs for more details."
Steps to reproduce:
1. Deploy MOS 9.1 snapshot #323 with 3 controller+mongo, 2 computes+ceph, 1 ironic node (DVR) with fuel-qa system-test based template.
2. Run OSTF functional tests after deploy
Expected results:
All tests should pass
Actual result:
Test "Instance live migration" failed
Reproducibility:
Sometimes
Description of the environment:
- Storage: Ceph (ephemeral-ceph, image-ceph, rados-ceph)
- Network model: Vlan + DVR
Additional information:
Same configuration, but without ceph pass OSTF. On previous builds fails another OSFT tests. After reverting last snapshot (which was made right after OSTF fail) this test pass.
Changed in mos: | |
assignee: | nobody → MOS Nova (mos-nova) |
milestone: | none → 9.2 |
importance: | Undecided → High |
status: | New → Confirmed |
tags: | added: area-nova |
MySQL was down at the moment when OSTF made a request to compute API:
2016-09-29 10:17:47.202 31319 ERROR nova.api. openstack. extensions DBConnectionError: (_mysql_ exceptions. OperationalErro r) (2013, "Lost connection to MySQL server at 'reading initial communication packet', system error: 0") [SQL: u'SELECT 1'] openstack. extensions openstack. wsgi [req-5b2ab66b- c59c-41b1- ae80-bb29e3e5b8 7b f770103c049444a 0b96607c84cfa64 f9 9e1b11b0f43847e 9af03901fb9d53c db - - -] HTTP exception thrown: Unexpected API Error. Please report this at http:// bugs.launchpad. net/nova/ and attach the Nova API log if possible.
2016-09-29 10:17:47.202 31319 ERROR nova.api.
2016-09-29 10:17:47.231 31319 INFO nova.api.
<27>Sep 29 10:17:38 node-1 ocf-mysql-wss: ERROR: p_mysqld: check_if_ galera_ pc(): But I'm running a new cluster, PID:31273, this is a split-brain! 8625-11e6- 972b-07e3d90a05 af:7755, which was not expected resource- agents/ mysql-wss/ mysql-wss. pid of MySQL server not found. Sleeping for 2 seconds. 0 retries left
<27>Sep 29 10:17:38 node-1 ocf-mysql-wss: ERROR: p_mysqld: mysql_monitor(): I'm a master, and my GTID: ea7f8f4a-
<27>Sep 29 10:17:44 node-1 ocf-mysql-wss: ERROR: p_mysqld: mysql_status(): PIDFile /var/run/
<27>Sep 29 10:17:46 node-1 ocf-mysql-wss: ERROR: p_mysqld: mysql_status(): MySQL is not running
<129>Sep 29 10:17:50 node-1 haproxy[16504]: Server mysqld/node-1 is DOWN, reason: Layer7 wrong status, code: 503, info: "Service Unavailable", check duration: 17ms. 0 active and 2 backup servers left. Running on backup. 93 sessions active, 0 requeued, 0 remaining in queue.
(END)
No OOM entries in logs. I think it's the very issue we saw in https:/ /bugs.launchpad .net/fuel/ +bug/1620268