[2019-08-22 18:19:19,872] 1538 WARNING MainThread ssh.get_active_controller:: No ssh client found for ip_20_27 [2019-08-22 18:19:19,872] 151 INFO MainThread ssh.connect :: Attempt to connect to host - 128.224.151.49 [2019-08-22 18:19:21,132] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 18:19:21,235] 423 DEBUG MainThread ssh.expect :: Output: controller-0:~$ [2019-08-22 18:19:21,236] 183 INFO MainThread ssh.connect :: Login successful! [2019-08-22 18:19:21,236] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 18:19:21,337] 423 DEBUG MainThread ssh.expect :: Output: controller-0:~$ [2019-08-22 18:19:21,339] 423 DEBUG MainThread ssh.expect :: Output: controller-0:~$ [2019-08-22 18:19:24,342] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:24,342] 301 DEBUG MainThread ssh.send :: Send 'unset PROMPT_COMMAND' [2019-08-22 18:19:24,445] 423 DEBUG MainThread ssh.expect :: Output: controller-0:~$ [2019-08-22 18:19:24,446] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:24,446] 301 DEBUG MainThread ssh.send :: Send 'export TMOUT=0' [2019-08-22 18:19:24,549] 423 DEBUG MainThread ssh.expect :: Output: controller-0:~$ [2019-08-22 18:19:24,549] 1630 INFO MainThread ssh.set_active_controller:: Active controller client for ip_20_27 is set. Host ip/name: 128.224.151.49 [2019-08-22 18:19:24,549] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:24,549] 301 DEBUG MainThread ssh.send :: Send 'cat /etc/platform/openrc' [2019-08-22 18:19:24,653] 423 DEBUG MainThread ssh.expect :: Output: unset OS_SERVICE_TOKEN export OS_ENDPOINT_TYPE=internalURL export CINDER_ENDPOINT_TYPE=internalURL export OS_USERNAME=admin export OS_PASSWORD=`TERM=linux /opt/platform/.keyring/19.09/.CREDENTIAL 2>/dev/null` export OS_AUTH_TYPE=password export OS_AUTH_URL=http://192.168.204.2:5000/v3 export OS_PROJECT_NAME=admin export OS_USER_DOMAIN_NAME=Default export OS_PROJECT_DOMAIN_NAME=Default export OS_IDENTITY_API_VERSION=3 export OS_REGION_NAME=RegionOne export OS_INTERFACE=internal if [ ! -z "${OS_PASSWORD}" ]; then export PS1='[\u@\h \W(keystone_$OS_USERNAME)]\$ ' else echo 'Openstack Admin credentials can only be loaded from the active controller.' export PS1='\h:\w\$ ' fi controller-0:~$ [2019-08-22 18:19:24,653] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:24,756] 423 DEBUG MainThread ssh.expect :: Output: 0 controller-0:~$ [2019-08-22 18:19:24,757] 301 DEBUG MainThread ssh.send :: Send 'source /etc/platform/openrc' [2019-08-22 18:19:25,461] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:25,462] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:25,565] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:25,565] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:25,565] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-interface internal --os-region-name RegionOne endpoint list' [2019-08-22 18:19:27,726] 423 DEBUG MainThread ssh.expect :: Output: +----------------------------------+-----------+--------------+-----------------+---------+-----------+-------------------------------+ | ID | Region | Service Name | Service Type | Enabled | Interface | URL | +----------------------------------+-----------+--------------+-----------------+---------+-----------+-------------------------------+ | 5d15ebc3118748b9be82ae1496a99cbe | RegionOne | fm | faultmanagement | True | admin | http://192.168.204.2:18002 | | 1029542c6a3641b48a4e3c6df4f0e424 | RegionOne | fm | faultmanagement | True | internal | http://192.168.204.2:18002 | | fd54bc43197247d3acc1cba2e4c72fc4 | RegionOne | fm | faultmanagement | True | public | http://128.224.151.49:18002 | | 3bb26437ce6444479ee83fc88e8e11da | RegionOne | patching | patching | True | admin | http://192.168.204.2:5491 | | 538f18127cc44cb397217b75f32b32fc | RegionOne | patching | patching | True | internal | http://192.168.204.2:5491 | | 113ab897582f4d10991cb0f8b4beb38a | RegionOne | patching | patching | True | public | http://128.224.151.49:15491 | | f43fbd1cf653458bb4693e92bc7e1011 | RegionOne | vim | nfv | True | admin | http://192.168.204.2:4545 | | 60ad82fa96664064bfc2e7fe2d8f1305 | RegionOne | vim | nfv | True | internal | http://192.168.204.2:4545 | | 591e6ef021df453b9cfe51e8158201ee | RegionOne | vim | nfv | True | public | http://128.224.151.49:4545 | | 0b57d8b2394d457e986c496a8955fa40 | RegionOne | smapi | smapi | True | admin | http://192.168.204.2:7777 | | 16532bbb8f3e4a97969182e745d318f3 | RegionOne | smapi | smapi | True | internal | http://192.168.204.2:7777 | | 424a7481e56644d8aad7ab0cca11bb2a | RegionOne | smapi | smapi | True | public | http://128.224.151.49:7777 | | 1057a26bf9e94c65985a2ab8efa1923e | RegionOne | keystone | identity | True | admin | http://192.168.204.2:5000/v3 | | ed451e59940f4b41836513569c22aba7 | RegionOne | keystone | identity | True | internal | http://192.168.204.2:5000/v3 | | 3c1989e5147e49d1b921c1839eb22cc6 | RegionOne | keystone | identity | True | public | http://128.224.151.49:5000/v3 | | 4e90475044ab441d85466a9052b735f9 | RegionOne | barbican | key-manager | True | admin | http://192.168.204.2:9311 | | 9e4b6054f0b64f27a43c940c89205432 | RegionOne | barbican | key-manager | True | internal | http://192.168.204.2:9311 | | 7ff1ea2191254bcf92563d657fa5bd92 | RegionOne | barbican | key-manager | True | public | http://128.224.151.49:9311 | | 68ad9a2834f946d2907a73a675946d6f | RegionOne | sysinv | platform | True | admin | http://192.168.204.2:6385/v1 | | 9d63ed0377c24896a41c55cd7ac2ec0e | RegionOne | sysinv | platform | True | internal | http://192.168.204.2:6385/v1 | | c0cb1d7f221e4e0ab3df9789c30bd24e | RegionOne | sysinv | platform | True | public | http://128.224.151.49:6385/v1 | +----------------------------------+-----------+--------------+-----------------+---------+-----------+-------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:27,727] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:27,830] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:27,830] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:27,830] 301 DEBUG MainThread ssh.send :: Send 'unset OS_REGION_NAME' [2019-08-22 18:19:27,933] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:27,933] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:28,036] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:28,036] 575 DEBUG MainThread table_parser.get_values:: Returning matching URL value(s): ['http://128.224.151.49:5000/v3'] [2019-08-22 18:19:28,051] 268 DEBUG MainThread conftest.testcase_log:: ----------------------------------------------------------------- Setup started for: testcases/stress/nova/test_migrate.py::test_migrate_stress[volume-1000] [2019-08-22 18:19:28,053] 21 INFO MainThread conftest.setup_test_session:: ====================== Setup Step 1: (session) Setting up test session... [2019-08-22 18:19:28,053] 85 INFO MainThread setups.setup_primary_tenant:: Primary Tenant for test session is set to tenant1 [2019-08-22 18:19:28,054] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:28,054] 301 DEBUG MainThread ssh.send :: Send 'cat /etc/build.info' [2019-08-22 18:19:28,158] 423 DEBUG MainThread ssh.expect :: Output: ### ### StarlingX ### Release 19.09 ### ### Wind River Systems, Inc. ### SW_VERSION="19.09" BUILD_TARGET="Host Installer" BUILD_TYPE="Formal" BUILD_ID="2019-08-20_20-59-00" SRC_BUILD_ID="16" JOB="StarlingX_2.0_build" BUILD_BY="jenkins" BUILD_NUMBER="16" BUILD_HOST="yow-cgts4-lx.wrs.com" BUILD_DATE="2019-08-20 21:00:35 -0400" [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:28,158] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:28,261] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:28,261] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 18:19:28,261] 301 DEBUG MainThread ssh.send :: Send 'sudo sw-patch query' [2019-08-22 18:19:28,382] 423 DEBUG MainThread ssh.expect :: Output: Password: [2019-08-22 18:19:28,382] 301 DEBUG MainThread ssh.send :: Send 'Li69nux*' [2019-08-22 18:19:28,707] 423 DEBUG MainThread ssh.expect :: Output: Patch ID RR Release Patch State ======== == ======= =========== [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:28,708] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:28,811] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:28,811] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 18:19:28,914] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:28,914] 139 DEBUG MainThread ssh.connect :: Already connected to 128.224.151.49. Do nothing. [2019-08-22 18:19:28,915] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:28,915] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list' [2019-08-22 18:19:30,579] 423 DEBUG MainThread ssh.expect :: Output: +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applied | completed | | stx-openstack | 1.0-17-centos-stable- | armada-manifest | stx-openstack.yaml | applied | Application update from version 1.0-17-centos-stable-versioned to version 1.0-17-centos-stable-latest | | | latest | | | | completed. | | | | | | | | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:30,579] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:30,682] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:30,683] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: stx-openstack [2019-08-22 18:19:30,683] 736 INFO MainThread container_helper.is_stx_openstack_deployed:: ['applied'] [2019-08-22 18:19:30,683] 151 INFO MainThread ssh.connect :: Attempt to connect to host - 128.224.186.181 [2019-08-22 18:19:31,988] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 18:19:32,090] 423 DEBUG MainThread ssh.expect :: Output: svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:19:32,090] 183 INFO MainThread ssh.connect :: Login successful! [2019-08-22 18:19:32,090] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 18:19:32,192] 423 DEBUG MainThread ssh.expect :: Output: svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:19:32,192] 423 DEBUG MainThread ssh.expect :: Output: svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:19:35,196] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:35,196] 301 DEBUG MainThread ssh.send :: Send 'unset PROMPT_COMMAND' [2019-08-22 18:19:35,298] 423 DEBUG MainThread ssh.expect :: Output: svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:19:35,298] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:35,298] 301 DEBUG MainThread ssh.send :: Send 'export TMOUT=0' [2019-08-22 18:19:35,400] 423 DEBUG MainThread ssh.expect :: Output: svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:19:35,400] 1471 INFO MainThread ssh.set_natbox_client:: NatBox 128.224.186.181 ssh client is set [2019-08-22 18:19:35,400] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:19:35,401] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:35,401] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list' [2019-08-22 18:19:37,047] 423 DEBUG MainThread ssh.expect :: Output: +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applied | completed | | stx-openstack | 1.0-17-centos-stable- | armada-manifest | stx-openstack.yaml | applied | Application update from version 1.0-17-centos-stable-versioned to version 1.0-17-centos-stable-latest | | | latest | | | | completed. | | | | | | | | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:37,047] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:37,151] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:37,151] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: stx-openstack [2019-08-22 18:19:37,151] 736 INFO MainThread container_helper.is_stx_openstack_deployed:: ['applied'] [2019-08-22 18:19:37,152] 121 INFO MainThread setups.setup_keypair:: scp key file from controller to NATBox [2019-08-22 18:19:37,152] 1530 INFO MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:37,152] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:37,152] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne keypair list' [2019-08-22 18:19:38,980] 423 DEBUG MainThread ssh.expect :: Output: +-----------------+-------------------------------------------------+ | Name | Fingerprint | +-----------------+-------------------------------------------------+ | keypair-tenant1 | 9d:f7:59:61:7d:c5:07:c5:5d:bb:74:ec:fb:7a:d4:70 | +-----------------+-------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:38,980] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:39,083] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:39,083] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:39,083] 301 DEBUG MainThread ssh.send :: Send 'groups sysadmin' [2019-08-22 18:19:39,188] 423 DEBUG MainThread ssh.expect :: Output: sysadmin : sys_protected root [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:39,188] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:39,291] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:39,291] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:39,291] 301 DEBUG MainThread ssh.send :: Send 'stat ~/keyfile_ip_20_27.pem' [2019-08-22 18:19:39,396] 423 DEBUG MainThread ssh.expect :: Output: File: ‘/home/sysadmin/keyfile_ip_20_27.pem’ Size: 1675 Blocks: 8 IO Block: 4096 regular file Device: 803h/2051d Inode: 145280 Links: 1 Access: (0600/-rw-------) Uid: (42425/sysadmin) Gid: ( 345/sys_protected) Access: 2019-08-22 17:12:49.907815276 +0000 Modify: 2019-08-22 17:12:48.230794989 +0000 Change: 2019-08-22 17:12:48.982804086 +0000 Birth: - [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:39,396] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:39,499] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:39,499] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:39,499] 301 DEBUG MainThread ssh.send :: Send 'mkdir -p ~/priv_keys' [2019-08-22 18:19:39,605] 423 DEBUG MainThread ssh.expect :: Output: svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:19:39,605] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:39,707] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:19:39,707] 301 DEBUG MainThread ssh.send :: Send 'scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null sysadmin@128.224.151.49:~/keyfile_ip_20_27.pem ~/priv_keys/keyfile_ip_20_27.pem' [2019-08-22 18:19:39,889] 423 DEBUG MainThread ssh.expect :: Output: Warning: Permanently added '128.224.151.49' (ECDSA) to the list of known hosts. Release 19.09 ------------------------------------------------------------------------ W A R N I N G *** W A R N I N G *** W A R N I N G *** W A R N I N G *** ------------------------------------------------------------------------ THIS IS A PRIVATE COMPUTER SYSTEM. This computer system including all related equipment, network devices (specifically including Internet access), are provided only for authorized use. All computer systems may be monitored for all lawful purposes, including to ensure that their use is authorized, for management of the system, to facilitate protection against unauthorized access, and to verify security procedures, survivability and operational security. Monitoring includes active attacks by authorized personnel and their entities to test or verify the security of the system. During monitoring, information may be examined, recorded, copied and used for authorized purposes. All information including personal information, placed on or sent over this system may be monitored. Uses of this system, authorized or unauthorized, constitutes consent to monitoring of this system. Unauthorized use may subject you to criminal prosecution. Evidence of any such unauthorized use collected during monitoring may be used for administrative, criminal or other adverse action. Use of this system constitutes consent to monitoring for these purposes. sysadmin@128.224.151.49's password: [2019-08-22 18:19:39,889] 301 DEBUG MainThread ssh.send :: Send 'Li69nux*' [2019-08-22 18:19:40,077] 423 DEBUG MainThread ssh.expect :: Output: keyfile_ip_20_27.pem 0% 0 0.0KB/s --:-- ETA keyfile_ip_20_27.pem 100% 1675 1.6KB/s 00:00 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:19:40,077] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:40,179] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:19:40,179] 242 INFO MainThread setups.setup_keypair:: private key is copied to NatBox: ~/priv_keys/keyfile_ip_20_27.pem [2019-08-22 18:19:40,180] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:40,180] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne host-show controller-0' [2019-08-22 18:19:41,857] 423 DEBUG MainThread ssh.expect :: Output: +---------------------+----------------------------------------------------------------------+ | Property | Value | +---------------------+----------------------------------------------------------------------+ | action | none | | administrative | unlocked | | availability | available | | bm_ip | None | | bm_type | None | | bm_username | None | | boot_device | /dev/disk/by-path/pci-0000:09:00.0-sas-0x5001e67657b7e000-lun-0 | | capabilities | {u'stor_function': u'monitor', u'Personality': u'Controller-Active'} | | config_applied | 3efde92d-4a2c-4827-a482-ad59fb22c397 | | config_status | None | | config_target | 3efde92d-4a2c-4827-a482-ad59fb22c397 | | console | ttyS0,115200n8 | | created_at | 2019-08-21T18:56:12.621476+00:00 | | hostname | controller-0 | | id | 1 | | install_output | text | | install_state | None | | install_state_info | None | | inv_state | inventoried | | invprovision | provisioned | | location | {} | | mgmt_ip | 192.168.204.3 | | mgmt_mac | 00:1e:67:65:7b:7f | | operational | enabled | | personality | controller | | reserved | False | | rootfs_device | /dev/disk/by-path/pci-0000:09:00.0-sas-0x5001e67657b7e000-lun-0 | | serialid | None | | software_load | 19.09 | | task | | | tboot | false | | ttys_dcd | None | | updated_at | 2019-08-22T18:19:22.898606+00:00 | | uptime | 83256 | | uuid | e270d106-baa2-4a7e-ba55-bc00bfaf74ff | | vim_progress_status | services-enabled | +---------------------+----------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:41,857] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:41,960] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:41,961] 673 WARNING MainThread table_parser.get_value_two_col_table:: Field subfunctions is not found in table. [2019-08-22 18:19:41,961] 147 INFO MainThread system_helper.is_aio_system:: This is not small footprint system. [2019-08-22 18:19:41,961] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:41,961] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne host-list' [2019-08-22 18:19:43,611] 423 DEBUG MainThread ssh.expect :: Output: +----+--------------+-------------+----------------+-------------+--------------+ | id | hostname | personality | administrative | operational | availability | +----+--------------+-------------+----------------+-------------+--------------+ | 1 | controller-0 | controller | unlocked | enabled | available | | 2 | compute-1 | worker | unlocked | enabled | available | | 3 | compute-2 | worker | unlocked | enabled | available | | 4 | compute-3 | worker | locked | disabled | online | | 5 | controller-1 | controller | unlocked | enabled | available | | 6 | storage-0 | storage | unlocked | enabled | available | | 7 | storage-1 | storage | unlocked | enabled | available | | 8 | compute-0 | worker | unlocked | enabled | available | +----+--------------+-------------+----------------+-------------+--------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:43,612] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:43,715] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:43,715] 282 DEBUG MainThread system_helper.get_hosts:: Filtered hosts: ['storage-0', 'storage-1'] [2019-08-22 18:19:43,716] 54 INFO MainThread system_helper.get_sys_type:: ============= System type: Storage ============== [2019-08-22 18:19:43,716] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:43,716] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:43,716] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne host-list' [2019-08-22 18:19:45,378] 423 DEBUG MainThread ssh.expect :: Output: +----+--------------+-------------+----------------+-------------+--------------+ | id | hostname | personality | administrative | operational | availability | +----+--------------+-------------+----------------+-------------+--------------+ | 1 | controller-0 | controller | unlocked | enabled | available | | 2 | compute-1 | worker | unlocked | enabled | available | | 3 | compute-2 | worker | unlocked | enabled | available | | 4 | compute-3 | worker | locked | disabled | online | | 5 | controller-1 | controller | unlocked | enabled | available | | 6 | storage-0 | storage | unlocked | enabled | available | | 7 | storage-1 | storage | unlocked | enabled | available | | 8 | compute-0 | worker | unlocked | enabled | available | +----+--------------+-------------+----------------+-------------+--------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:45,378] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:45,481] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:45,482] 282 DEBUG MainThread system_helper.get_hosts:: Filtered hosts: ['controller-0', 'controller-1'] [2019-08-22 18:19:45,482] 367 INFO MainThread setups._rsync_files_to_con1:: rsync test files from controller-0 to controller-1 if not already done [2019-08-22 18:19:45,482] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:45,482] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:45,482] 301 DEBUG MainThread ssh.send :: Send 'hostname' [2019-08-22 18:19:45,586] 423 DEBUG MainThread ssh.expect :: Output: controller-0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:45,587] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 18:19:45,689] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:45,690] 1185 INFO MainThread ssh.connect :: Attempt to connect to controller-1 from 128.224.151.49... [2019-08-22 18:19:45,690] 301 DEBUG MainThread ssh.send :: Send '/usr/bin/ssh -o RSAAuthentication=no -o PubkeyAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null sysadmin@controller-1' [2019-08-22 18:19:45,864] 423 DEBUG MainThread ssh.expect :: Output: Warning: Permanently added 'controller-1,192.168.204.4' (ECDSA) to the list of known hosts. Release 19.09 ------------------------------------------------------------------------ W A R N I N G *** W A R N I N G *** W A R N I N G *** W A R N I N G *** ------------------------------------------------------------------------ THIS IS A PRIVATE COMPUTER SYSTEM. This computer system including all related equipment, network devices (specifically including Internet access), are provided only for authorized use. All computer systems may be monitored for all lawful purposes, including to ensure that their use is authorized, for management of the system, to facilitate protection against unauthorized access, and to verify security procedures, survivability and operational security. Monitoring includes active attacks by authorized personnel and their entities to test or verify the security of the system. During monitoring, information may be examined, recorded, copied and used for authorized purposes. All information including personal information, placed on or sent over this system may be monitored. Uses of this system, authorized or unauthorized, constitutes consent to monitoring of this system. Unauthorized use may subject you to criminal prosecution. Evidence of any such unauthorized use collected during monitoring may be used for administrative, criminal or other adverse action. Use of this system constitutes consent to monitoring for these purposes. sysadmin@controller-1's password: [2019-08-22 18:19:45,864] 301 DEBUG MainThread ssh.send :: Send 'Li69nux*' [2019-08-22 18:19:46,119] 423 DEBUG MainThread ssh.expect :: Output: Last login: Thu Aug 22 18:18:25 2019 from 192.168.204.3 /etc/motd.d/00-header:  WARNING: Unauthorized access to this system is forbidden and will be prosecuted by law. By accessing this system, you agree that your actions may be monitored if unauthorized usage is suspected. [?1034hcontroller-1:~$ [2019-08-22 18:19:46,119] 1220 INFO MainThread ssh.connect :: Successfully connected to controller-1 from 128.224.151.49! [2019-08-22 18:19:46,120] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:46,120] 301 DEBUG MainThread ssh.send :: Send 'export TMOUT=0' [2019-08-22 18:19:46,223] 423 DEBUG MainThread ssh.expect :: Output: controller-1:~$ [2019-08-22 18:19:46,223] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:46,326] 423 DEBUG MainThread ssh.expect :: Output: 0 controller-1:~$ [2019-08-22 18:19:46,327] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:46,327] 301 DEBUG MainThread ssh.send :: Send 'hostname' [2019-08-22 18:19:46,431] 423 DEBUG MainThread ssh.expect :: Output: controller-1 controller-1:~$ [2019-08-22 18:19:46,431] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:46,432] 301 DEBUG MainThread ssh.send :: Send 'stat /home/sysadmin//images/tis-centos-guest.img' [2019-08-22 18:19:46,536] 423 DEBUG MainThread ssh.expect :: Output: File: ‘/home/sysadmin//images/tis-centos-guest.img’ Size: 1611581440 Blocks: 3147632 IO Block: 4096 regular file Device: 803h/2051d Inode: 792966 Links: 1 Access: (0664/-rw-rw-r--) Uid: (42425/sysadmin) Gid: ( 345/sys_protected) Access: 2019-08-22 17:13:11.631135520 +0000 Modify: 2019-05-21 18:43:13.000000000 +0000 Change: 2019-08-22 17:13:11.631135520 +0000 Birth: - controller-1:~$ [2019-08-22 18:19:46,537] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:46,640] 423 DEBUG MainThread ssh.expect :: Output: 0 controller-1:~$ [2019-08-22 18:19:46,640] 377 INFO MainThread setups._rsync_files_to_con1:: Test files already exist on controller-1. Skip rsync. [2019-08-22 18:19:46,640] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 18:19:46,743] 423 DEBUG MainThread ssh.expect :: Output: controller-1:~$ [2019-08-22 18:19:46,744] 301 DEBUG MainThread ssh.send :: Send 'exit' [2019-08-22 18:19:46,848] 423 DEBUG MainThread ssh.expect :: Output: exit logout Connection to controller-1 closed. [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:46,849] 1257 INFO MainThread ssh.close :: ssh session to controller-1 is closed and returned to parent session 128.224.151.49 [2019-08-22 18:19:46,849] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:46,850] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:46,850] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list' [2019-08-22 18:19:48,507] 423 DEBUG MainThread ssh.expect :: Output: +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applied | completed | | stx-openstack | 1.0-17-centos-stable- | armada-manifest | stx-openstack.yaml | applied | Application update from version 1.0-17-centos-stable-versioned to version 1.0-17-centos-stable-latest | | | latest | | | | completed. | | | | | | | | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:48,507] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:48,610] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:48,610] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: stx-openstack [2019-08-22 18:19:48,610] 736 INFO MainThread container_helper.is_stx_openstack_deployed:: ['applied'] [2019-08-22 18:19:48,611] 133 INFO MainThread resource_create.__create_image:: ====================== Setup Step 2: (session) Get or create a glance image with tis-centos-guest guest OS [2019-08-22 18:19:48,611] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:48,611] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:48,611] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne image list' [2019-08-22 18:19:50,403] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+------------------+--------+ | ID | Name | Status | +--------------------------------------+------------------+--------+ | fd57e28e-6ea4-4380-ba0d-32d0f787c152 | tis-centos-guest | active | +--------------------------------------+------------------+--------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:50,403] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:50,506] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:50,507] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:50,507] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:50,508] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list' [2019-08-22 18:19:52,155] 423 DEBUG MainThread ssh.expect :: Output: +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applied | completed | | stx-openstack | 1.0-17-centos-stable- | armada-manifest | stx-openstack.yaml | applied | Application update from version 1.0-17-centos-stable-versioned to version 1.0-17-centos-stable-latest | | | latest | | | | completed. | | | | | | | | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:52,155] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:52,258] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:52,259] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: stx-openstack [2019-08-22 18:19:52,259] 736 INFO MainThread container_helper.is_stx_openstack_deployed:: ['applied'] [2019-08-22 18:19:52,259] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:52,259] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:52,259] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne security group list' [2019-08-22 18:19:54,029] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags | +--------------------------------------+---------+------------------------+----------------------------------+------+ | 37e619d8-1724-4dc7-a130-d4df9ffecb0d | default | Default security group | 71e4950fb29c40d8ac13742b4990dfad | [] | +--------------------------------------+---------+------------------------+----------------------------------+------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:54,029] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:54,132] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:54,132] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:54,133] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:54,133] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne security group rule list --long' [2019-08-22 18:19:55,891] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------+-----------+------------+-----------+-----------+--------------------------------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Direction | Ethertype | Remote Security Group | Security Group | +--------------------------------------+-------------+-----------+------------+-----------+-----------+--------------------------------------+--------------------------------------+ | 828f8902-681c-4dbd-b5a6-1e5ee477c1c9 | None | None | | egress | IPv4 | None | 37e619d8-1724-4dc7-a130-d4df9ffecb0d | | 927ef8e5-b10a-4f72-8c46-aefaa5f8b6f0 | icmp | 0.0.0.0/0 | | ingress | IPv4 | None | 37e619d8-1724-4dc7-a130-d4df9ffecb0d | | c0f13b7f-e844-402e-baad-eda4113097b5 | None | None | | ingress | IPv6 | 37e619d8-1724-4dc7-a130-d4df9ffecb0d | 37e619d8-1724-4dc7-a130-d4df9ffecb0d | | dd9ff74d-1003-44a6-bcdc-5e5d20f613eb | None | None | | ingress | IPv4 | 37e619d8-1724-4dc7-a130-d4df9ffecb0d | 37e619d8-1724-4dc7-a130-d4df9ffecb0d | | e13c763b-5030-4071-ba8d-1d9582b8ed0c | None | None | | egress | IPv6 | None | 37e619d8-1724-4dc7-a130-d4df9ffecb0d | | e5088825-d652-41fd-b90d-aa4c7245f5b6 | tcp | 0.0.0.0/0 | 22:22 | ingress | IPv4 | None | 37e619d8-1724-4dc7-a130-d4df9ffecb0d | +--------------------------------------+-------------+-----------+------------+-----------+-----------+--------------------------------------+--------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:55,891] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:55,995] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:55,995] 127 INFO MainThread verify_fixtures.pre_alarms_session:: Default security group rules for tenant1 already configured to allow ping and ssh [2019-08-22 18:19:55,995] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:55,996] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:55,996] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant2' --os-password 'Li69nux*' --os-project-name tenant2 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne security group list' [2019-08-22 18:19:57,776] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+---------+------------------------+----------------------------------+------+ | ID | Name | Description | Project | Tags | +--------------------------------------+---------+------------------------+----------------------------------+------+ | 04098ac2-bff0-40bb-b17b-4e4c78823856 | default | Default security group | 5fad34de5c5b4397b8e0fc2a83d94854 | [] | +--------------------------------------+---------+------------------------+----------------------------------+------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:57,777] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:57,879] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:57,880] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:57,880] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:57,880] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant2' --os-password 'Li69nux*' --os-project-name tenant2 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne security group rule list --long' [2019-08-22 18:19:59,618] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------+-----------+------------+-----------+-----------+--------------------------------------+--------------------------------------+ | ID | IP Protocol | IP Range | Port Range | Direction | Ethertype | Remote Security Group | Security Group | +--------------------------------------+-------------+-----------+------------+-----------+-----------+--------------------------------------+--------------------------------------+ | 0c48af14-8bee-4ab4-9ad2-283734affa98 | None | None | | egress | IPv4 | None | 04098ac2-bff0-40bb-b17b-4e4c78823856 | | 6d014427-102d-46d2-8ec2-cb9f987f0354 | None | None | | ingress | IPv6 | 04098ac2-bff0-40bb-b17b-4e4c78823856 | 04098ac2-bff0-40bb-b17b-4e4c78823856 | | 75df898e-f9ea-47ab-a297-6e47fd362e0a | None | None | | egress | IPv6 | None | 04098ac2-bff0-40bb-b17b-4e4c78823856 | | bf347ce8-75f8-437c-931e-2119db614811 | None | None | | ingress | IPv4 | 04098ac2-bff0-40bb-b17b-4e4c78823856 | 04098ac2-bff0-40bb-b17b-4e4c78823856 | | d5e366cc-384f-4df6-a5d8-40a4992be812 | icmp | 0.0.0.0/0 | | ingress | IPv4 | None | 04098ac2-bff0-40bb-b17b-4e4c78823856 | | db603990-7ac0-4180-aa4e-7e2a8fb23168 | tcp | 0.0.0.0/0 | 22:22 | ingress | IPv4 | None | 04098ac2-bff0-40bb-b17b-4e4c78823856 | +--------------------------------------+-------------+-----------+------------+-----------+-----------+--------------------------------------+--------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:59,618] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:19:59,721] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:19:59,722] 127 INFO MainThread verify_fixtures.pre_alarms_session:: Default security group rules for tenant2 already configured to allow ping and ssh [2019-08-22 18:19:59,722] 152 INFO MainThread verify_fixtures.__get_alarms:: ====================== Setup Step 3: (session) Gathering system health info before test session begins. [2019-08-22 18:19:59,722] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:19:59,723] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:19:59,723] 301 DEBUG MainThread ssh.send :: Send 'fm --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne alarm-list --nowrap --uuid' [2019-08-22 18:20:01,555] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+----------+-----------------------------------------------------------------------+---------------------------------------+----------+----------------------------+ | UUID | Alarm ID | Reason Text | Entity ID | Severity | Time Stamp | +--------------------------------------+----------+-----------------------------------------------------------------------+---------------------------------------+----------+----------------------------+ | d0a71337-9899-4edf-bb52-531cc0a8c452 | 200.001 | compute-3 was administratively locked to take it out-of-service. | host=compute-3 | warning | 2019-08-22T17:28:13.502225 | | e4bc4da7-ed1c-4bc6-b105-c8d028b39441 | 100.114 | NTP address 2607:5300:203:2 is not a valid or a reachable NTP server. | host=controller-1.ntp=2607:5300:203:2 | minor | 2019-08-21T19:55:21.494958 | | 760ce36e-94b3-42a1-9e90-4ad1b7381279 | 100.114 | NTP address 2602:fde5:2a::1 is not a valid or a reachable NTP server. | host=controller-0.ntp=2602:fde5:2a::1 | minor | 2019-08-21T19:31:50.141475 | +--------------------------------------+----------+-----------------------------------------------------------------------+---------------------------------------+----------+----------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:01,555] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:01,658] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:01,659] 1211 DEBUG MainThread conftest.config_host_module:: Empty config host module [2019-08-22 18:20:01,659] 1205 DEBUG MainThread conftest.config_host_class:: Empty config host class [2019-08-22 18:20:01,660] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:01,660] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:01,660] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne hypervisor list' [2019-08-22 18:20:03,648] 423 DEBUG MainThread ssh.expect :: Output: +----+---------------------+-----------------+-----------------+-------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | +----+---------------------+-----------------+-----------------+-------+ | 5 | compute-2 | QEMU | 192.168.206.156 | up | | 8 | compute-0 | QEMU | 192.168.206.194 | up | | 11 | compute-3 | QEMU | 192.168.206.126 | down | | 14 | compute-1 | QEMU | 192.168.206.139 | up | +----+---------------------+-----------------+-----------------+-------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:03,648] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:03,751] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:03,752] 152 INFO MainThread verify_fixtures.__get_alarms:: ====================== Setup Step 4: (function) Gathering system health info before test function begins. [2019-08-22 18:20:03,753] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:03,753] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:03,753] 301 DEBUG MainThread ssh.send :: Send 'fm --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne alarm-list --nowrap --uuid' [2019-08-22 18:20:05,220] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+----------+-----------------------------------------------------------------------+---------------------------------------+----------+----------------------------+ | UUID | Alarm ID | Reason Text | Entity ID | Severity | Time Stamp | +--------------------------------------+----------+-----------------------------------------------------------------------+---------------------------------------+----------+----------------------------+ | d0a71337-9899-4edf-bb52-531cc0a8c452 | 200.001 | compute-3 was administratively locked to take it out-of-service. | host=compute-3 | warning | 2019-08-22T17:28:13.502225 | | e4bc4da7-ed1c-4bc6-b105-c8d028b39441 | 100.114 | NTP address 2607:5300:203:2 is not a valid or a reachable NTP server. | host=controller-1.ntp=2607:5300:203:2 | minor | 2019-08-21T19:55:21.494958 | | 760ce36e-94b3-42a1-9e90-4ad1b7381279 | 100.114 | NTP address 2602:fde5:2a::1 is not a valid or a reachable NTP server. | host=controller-0.ntp=2602:fde5:2a::1 | minor | 2019-08-21T19:31:50.141475 | +--------------------------------------+----------+-----------------------------------------------------------------------+---------------------------------------+----------+----------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:05,220] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:05,323] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:05,323] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:05,324] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:05,324] 301 DEBUG MainThread ssh.send :: Send 'kubectl get pod -o=wide --all-namespaces --field-selector=status.phase!=Running,status.phase!=Succeeded' [2019-08-22 18:20:05,568] 423 DEBUG MainThread ssh.expect :: Output: NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES kube-system ceph-pools-audit-1566498000-22n6d 0/1 ContainerCreating 0 3s controller-0 openstack cinder-volume-usage-audit-1566498000-kj872 0/1 Init:0/1 0 3s controller-1 openstack heat-engine-cleaner-1566498000-vlgzk 0/1 Init:0/1 0 3s controller-1 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:05,569] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:05,672] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:05,672] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:05,672] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:05,673] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne application-list' [2019-08-22 18:20:07,345] 423 DEBUG MainThread ssh.expect :: Output: +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | application | version | manifest name | manifest file | status | progress | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ | platform-integ-apps | 1.0-7 | platform-integration-manifest | manifest.yaml | applied | completed | | stx-openstack | 1.0-17-centos-stable- | armada-manifest | stx-openstack.yaml | applied | Application update from version 1.0-17-centos-stable-versioned to version 1.0-17-centos-stable-latest | | | latest | | | | completed. | | | | | | | | +---------------------+---------------------------+-------------------------------+--------------------+---------+-------------------------------------------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:07,345] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:07,448] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:07,449] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: stx-openstack [2019-08-22 18:20:07,452] 266 DEBUG MainThread conftest.testcase_log:: +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Test steps started for: testcases/stress/nova/test_migrate.py::test_migrate_stress[volume-1000] [2019-08-22 18:20:07,452] 150 INFO MainThread test_migrate.test_migrate_stress:: ====================== Test Step 1: Launch a VM from volume [2019-08-22 18:20:07,452] 810 INFO MainThread vm_helper.boot_vm :: Processing boot_vm args... [2019-08-22 18:20:07,452] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:07,452] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:07,453] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne keypair list' [2019-08-22 18:20:09,276] 423 DEBUG MainThread ssh.expect :: Output: +-----------------+-------------------------------------------------+ | Name | Fingerprint | +-----------------+-------------------------------------------------+ | keypair-tenant1 | 9d:f7:59:61:7d:c5:07:c5:5d:bb:74:ec:fb:7a:d4:70 | +-----------------+-------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:09,276] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:09,379] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:09,379] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:09,379] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:09,379] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne flavor list' [2019-08-22 18:20:11,248] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+------------------------+-------+------+-----------+-------+-----------+ | ID | Name | RAM | Disk | Ephemeral | VCPUs | Is Public | +--------------------------------------+------------------------+-------+------+-----------+-------+-----------+ | 02bdd558-7d29-4228-8eb0-746eb11ac532 | dpdk-medium-ded-remote | 1024 | 1 | 0 | 2 | True | | 0d55d069-cc7b-4033-85d2-f7c6372cabf6 | testhuge | 1024 | 10 | 0 | 4 | True | | 1383ceb4-aa2c-4d6e-a71f-6e810056d077 | small.float | 512 | 2 | 0 | 1 | True | | 4792a546-85c9-4c6b-9721-d15f6bc4d280 | m1.medium | 4096 | 40 | 0 | 2 | True | | 56744dff-79e4-4164-9150-316b81a1bc6f | m1.tiny | 512 | 1 | 0 | 1 | True | | 56eee7f5-8330-4add-9fb9-4431564b5be7 | test1 | 1024 | 10 | 0 | 4 | True | | 7045317c-ace2-4d78-ab39-f3ac6942798d | m1.small | 2048 | 20 | 0 | 1 | True | | 8cd9c74b-7d58-4ebf-9ad6-66458c15235b | small | 512 | 2 | 0 | 1 | True | | 9c19b85c-d625-490c-a338-3573252d923f | small-ded-remote | 512 | 1 | 0 | 1 | True | | adfed7ad-659f-4a46-9f78-d7fddcb94449 | small-float-remote | 512 | 1 | 0 | 1 | True | | b8f20ebe-4c60-4d9d-81e7-f30332e01049 | flavor-default-size2 | 1024 | 2 | 0 | 1 | True | | c5b85c32-cd19-4b8c-86c5-ee40f0568634 | m1.xlarge | 16384 | 160 | 0 | 8 | True | | c8557adf-b069-43c8-9db0-2af517216234 | m1.large | 8192 | 80 | 0 | 4 | True | | e0b0d101-67d7-4ca7-83ba-83649d2f9e57 | medium.dpdk | 1024 | 2 | 0 | 2 | True | +--------------------------------------+------------------------+-------+------+-----------+-------+-----------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:11,248] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:11,351] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:11,352] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:11,352] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:11,352] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne network list' [2019-08-22 18:20:13,286] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------+ | ID | Name | Subnets | +--------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------+ | 1fce30a6-3cf9-4631-b799-090aa8fc4aa9 | internal0-net1 | d0ae4b3f-48da-4612-ac81-899b4983c004 | | 30fed34d-e58d-47b7-902f-7566ccc264a9 | tenant1-net5 | 86923b3b-d82a-4fb3-9022-8062261fa711 | | 31034e2e-9cf2-4500-ace2-49235a03499b | external-net0 | d074ae1e-7fcb-465c-b8f4-96641156ba84 | | 80cffa51-0a10-4495-aa3c-af1130c23b4c | internal0-net1-1 | 644f1eed-a037-47c4-9472-0e5bb3a2ff49 | | 81139da8-13cb-44dc-9f76-3b0139f0a689 | tenant1-net3 | 9c62b71f-f93f-4a2d-8af8-0c3706e43b94 | | 913bc604-609e-472f-ad7d-1c9afd680e13 | tenant1-net1 | 17e893b6-4575-4785-8bf2-5406ff80d34d | | 98df63a8-5d96-44e3-b9db-9a50d3950f8d | internal0-net0-1 | fb8f2e5f-336d-4b58-a564-0b7a6268d6f3 | | 9bbd6516-6f6a-4a5b-bf64-cb01d5af3a83 | tenant1-net4 | 3f82b2e6-f46b-4b92-b3b7-5e035f7cab51 | | a1798c88-b7ce-460d-863b-2f09eb8c11e9 | internal0-net0-2 | 56ee0cbc-962e-4f46-a216-4bda76b181ce | | b84aa969-5813-41c4-9123-e2ef80ab35d3 | internal0-net0-3 | 41be0471-7241-4102-99e9-14dc53912f81 | | dcfc9285-916f-41b5-b5d4-cc9dcdc1bd99 | tenant1-mgmt-net | 065469c8-d801-47fb-a4f6-5fa45dbb2df7, 089f4c0e-3b8e-4d02-9dc3-195f410cd99c, 0ceac9b3-8c6a-4698-8cd0-bc31ec320822 | | e7151f9f-7f51-4206-9532-c896aa2614af | tenant1-net2 | d0083e89-a609-4e3d-a07c-c621a04cc7e5 | | f9010617-183e-4c6c-a4a9-659238bf8982 | internal0-net0 | ff3ed495-7e94-4e33-bf24-00a31a96b58f | | f9a525cc-3e5b-4654-a401-c58d9969c1ab | tenant1-net0 | a2204b0f-fcb7-43a0-ac67-c151e75fef38 | +--------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:13,286] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:13,389] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:13,390] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:13,390] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:13,390] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne network list' [2019-08-22 18:20:15,337] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------+ | ID | Name | Subnets | +--------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------+ | 1fce30a6-3cf9-4631-b799-090aa8fc4aa9 | internal0-net1 | d0ae4b3f-48da-4612-ac81-899b4983c004 | | 30fed34d-e58d-47b7-902f-7566ccc264a9 | tenant1-net5 | 86923b3b-d82a-4fb3-9022-8062261fa711 | | 31034e2e-9cf2-4500-ace2-49235a03499b | external-net0 | d074ae1e-7fcb-465c-b8f4-96641156ba84 | | 80cffa51-0a10-4495-aa3c-af1130c23b4c | internal0-net1-1 | 644f1eed-a037-47c4-9472-0e5bb3a2ff49 | | 81139da8-13cb-44dc-9f76-3b0139f0a689 | tenant1-net3 | 9c62b71f-f93f-4a2d-8af8-0c3706e43b94 | | 913bc604-609e-472f-ad7d-1c9afd680e13 | tenant1-net1 | 17e893b6-4575-4785-8bf2-5406ff80d34d | | 98df63a8-5d96-44e3-b9db-9a50d3950f8d | internal0-net0-1 | fb8f2e5f-336d-4b58-a564-0b7a6268d6f3 | | 9bbd6516-6f6a-4a5b-bf64-cb01d5af3a83 | tenant1-net4 | 3f82b2e6-f46b-4b92-b3b7-5e035f7cab51 | | a1798c88-b7ce-460d-863b-2f09eb8c11e9 | internal0-net0-2 | 56ee0cbc-962e-4f46-a216-4bda76b181ce | | b84aa969-5813-41c4-9123-e2ef80ab35d3 | internal0-net0-3 | 41be0471-7241-4102-99e9-14dc53912f81 | | dcfc9285-916f-41b5-b5d4-cc9dcdc1bd99 | tenant1-mgmt-net | 065469c8-d801-47fb-a4f6-5fa45dbb2df7, 089f4c0e-3b8e-4d02-9dc3-195f410cd99c, 0ceac9b3-8c6a-4698-8cd0-bc31ec320822 | | e7151f9f-7f51-4206-9532-c896aa2614af | tenant1-net2 | d0083e89-a609-4e3d-a07c-c621a04cc7e5 | | f9010617-183e-4c6c-a4a9-659238bf8982 | internal0-net0 | ff3ed495-7e94-4e33-bf24-00a31a96b58f | | f9a525cc-3e5b-4654-a401-c58d9969c1ab | tenant1-net0 | a2204b0f-fcb7-43a0-ac67-c151e75fef38 | +--------------------------------------+------------------+------------------------------------------------------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:15,337] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:15,440] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:15,440] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:15,441] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:15,441] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne image list' [2019-08-22 18:20:17,239] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+------------------+--------+ | ID | Name | Status | +--------------------------------------+------------------+--------+ | fd57e28e-6ea4-4380-ba0d-32d0f787c152 | tis-centos-guest | active | +--------------------------------------+------------------+--------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:17,239] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:17,342] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:17,342] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:17,343] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:17,343] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne volume list --a --long' [2019-08-22 18:20:19,354] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:19,354] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:19,457] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:19,458] 225 DEBUG MainThread table_parser.table :: No table returned [2019-08-22 18:20:19,458] 289 INFO MainThread cinder_helper.create_volume:: Creating Volume with args: --bootable --image=fd57e28e-6ea4-4380-ba0d-32d0f787c152 --size=2 vol-tenant1-volume-stress-1-1 [2019-08-22 18:20:19,458] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:19,458] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:19,458] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne volume create --bootable --image=fd57e28e-6ea4-4380-ba0d-32d0f787c152 --size=2 vol-tenant1-volume-stress-1-1' [2019-08-22 18:20:21,665] 423 DEBUG MainThread ssh.expect :: Output: +---------------------+--------------------------------------+ | Field | Value | +---------------------+--------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | false | | consistencygroup_id | None | | created_at | 2019-08-22T18:20:21.000000 | | description | None | | encrypted | False | | id | 753449a4-494b-42f5-aa19-768f8fc0d60e | | multiattach | False | | name | vol-tenant1-volume-stress-1-1 | | properties | | | replication_status | None | | size | 2 | | snapshot_id | None | | source_volid | None | | status | creating | | type | ceph-store | | updated_at | None | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | +---------------------+--------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:21,666] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:21,769] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:21,769] 301 INFO MainThread cinder_helper.create_volume:: Post action check started for create volume. [2019-08-22 18:20:21,770] 397 INFO MainThread cinder_helper.__wait_for_vol_status:: Waiting for cinder volume 753449a4-494b-42f5-aa19-768f8fc0d60e status: ('available',) [2019-08-22 18:20:21,770] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:21,770] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:21,770] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne volume show 753449a4-494b-42f5-aa19-768f8fc0d60e' [2019-08-22 18:20:23,604] 423 DEBUG MainThread ssh.expect :: Output: +------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | attachments | [] | | availability_zone | nova | | bootable | true | | consistencygroup_id | None | | created_at | 2019-08-22T18:20:21.000000 | | description | None | | encrypted | False | | id | 753449a4-494b-42f5-aa19-768f8fc0d60e | | multiattach | False | | name | vol-tenant1-volume-stress-1-1 | | os-vol-tenant-attr:tenant_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | replication_status | None | | size | 2 | | snapshot_id | None | | source_volid | None | | status | available | | type | ceph-store | | updated_at | 2019-08-22T18:20:22.000000 | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volume_image_metadata | {u'container_format': u'bare', u'min_ram': u'0', u'disk_format': u'raw', u'image_name': u'tis-centos-guest', u'image_id': u'fd57e28e-6ea4-4380-ba0d-32d0f787c152', u'checksum': u'b0e308015909fd2ac4c7472e19f149de', u'min_disk': u'0', u'size': u'1611581440'} | +------------------------------+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:23,604] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:23,707] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:23,708] 406 INFO MainThread cinder_helper.__wait_for_vol_status:: Volume 753449a4-494b-42f5-aa19-768f8fc0d60e is in available state [2019-08-22 18:20:23,708] 307 INFO MainThread cinder_helper.create_volume:: Volume is created and in available state: 753449a4-494b-42f5-aa19-768f8fc0d60e [2019-08-22 18:20:23,708] 957 INFO MainThread vm_helper.boot_vm :: Booting VM tenant1-volume-stress-1 with args: --flavor=b8f20ebe-4c60-4d9d-81e7-f30332e01049 --boot-volume=753449a4-494b-42f5-aa19-768f8fc0d60e --key-name=keypair-tenant1 --poll --nic net-id=dcfc9285-916f-41b5-b5d4-cc9dcdc1bd99 --nic net-id=30fed34d-e58d-47b7-902f-7566ccc264a9 tenant1-volume-stress-1 [2019-08-22 18:20:23,709] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:23,709] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:23,709] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne boot --flavor=b8f20ebe-4c60-4d9d-81e7-f30332e01049 --boot-volume=753449a4-494b-42f5-aa19-768f8fc0d60e --key-name=keypair-tenant1 --poll --nic net-id=dcfc9285-916f-41b5-b5d4-cc9dcdc1bd99 --nic net-id=30fed34d-e58d-47b7-902f-7566ccc264a9 tenant1-volume-stress-1' [2019-08-22 18:20:47,614] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------------------------------+ | Property | Value | +--------------------------------------+-------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | | | OS-EXT-STS:power_state | 0 | | OS-EXT-STS:task_state | scheduling | | OS-EXT-STS:vm_state | building | | OS-SRV-USG:launched_at | - | | OS-SRV-USG:terminated_at | - | | accessIPv4 | | | accessIPv6 | | | adminPass | 6zUngR28ipJi | | config_drive | | | created | 2019-08-22T18:20:26Z | | description | - | | flavor:disk | 2 | | flavor:ephemeral | 0 | | flavor:extra_specs | {"hw:mem_page_size": "large"} | | flavor:original_name | flavor-default-size2 | | flavor:ram | 1024 | | flavor:swap | 0 | | flavor:vcpus | 1 | | hostId | | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | Attempt to boot from volume - no image supplied | | key_name | keypair-tenant1 | | locked | False | | metadata | {} | | name | tenant1-volume-stress-1 | | os-extended-volumes:volumes_attached | [] | | progress | 0 | | security_groups | default | | server_groups | [] | | status | BUILD | | tags | [] | | tenant_id | 71e4950fb29c40d8ac13742b4990dfad | | trusted_image_certificates | - | | updated | 2019-08-22T18:20:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | +--------------------------------------+-------------------------------------------------+ Server building... 0% complete Server building... 0% complete Server building... 0% complete Server building... 0% complete Server building... 100% complete Finished [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:47,615] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:47,718] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:47,718] 128 DEBUG MainThread table_parser.__table :: skipping invalid table line: [2019-08-22 18:20:47,718] 128 DEBUG MainThread table_parser.__table :: skipping invalid table line: Server building... 0% complete Server building... 0% complete Server building... 0% complete Server building... 0% complete Server building... 100% complete [2019-08-22 18:20:47,719] 128 DEBUG MainThread table_parser.__table :: skipping invalid table line: Finished [2019-08-22 18:20:47,719] 983 INFO MainThread vm_helper.boot_vm :: Post action check... [2019-08-22 18:20:47,719] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:47,719] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:47,719] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'tenant1' --os-password 'Li69nux*' --os-project-name tenant1 --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:20:49,810] 423 DEBUG MainThread ssh.expect :: Output: +-----------------------------+-------------------------------------------------------------+ | Field | Value | +-----------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:20:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-----------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:49,810] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:49,913] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:49,914] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:20:49,914] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached ACTIVE [2019-08-22 18:20:49,914] 1006 INFO MainThread vm_helper.boot_vm :: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c is booted successfully. [2019-08-22 18:20:49,914] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:20:49,914] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:20:49,915] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:49,915] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:20:52,108] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:52,108] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:20:52,211] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:20:52,212] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:20:52,212] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:20:52,212] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:20:52,213] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:20:52,213] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:21:04,317] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.586 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 1 received, 66% packet loss, time 1999ms rtt min/avg/max/mdev = 0.586/0.586/0.586/0.000 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:21:04,317] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:04,419] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:21:04,419] 2558 WARNING MainThread network_helper.ping_server:: Some packets dropped when ping from 128.224.186.181 ssh session to 192.168.104.70. Packet loss rate: 66% [2019-08-22 18:21:04,420] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 66} [2019-08-22 18:21:09,425] 157 INFO MainThread test_migrate.test_migrate_stress:: ====================== Test Step 2: Attempt to block migration on boot-from-volume VM and ensure if fails [2019-08-22 18:21:09,425] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:09,426] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:09,426] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:11,477] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:20:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:11,477] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:11,580] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:11,580] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:11,580] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:11,581] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:11,581] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:13,619] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:20:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:13,619] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:13,723] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:13,723] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:13,723] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:21:13,723] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration --block-migrate 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:21:13,724] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:13,724] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:13,724] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration --block-migrate 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:15,855] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:15,855] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:15,958] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:15,958] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:21:15,958] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:15,959] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:15,959] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:18,317] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:18,317] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:18,420] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:18,420] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:21,424] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:21,424] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:21,424] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:23,490] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:23,491] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:23,593] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:23,594] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:26,597] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:26,597] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:26,597] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:28,615] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:28,616] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:28,719] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:28,719] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:31,723] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:31,723] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:31,723] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:33,756] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:33,757] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:33,859] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:33,860] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:36,863] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:36,863] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:36,864] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:38,880] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:38,880] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:38,983] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:38,983] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:41,987] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:41,987] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:41,987] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:44,310] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:44,310] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:44,413] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:44,413] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:47,417] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:47,417] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:47,417] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:49,513] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:49,513] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:49,616] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:49,617] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:52,619] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:52,619] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:52,619] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:54,637] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:54,638] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:21:54,740] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:54,741] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:21:57,744] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:21:57,744] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:21:57,745] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:21:59,943] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:21:59,943] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:00,046] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:00,047] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:03,050] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:03,050] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:03,050] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:22:05,107] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:05,107] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:05,210] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:05,210] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:08,214] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:08,214] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:08,214] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:22:10,429] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:10,429] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:10,532] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:10,532] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:13,536] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:13,536] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:13,536] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:22:15,570] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:15,570] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:15,673] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:15,674] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:18,675] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:18,675] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:18,675] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:22:20,698] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:20,699] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:20,801] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:20,802] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:20,802] 1771 WARNING MainThread vm_helper.wait_for_vm_status:: Timed out waiting for vm status: ['MIGRATING']. Actual vm status: ACTIVE [2019-08-22 18:22:20,802] 1306 WARNING MainThread vm_helper.live_migrate_vm:: VM did not reach MIGRATING state after triggering live-migration [2019-08-22 18:22:20,802] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:22:22,805] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:22,805] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:22,805] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:22:24,847] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:24,847] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:24,950] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:24,951] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:24,951] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:22:24,951] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:24,951] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:24,951] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:22:26,983] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:26,983] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:27,086] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:27,086] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:30,090] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:30,090] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:30,090] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:22:32,100] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:32,100] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:32,204] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:32,204] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:35,207] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:35,208] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:35,208] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:22:37,255] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:37,255] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:37,359] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:37,359] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:40,362] 1348 WARNING MainThread vm_helper.live_migrate_vm:: Live migration of vm 698586e7-0ff0-4f52-a9f5-76ff1693539c failed. Checking if this is expected failure... [2019-08-22 18:22:40,363] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:40,363] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:40,363] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:22:42,536] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:42,536] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:42,639] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:42,639] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:42,639] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:42,639] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:22:44,690] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:44,690] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:44,793] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:44,794] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:22:44,794] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:44,794] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne volume show 753449a4-494b-42f5-aa19-768f8fc0d60e' [2019-08-22 18:22:46,595] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | Field | Value | +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | attachments | [{u'server_id': u'698586e7-0ff0-4f52-a9f5-76ff1693539c', u'attachment_id': u'873a3de3-ca12-41f2-95e1-9e18f89ef035', u'attached_at': u'2019-08-22T18:20:28.000000', u'host_name': u'compute-1', u'volume_id': u'753449a4-494b-42f5-aa19-768f8fc0d60e', u'device': u'/dev/vda', u'id': u'753449a4-494b-42f5-aa19-768f8fc0d60e'}] | | availability_zone | nova | | bootable | true | | consistencygroup_id | None | | created_at | 2019-08-22T18:20:21.000000 | | description | None | | encrypted | False | | id | 753449a4-494b-42f5-aa19-768f8fc0d60e | | migration_status | None | | multiattach | False | | name | vol-tenant1-volume-stress-1-1 | | os-vol-host-attr:host | cinder-volume-worker@ceph-store#ceph-store | | os-vol-mig-status-attr:migstat | None | | os-vol-mig-status-attr:name_id | None | | os-vol-tenant-attr:tenant_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | replication_status | None | | size | 2 | | snapshot_id | None | | source_volid | None | | status | in-use | | type | ceph-store | | updated_at | 2019-08-22T18:20:28.000000 | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volume_image_metadata | {u'container_format': u'bare', u'min_ram': u'0', u'disk_format': u'raw', u'image_name': u'tis-centos-guest', u'image_id': u'fd57e28e-6ea4-4380-ba0d-32d0f787c152', u'checksum': u'b0e308015909fd2ac4c7472e19f149de', u'min_disk': u'0', u'size': u'1611581440'} | +--------------------------------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:46,595] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:46,698] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:46,699] 338 INFO MainThread cinder_helper.get_volume_show_values:: val: {u'container_format': u'bare', u'min_ram': u'0', u'disk_format': u'raw', u'image_name': u'tis-centos-guest', u'image_id': u'fd57e28e-6ea4-4380-ba0d-32d0f787c152', u'checksum': u'b0e308015909fd2ac4c7472e19f149de', u'min_disk': u'0', u'size': u'1611581440'} [2019-08-22 18:22:46,699] 6824 INFO MainThread vm_helper._get_boot_info:: VM booted from volume. [2019-08-22 18:22:46,699] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:46,699] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne flavor show b8f20ebe-4c60-4d9d-81e7-f30332e01049' [2019-08-22 18:22:48,551] 423 DEBUG MainThread ssh.expect :: Output: +----------------------------+--------------------------------------+ | Field | Value | +----------------------------+--------------------------------------+ | OS-FLV-DISABLED:disabled | False | | OS-FLV-EXT-DATA:ephemeral | 0 | | access_project_ids | None | | disk | 2 | | id | b8f20ebe-4c60-4d9d-81e7-f30332e01049 | | name | flavor-default-size2 | | os-flavor-access:is_public | True | | properties | hw:mem_page_size='large' | | ram | 1024 | | rxtx_factor | 1.0 | | swap | | | vcpus | 1 | +----------------------------+--------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:48,551] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:48,654] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:48,655] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:48,655] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:48,655] 301 DEBUG MainThread ssh.send :: Send 'kubectl get pod -o=wide -n=openstack -l=application=nova,component=compute' [2019-08-22 18:22:48,899] 423 DEBUG MainThread ssh.expect :: Output: NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nova-compute-compute-0-5621f953-x6xxb 2/2 Running 0 172m 192.168.204.68 compute-0 nova-compute-compute-1-532206f8-wn47c 2/2 Running 0 172m 192.168.204.77 compute-1 nova-compute-compute-2-cad8c828-tkhcq 2/2 Running 0 172m 192.168.204.58 compute-2 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:48,900] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:49,002] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:49,003] 1090 INFO MainThread kube_helper.get_openstack_configs:: Getting /etc/nova/nova.conf {'libvirt': 'images_type'} values from openstack pods: ['nova-compute-compute-1-532206f8-wn47c'] [2019-08-22 18:22:49,003] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:49,003] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:49,003] 301 DEBUG MainThread ssh.send :: Send 'kubectl exec nova-compute-compute-1-532206f8-wn47c -n openstack -- cat /etc/nova/nova.conf | awk '{ if ( $1 ~ /^\[.*\]/ || $1 ~ /^images_type/) print }' | grep --color=never --group-separator='' -B 1 -v '\[.*\]'' [2019-08-22 18:22:49,456] 423 DEBUG MainThread ssh.expect :: Output: Defaulting container name to nova-compute. Use 'kubectl describe pod/nova-compute-compute-1-532206f8-wn47c -n openstack' to see all of the containers in this pod. [libvirt] images_type = default [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:49,457] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:49,560] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:49,560] 2698 INFO MainThread host_helper.get_host_instance_backing:: compute-1 instance backing: local_image [2019-08-22 18:22:49,561] 1386 WARNING MainThread vm_helper._is_live_migration_allowed:: Live block migration is not supported for boot-from-volume vm with local_image storage [2019-08-22 18:22:49,561] 1363 DEBUG MainThread vm_helper.live_migrate_vm:: System does not allow live migrating vm 698586e7-0ff0-4f52-a9f5-76ff1693539c as expected. [2019-08-22 18:22:49,561] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:22:49,561] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:49,561] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:49,561] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:22:51,779] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:51,779] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:51,882] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:22:51,882] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:22:51,883] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:22:51,883] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:22:51,883] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:51,883] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:22:53,987] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.16 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.533 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.303 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.303/0.667/1.165/0.364 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:22:53,987] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:22:54,089] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:22:54,089] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:22:54,089] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:22:59,095] 162 INFO MainThread test_migrate.test_migrate_stress:: ====================== Test Step 3: Live migrate and ping vm 1000 times [2019-08-22 18:22:59,095] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter1 [2019-08-22 18:22:59,095] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:22:59,095] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:22:59,096] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:01,102] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:01,102] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:01,205] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:01,206] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:01,206] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:01,206] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:01,206] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:03,215] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:21:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:03,216] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:03,318] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:03,319] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:03,319] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:23:03,319] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:23:03,319] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:03,320] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:03,320] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:05,229] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:05,229] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:05,332] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:05,332] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:23:05,333] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:05,333] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:05,333] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:07,513] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:23:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:07,513] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:07,616] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:07,617] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:07,617] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:23:07,617] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:23:09,619] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:09,620] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:09,620] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:11,662] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:23:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:11,662] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:11,765] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:11,766] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:13,768] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:13,768] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:13,769] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:15,769] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:23:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:15,769] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:15,873] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:15,873] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:17,875] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:17,876] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:17,876] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:19,959] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:23:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:19,959] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:20,062] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:20,062] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:22,065] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:22,065] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:22,065] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:24,220] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:23:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:24,221] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:24,324] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:24,324] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:26,327] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:26,327] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:26,327] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:28,357] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:23:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:28,357] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:28,460] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:28,461] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:30,462] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:30,462] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:30,462] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:32,461] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:23:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:32,461] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:32,564] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:32,565] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:32,565] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:23:32,565] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:32,565] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:32,565] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:34,607] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:23:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:34,608] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:34,711] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:34,711] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:34,711] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-0 [2019-08-22 18:23:34,712] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:23:34,712] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:34,712] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:34,712] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:23:36,929] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:36,929] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:37,032] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:37,033] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:23:37,033] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:23:37,033] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:23:37,033] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:37,033] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:23:39,138] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.05 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.434 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.236 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.236/0.574/1.054/0.349 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:23:39,138] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:39,240] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:23:39,240] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:23:39,240] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:23:44,243] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter2 [2019-08-22 18:23:44,243] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:44,243] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:44,243] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:46,262] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:23:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:46,262] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:46,365] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:46,365] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:46,366] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:46,366] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:46,366] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:48,369] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:23:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:48,369] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:48,472] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:48,472] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:48,472] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:23:48,472] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:23:48,473] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:48,473] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:48,473] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:50,341] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:50,342] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:50,445] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:50,445] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:23:50,445] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:50,446] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:50,446] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:52,451] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:23:50Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:52,451] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:52,554] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:52,555] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:52,555] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:23:52,556] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:23:54,558] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:54,558] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:54,559] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:23:56,642] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:23:50Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:56,642] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:23:56,745] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:23:56,746] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:23:58,748] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:23:58,748] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:23:58,749] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:00,823] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:23:50Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:00,824] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:00,927] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:00,927] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:02,929] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:02,929] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:02,929] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:05,101] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:24:02Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:05,101] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:05,204] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:05,205] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:07,207] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:07,207] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:07,208] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:09,261] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:24:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:09,261] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:09,364] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:09,365] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:11,366] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:11,366] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:11,366] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:13,373] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:24:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:13,373] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:13,476] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:13,476] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:15,478] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:15,478] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:15,478] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:17,568] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:24:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:17,568] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:17,671] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:17,672] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:19,674] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:19,675] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:19,675] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:21,718] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:24:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:21,718] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:21,821] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:21,822] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:21,822] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:24:21,822] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:21,822] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:21,823] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:23,884] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:24:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:23,884] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:23,987] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:23,987] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:23,988] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 18:24:23,988] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:24:23,988] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:23,988] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:23,988] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:24:26,183] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:26,183] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:26,286] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:26,286] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:24:26,287] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:24:26,287] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:24:26,287] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:26,287] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:24:38,407] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2016ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:24:38,407] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:38,509] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:24:38,509] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:24:41,512] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:24:41,513] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:24:41,513] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:24:41,513] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:41,513] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:41,514] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:24:43,912] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:43,912] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:44,015] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:44,016] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:24:44,016] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:24:44,016] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:24:44,016] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:44,016] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:24:46,121] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.964 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.608 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.350 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.350/0.640/0.964/0.253 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:24:46,122] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:46,223] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:24:46,223] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:24:46,224] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:24:51,229] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter3 [2019-08-22 18:24:51,229] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:51,229] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:51,229] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:53,242] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:24:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:53,242] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:53,345] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:53,346] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:53,346] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:53,346] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:53,346] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:55,470] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:24:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:55,470] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:55,574] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:55,574] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:55,574] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:24:55,575] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:24:55,575] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:55,575] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:55,575] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:57,573] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:57,573] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:57,676] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:57,677] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:24:57,677] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:24:57,677] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:24:57,677] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:24:59,674] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:24:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:59,674] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:24:59,777] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:24:59,777] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:24:59,777] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:24:59,778] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:25:01,779] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:01,779] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:01,779] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:04,305] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:24:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:04,305] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:04,408] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:04,408] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:06,410] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:06,410] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:06,410] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:08,422] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:24:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:08,423] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:08,525] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:08,526] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:10,528] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:10,528] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:10,529] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:12,552] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:25:11Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:12,552] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:12,656] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:12,656] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:14,658] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:14,659] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:14,659] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:16,677] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:25:11Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:16,677] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:16,780] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:16,781] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:18,783] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:18,783] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:18,783] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:20,818] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:25:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:20,818] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:20,921] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:20,922] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:22,924] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:22,924] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:22,924] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:25,360] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:25:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:25,360] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:25,463] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:25,464] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:27,466] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:27,467] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:27,467] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:29,534] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:25:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:29,534] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:29,638] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:29,638] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:29,638] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:25:29,638] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:29,638] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:29,639] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:31,718] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:25:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:31,718] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:31,821] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:31,822] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:31,822] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 18:25:31,822] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:25:31,822] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:31,822] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:31,823] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:25:34,009] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:34,009] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:34,112] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:34,113] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:25:34,113] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:25:34,113] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:25:34,113] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:34,113] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:25:36,217] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.617 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.730 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.677 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.617/0.674/0.730/0.055 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:25:36,218] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:36,320] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:25:36,320] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:25:36,320] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:25:41,325] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter4 [2019-08-22 18:25:41,326] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:41,326] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:41,326] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:43,340] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:25:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:43,341] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:43,444] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:43,444] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:43,444] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:43,445] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:43,445] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:45,458] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:25:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:45,458] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:45,561] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:45,562] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:45,562] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:25:45,562] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:25:45,562] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:45,562] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:45,562] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:47,622] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:47,622] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:47,725] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:47,726] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:25:47,726] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:47,726] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:47,726] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:49,877] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:25:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:49,877] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:49,980] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:49,981] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:49,981] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:25:49,981] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:25:51,983] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:51,984] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:51,984] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:54,484] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:25:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:54,485] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:54,588] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:54,588] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:25:56,590] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:25:56,590] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:25:56,591] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:25:58,578] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:25:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:58,578] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:25:58,680] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:25:58,681] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:00,683] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:00,683] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:00,684] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:02,684] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:25:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:02,684] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:02,788] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:02,788] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:04,789] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:04,789] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:04,789] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:07,103] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:07,103] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:07,206] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:07,207] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:09,209] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:09,209] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:09,209] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:11,266] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:11,266] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:11,369] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:11,370] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:13,372] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:13,372] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:13,373] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:15,560] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:15,560] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:15,663] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:15,664] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:17,666] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:17,666] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:17,666] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:19,708] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:26:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:19,708] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:19,812] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:19,812] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:19,813] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:26:19,813] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:19,813] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:19,813] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:21,907] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:26:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:21,907] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:22,010] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:22,011] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:22,011] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 18:26:22,011] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:26:22,011] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:22,011] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:22,012] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:26:24,431] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:24,432] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:24,535] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:24,535] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:26:24,535] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:26:24,535] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:26:24,536] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:24,536] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:26:26,641] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.33 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.449 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.350 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.350/0.712/1.338/0.444 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:26:26,641] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:26,743] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:26:26,743] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:26:26,743] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:26:31,749] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter5 [2019-08-22 18:26:31,749] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:31,749] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:31,749] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:33,811] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:26:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:33,812] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:33,914] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:33,915] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:33,915] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:33,915] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:33,915] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:35,944] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:26:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:35,945] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:36,047] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:36,048] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:36,048] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:26:36,048] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:26:36,048] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:36,048] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:36,048] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:37,930] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:37,930] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:38,033] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:38,033] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:26:38,033] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:38,033] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:38,034] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:40,468] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:40,468] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:40,571] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:40,572] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:40,572] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:26:40,572] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:26:42,575] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:42,575] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:42,575] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:44,748] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:44,748] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:44,851] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:44,852] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:46,854] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:46,854] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:46,854] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:48,885] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:48,885] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:48,989] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:48,989] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:50,991] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:50,991] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:50,992] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:53,078] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:53,078] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:53,181] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:53,181] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:55,184] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:55,184] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:55,184] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:26:57,294] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:57,294] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:26:57,397] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:26:57,397] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:26:59,400] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:26:59,400] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:26:59,400] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:01,474] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:01,475] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:01,578] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:01,578] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:03,580] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:03,581] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:03,581] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:06,004] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:26:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:06,004] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:06,107] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:06,107] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:08,109] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:08,109] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:08,110] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:10,166] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:27:09Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:10,166] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:10,269] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:10,269] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:10,269] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:27:10,270] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:10,270] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:10,270] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:12,645] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:27:09Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:12,646] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:12,749] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:12,750] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:12,750] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 18:27:12,750] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:27:12,750] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:12,750] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:12,750] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:27:14,977] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:14,977] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:15,080] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:15,081] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:27:15,081] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:27:15,081] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:27:15,081] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:15,081] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:27:17,186] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=2.58 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.289 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.283 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.283/1.053/2.589/1.086 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:27:17,186] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:17,287] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:27:17,288] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:27:17,288] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:27:22,293] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter6 [2019-08-22 18:27:22,294] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:22,294] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:22,294] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:24,333] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:27:09Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:24,333] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:24,436] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:24,436] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:24,437] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:24,437] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:24,437] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:26,468] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:27:09Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:26,468] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:26,572] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:26,572] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:26,572] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:27:26,572] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:27:26,573] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:26,573] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:26,573] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:28,434] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:28,434] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:28,537] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:28,537] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:27:28,538] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:28,538] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:28,538] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:30,553] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:27:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:30,553] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:30,656] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:30,656] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:30,656] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:27:30,656] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:27:32,659] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:32,659] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:32,659] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:34,910] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:27:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:34,910] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:35,013] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:35,014] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:37,016] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:37,016] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:37,017] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:39,241] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:27:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:39,242] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:39,345] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:39,345] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:41,347] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:41,348] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:41,348] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:43,422] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:27:39Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:43,423] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:43,525] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:43,526] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:45,527] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:45,527] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:45,527] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:47,544] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:27:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:47,544] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:47,647] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:47,647] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:49,650] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:49,650] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:49,650] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:51,680] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:27:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:51,680] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:51,783] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:51,783] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:53,785] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:53,786] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:53,786] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:55,810] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:27:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:55,810] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:27:55,914] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:55,914] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:27:57,917] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:27:57,917] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:27:57,917] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:27:59,955] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:27:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:27:59,955] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:00,058] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:00,058] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:00,059] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:28:00,059] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:00,059] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:00,059] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:02,109] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:27:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:02,109] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:02,212] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:02,213] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:02,213] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 18:28:02,213] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:28:02,214] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:02,214] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:02,214] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:28:04,371] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:04,371] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:04,474] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:04,475] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:28:04,475] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:28:04,475] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:28:04,475] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:04,475] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:28:06,582] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.30 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.951 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.609 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.609/0.954/1.303/0.284 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:28:06,583] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:06,685] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:28:06,685] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:28:06,685] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:28:11,691] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter7 [2019-08-22 18:28:11,691] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:11,691] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:11,691] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:13,725] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:27:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:13,726] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:13,828] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:13,829] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:13,829] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:13,829] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:13,830] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:15,871] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:27:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:15,871] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:15,974] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:15,975] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:15,975] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:28:15,975] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:28:15,975] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:15,975] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:15,976] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:17,837] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:17,837] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:17,940] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:17,940] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:28:17,940] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:17,941] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:17,941] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:20,001] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:28:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:20,002] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:20,105] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:20,105] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:20,105] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:28:20,105] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:28:22,108] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:22,108] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:22,108] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:24,152] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:28:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:24,152] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:24,255] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:24,256] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:26,258] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:26,258] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:26,258] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:28,265] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:28:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:28,265] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:28,369] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:28,369] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:30,371] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:30,371] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:30,371] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:32,416] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:28:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:32,416] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:32,520] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:32,520] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:34,522] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:34,523] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:34,523] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:36,568] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:28:33Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:36,569] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:36,671] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:36,672] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:38,674] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:38,674] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:38,675] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:40,737] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:28:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:40,737] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:40,841] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:40,841] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:42,843] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:42,844] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:42,844] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:44,918] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:28:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:44,918] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:45,021] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:45,022] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:47,024] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:47,024] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:47,024] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:49,090] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:28:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:49,090] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:49,193] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:49,193] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:51,196] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:51,196] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:51,196] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:53,270] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:28:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:53,270] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:53,373] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:53,373] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:55,375] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:55,376] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:55,376] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:57,419] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:28:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:57,419] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:57,522] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:57,523] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:57,523] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:28:57,523] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:57,523] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:57,523] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:28:59,563] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:28:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:59,564] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:28:59,667] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:28:59,667] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:28:59,668] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 18:28:59,668] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:28:59,668] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:28:59,668] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:28:59,668] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:29:01,863] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:01,863] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:01,966] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:01,966] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:29:01,966] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:29:01,967] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:29:01,967] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:01,968] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:29:14,087] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2016ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:29:14,087] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:14,189] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:29:14,189] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:29:17,192] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:29:17,193] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:29:17,193] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:29:17,194] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:17,194] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:17,194] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:29:19,409] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:19,410] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:19,512] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:19,513] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:29:19,513] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:29:19,513] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:29:19,513] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:19,514] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:29:21,617] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.706 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.421 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.390 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.390/0.505/0.706/0.144 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:29:21,617] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:21,719] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:29:21,719] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:29:21,720] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:29:26,724] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter8 [2019-08-22 18:29:26,724] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:26,724] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:26,725] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:29:28,741] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:28:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:28,741] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:28,844] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:28,844] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:29:28,845] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:28,845] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:28,845] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:29:30,900] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:28:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:30,900] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:31,003] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:31,004] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:29:31,004] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:29:31,004] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:29:31,004] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:31,005] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:31,005] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:29:32,986] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:32,986] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:33,090] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:33,090] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:29:33,090] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:33,090] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:33,090] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:29:35,187] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:29:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:35,188] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:35,290] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:35,290] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:29:35,291] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:29:35,291] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:29:37,293] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:37,293] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:37,294] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:29:39,585] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:29:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:39,585] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:39,688] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:39,689] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:29:41,691] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:41,691] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:41,691] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:29:43,923] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:29:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:43,923] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:44,026] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:44,026] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:29:46,029] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:46,029] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:46,029] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:29:48,056] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:29:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:48,056] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:48,159] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:48,159] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:29:50,162] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:50,162] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:50,162] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:29:52,202] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:29:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:52,203] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:52,306] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:52,306] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:29:54,308] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:54,309] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:54,309] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:29:56,341] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:29:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:56,341] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:29:56,444] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:29:56,445] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:29:58,447] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:29:58,447] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:29:58,448] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:00,641] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:29:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:00,641] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:00,744] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:00,744] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:02,747] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:02,747] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:02,747] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:04,777] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:30:02Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:04,777] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:04,880] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:04,881] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:04,881] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:30:04,881] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:04,881] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:04,881] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:06,957] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:30:02Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:06,957] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:07,060] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:07,060] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:07,061] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-0 [2019-08-22 18:30:07,061] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:30:07,061] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:07,061] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:07,061] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:30:09,236] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:09,236] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:09,339] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:09,340] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:30:09,340] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:30:09,340] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:30:09,340] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:09,340] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:30:11,442] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.692 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.427 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.302 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.302/0.473/0.692/0.164 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:30:11,442] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:11,544] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:30:11,544] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:30:11,544] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:30:16,550] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter9 [2019-08-22 18:30:16,550] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:16,550] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:16,550] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:18,559] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:30:02Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:18,559] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:18,662] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:18,662] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:18,662] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:18,663] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:18,663] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:20,687] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:30:02Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:20,687] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:20,790] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:20,790] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:20,791] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:30:20,791] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:30:20,791] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:20,791] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:20,791] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:22,681] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:22,681] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:22,784] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:22,784] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:30:22,785] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:22,785] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:22,785] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:24,864] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:30:22Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:24,864] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:24,967] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:24,968] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:24,968] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:30:24,968] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:30:26,970] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:26,971] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:26,971] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:29,363] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:30:22Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:29,363] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:29,466] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:29,466] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:31,468] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:31,468] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:31,468] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:33,576] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:30:22Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:33,576] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:33,679] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:33,679] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:35,682] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:35,682] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:35,682] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:37,711] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:30:34Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:37,711] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:37,814] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:37,815] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:39,817] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:39,817] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:39,817] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:41,855] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:30:34Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:41,856] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:41,959] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:41,959] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:43,961] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:43,962] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:43,962] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:46,002] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:30:34Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:46,002] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:46,105] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:46,105] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:48,107] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:48,107] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:48,107] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:50,187] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:30:34Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:50,187] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:50,290] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:50,290] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:52,292] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:52,293] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:52,293] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:54,539] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:30:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:54,540] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:54,647] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:54,648] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:54,648] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:30:54,648] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:54,648] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:54,648] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:30:56,680] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:30:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:56,680] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:56,783] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:56,784] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:30:56,784] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 18:30:56,784] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:30:56,784] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:30:56,784] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:56,784] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:30:58,945] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:58,946] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:30:59,048] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:30:59,049] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:30:59,049] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:30:59,049] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:30:59,049] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:30:59,049] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:31:01,154] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=2.65 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.583 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.352 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.352/1.195/2.651/1.034 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:31:01,154] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:01,256] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:31:01,256] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:31:01,256] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:31:06,258] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter10 [2019-08-22 18:31:06,258] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:06,259] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:06,259] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:08,296] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:30:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:08,296] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:08,399] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:08,400] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:08,400] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:08,400] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:08,400] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:10,419] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:30:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:10,420] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:10,523] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:10,523] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:10,523] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:31:10,524] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:31:10,524] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:10,524] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:10,524] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:12,468] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:12,469] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:12,571] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:12,572] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:31:12,572] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:12,572] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:12,572] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:14,621] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:31:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:14,622] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:14,725] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:14,725] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:14,725] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:31:14,725] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:31:16,727] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:16,727] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:16,727] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:18,925] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:31:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:18,925] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:19,028] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:19,029] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:21,031] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:21,031] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:21,031] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:23,089] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:31:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:23,089] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:23,192] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:23,192] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:25,195] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:25,195] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:25,195] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:27,326] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:31:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:27,326] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:27,429] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:27,429] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:29,431] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:29,432] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:29,432] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:31,487] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:31:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:31,487] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:31,590] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:31,591] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:33,593] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:33,593] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:33,594] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:35,640] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:31:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:35,640] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:35,743] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:35,744] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:37,746] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:37,746] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:37,746] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:40,029] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:31:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:40,030] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:40,132] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:40,133] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:40,133] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:31:40,133] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:40,133] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:40,133] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:42,127] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:31:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:42,127] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:42,230] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:42,230] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:42,230] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-0 [2019-08-22 18:31:42,231] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:31:42,231] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:42,231] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:42,231] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:31:44,423] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:44,423] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:44,527] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:44,527] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:31:44,527] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:31:44,527] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:31:44,527] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:44,528] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:31:46,633] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=2.31 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.435 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.248 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.248/0.999/2.316/0.934 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:31:46,633] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:46,735] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:31:46,735] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:31:46,736] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:31:51,741] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter11 [2019-08-22 18:31:51,741] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:51,741] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:51,742] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:53,742] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:31:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:53,742] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:53,846] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:53,846] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:53,846] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:53,846] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:53,846] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:56,074] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:31:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:56,074] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:56,177] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:56,178] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:31:56,178] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:31:56,178] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:31:56,178] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:56,179] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:56,179] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:31:58,140] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:58,140] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:31:58,243] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:31:58,243] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:31:58,244] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:31:58,244] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:31:58,244] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:00,410] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:31:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:00,410] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:00,513] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:00,513] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:00,514] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:32:00,514] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:32:02,516] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:02,516] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:02,516] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:04,570] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:31:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:04,570] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:04,673] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:04,674] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:06,676] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:06,676] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:06,676] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:08,706] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:31:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:08,706] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:08,809] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:08,810] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:10,812] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:10,812] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:10,813] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:12,887] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:32:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:12,887] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:12,990] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:12,991] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:14,992] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:14,992] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:14,993] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:17,011] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:32:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:17,012] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:17,115] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:17,115] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:19,117] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:19,117] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:19,117] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:21,156] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:32:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:21,156] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:21,260] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:21,260] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:23,262] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:23,262] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:23,262] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:25,328] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:32:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:25,328] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:25,431] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:25,431] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:27,434] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:27,434] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:27,434] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:29,469] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:32:29Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:29,470] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:29,572] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:29,573] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:29,573] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:32:29,573] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:29,573] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:29,573] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:31,616] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:32:29Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:31,616] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:31,719] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:31,720] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:31,720] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 18:32:31,720] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:32:31,720] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:31,720] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:31,721] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:32:33,864] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:33,864] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:33,967] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:33,968] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:32:33,968] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:32:33,968] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:32:33,968] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:33,968] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:32:36,075] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=2.25 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=2.49 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.421 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.421/1.722/2.493/0.925 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:32:36,075] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:36,177] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:32:36,177] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:32:36,177] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:32:41,183] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter12 [2019-08-22 18:32:41,183] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:41,183] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:41,183] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:43,203] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:32:29Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:43,204] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:43,307] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:43,307] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:43,307] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:43,307] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:43,308] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:45,348] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:32:29Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:45,348] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:45,451] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:45,452] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:45,452] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:32:45,452] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:32:45,452] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:45,452] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:45,453] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:47,379] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:47,379] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:47,482] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:47,482] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:32:47,483] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:47,483] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:47,483] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:49,521] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:32:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:49,521] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:49,624] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:49,625] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:49,625] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:32:49,625] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:32:51,627] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:51,628] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:51,628] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:53,686] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:32:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:53,687] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:53,790] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:53,790] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:55,792] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:55,792] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:55,793] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:32:57,822] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:32:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:57,822] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:32:57,925] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:32:57,925] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:32:59,928] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:32:59,928] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:32:59,928] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:01,976] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:32:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:01,976] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:02,079] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:02,079] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:04,082] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:04,082] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:04,082] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:06,139] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:02Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:06,140] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:06,242] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:06,243] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:08,245] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:08,245] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:08,246] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:10,294] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:10,294] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:10,398] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:10,398] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:12,400] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:12,401] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:12,401] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:14,410] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:14,410] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:14,513] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:14,513] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:16,515] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:16,516] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:16,516] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:18,741] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:18,742] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:18,844] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:18,845] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:20,847] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:20,847] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:20,847] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:22,889] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:33:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:22,890] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:22,992] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:22,993] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:22,993] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:33:22,993] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:22,993] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:22,993] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:25,070] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:33:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:25,070] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:25,173] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:25,173] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:25,173] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 18:33:25,174] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:33:25,174] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:25,174] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:25,174] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:33:27,358] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:27,358] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:27,461] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:27,462] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:33:27,462] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:33:27,462] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:33:27,462] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:27,462] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:33:29,569] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=3.75 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=1.99 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.295 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.295/2.016/3.759/1.415 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:33:29,569] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:29,671] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:33:29,671] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:33:29,671] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:33:34,677] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter13 [2019-08-22 18:33:34,677] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:34,677] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:34,677] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:36,735] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:33:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:36,736] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:36,839] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:36,839] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:36,839] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:36,839] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:36,840] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:38,880] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:33:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:38,881] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:38,983] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:38,984] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:38,984] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:33:38,984] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:33:38,984] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:38,985] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:38,985] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:40,912] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:40,912] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:41,015] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:41,016] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:33:41,016] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:41,016] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:41,016] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:43,412] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:43,412] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:43,515] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:43,516] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:43,516] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:33:43,516] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:33:45,518] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:45,518] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:45,518] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:47,572] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:47,573] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:47,676] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:47,676] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:49,678] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:49,678] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:49,678] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:51,728] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:51,728] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:51,831] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:51,832] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:53,834] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:53,834] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:53,834] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:33:55,880] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:55,880] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:33:55,983] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:33:55,983] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:33:57,986] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:33:57,986] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:33:57,986] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:00,038] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 78 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:00,039] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:00,142] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:00,142] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:34:02,144] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:02,145] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:02,145] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:04,286] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 78 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:04,286] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:04,389] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:04,389] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:34:06,392] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:06,392] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:06,392] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:08,580] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 78 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:33:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:08,581] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:08,684] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:08,684] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:34:10,686] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:10,686] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:10,686] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:12,741] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:34:11Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:12,741] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:12,844] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:12,845] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:34:12,845] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:34:12,845] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:12,845] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:12,845] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:14,903] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:34:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:14,904] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:15,006] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:15,007] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:34:15,007] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 18:34:15,007] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:34:15,007] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:15,007] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:15,008] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:34:17,444] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:17,444] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:17,547] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:17,547] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:34:17,547] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:34:17,548] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:34:17,548] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:17,548] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:34:29,667] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2015ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:34:29,667] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:29,769] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:34:29,769] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:34:32,772] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:34:32,773] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:34:32,773] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:34:32,773] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:32,773] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:32,773] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:34:34,933] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:34,933] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:35,036] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:35,036] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:34:35,037] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:34:35,037] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:34:35,037] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:35,037] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:34:37,141] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.652 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.611 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.419 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.419/0.560/0.652/0.105 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:34:37,141] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:37,243] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:34:37,243] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:34:37,244] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:34:42,249] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter14 [2019-08-22 18:34:42,249] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:42,249] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:42,250] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:44,265] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:34:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:44,265] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:44,368] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:44,368] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:34:44,368] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:44,369] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:44,369] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:46,399] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:34:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:46,399] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:46,502] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:46,503] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:34:46,503] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:34:46,503] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:34:46,503] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:46,503] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:46,503] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:48,422] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:48,422] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:48,525] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:48,525] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:34:48,526] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:48,526] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:48,526] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:50,565] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:34:48Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:50,565] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:50,668] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:50,668] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:34:50,668] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:34:50,668] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:34:52,671] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:52,671] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:52,671] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:54,873] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:34:48Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:54,873] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:54,976] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:54,976] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:34:56,979] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:34:56,979] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:34:56,979] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:34:59,024] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:34:48Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:59,024] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:34:59,127] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:34:59,128] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:01,129] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:01,129] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:01,129] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:03,158] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:34:48Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:03,158] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:03,261] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:03,262] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:05,264] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:05,264] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:05,265] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:07,331] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:35:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:07,331] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:07,434] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:07,434] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:09,437] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:09,437] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:09,437] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:11,661] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:35:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:11,661] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:11,764] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:11,764] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:13,767] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:13,767] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:13,767] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:15,839] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:35:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:15,839] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:15,942] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:15,943] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:17,945] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:17,945] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:17,946] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:20,033] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:35:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:20,033] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:20,136] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:20,137] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:22,139] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:22,139] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:22,139] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:24,253] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:35:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:24,253] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:24,357] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:24,358] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:24,358] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:35:24,358] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:24,358] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:24,358] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:26,407] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:35:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:26,407] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:26,510] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:26,511] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:26,511] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 18:35:26,511] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:35:26,511] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:26,512] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:26,512] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:35:28,674] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:28,675] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:28,778] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:28,778] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:35:28,778] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:35:28,778] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:35:28,779] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:28,779] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:35:40,899] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2017ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:35:40,899] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:41,001] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:35:41,001] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:35:44,004] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:35:44,004] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:35:44,005] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:35:44,005] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:44,005] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:44,005] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:35:46,225] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:46,226] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:46,328] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:46,329] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:35:46,329] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:35:46,329] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:35:46,329] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:46,330] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:35:48,433] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.04 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.505 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.386 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.386/0.645/1.045/0.287 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:35:48,434] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:48,535] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:35:48,536] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:35:48,536] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:35:53,541] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter15 [2019-08-22 18:35:53,541] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:53,542] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:53,542] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:55,574] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:35:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:55,574] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:55,677] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:55,677] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:55,678] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:55,678] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:55,678] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:57,718] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:35:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:57,718] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:57,821] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:57,821] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:35:57,822] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:35:57,822] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:35:57,822] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:57,822] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:57,822] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:35:59,754] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:59,754] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:35:59,857] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:35:59,857] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:35:59,858] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:35:59,858] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:35:59,858] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:01,922] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:35:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:01,922] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:02,025] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:02,026] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:02,026] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:36:02,026] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:36:04,028] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:04,028] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:04,029] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:06,314] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:35:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:06,314] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:06,417] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:06,418] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:08,420] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:08,420] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:08,421] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:10,496] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:35:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:10,496] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:10,600] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:10,600] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:12,602] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:12,602] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:12,603] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:14,583] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:36:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:14,583] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:14,686] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:14,687] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:16,689] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:16,689] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:16,689] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:18,743] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:36:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:18,743] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:18,846] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:18,847] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:20,849] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:20,849] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:20,850] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:22,884] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:36:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:22,884] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:22,987] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:22,987] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:24,990] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:24,990] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:24,990] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:27,057] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:36:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:27,058] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:27,161] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:27,161] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:29,164] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:29,164] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:29,164] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:31,192] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:36:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:31,193] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:31,296] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:31,296] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:31,297] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:36:31,297] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:31,297] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:31,297] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:33,277] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:36:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:33,278] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:33,381] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:33,381] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:33,382] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-0 [2019-08-22 18:36:33,382] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:36:33,382] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:33,382] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:33,382] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:36:35,557] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:35,557] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:35,660] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:35,661] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:36:35,661] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:36:35,661] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:36:35,661] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:35,661] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:36:37,765] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.25 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.438 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.260 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.260/0.652/1.258/0.434 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:36:37,765] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:37,867] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:36:37,867] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:36:37,868] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:36:42,873] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter16 [2019-08-22 18:36:42,873] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:42,874] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:42,874] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:44,912] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:36:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:44,913] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:45,016] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:45,016] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:45,017] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:45,017] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:45,017] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:47,055] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:36:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:47,056] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:47,159] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:47,159] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:47,159] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:36:47,159] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:36:47,160] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:47,160] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:47,160] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:49,154] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:49,154] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:49,257] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:49,258] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:36:49,258] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:49,258] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:49,258] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:51,392] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:36:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:51,393] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:51,496] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:51,496] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:51,496] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:36:51,497] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:36:53,499] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:53,499] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:53,499] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:55,600] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:36:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:55,601] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:55,704] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:55,704] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:36:57,706] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:36:57,707] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:36:57,707] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:36:59,802] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:36:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:59,803] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:36:59,906] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:36:59,906] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:01,908] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:01,909] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:01,909] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:03,956] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:00Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:03,957] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:04,060] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:04,060] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:06,063] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:06,063] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:06,063] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:08,110] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 77 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:08,110] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:08,213] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:08,214] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:10,216] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:10,216] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:10,216] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:12,211] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 77 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:12,211] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:12,315] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:12,315] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:14,317] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:14,318] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:14,318] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:16,622] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 77 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:16,622] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:16,725] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:16,725] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:18,727] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:18,727] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:18,728] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:20,745] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:37:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:20,746] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:20,849] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:20,849] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:20,849] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:37:20,849] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:20,850] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:20,850] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:22,846] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:37:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:22,846] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:22,949] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:22,950] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:22,950] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 18:37:22,950] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:37:22,950] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:22,950] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:22,950] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:37:25,141] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:25,141] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:25,244] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:25,244] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:37:25,244] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:37:25,245] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:37:25,245] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:25,245] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:37:27,349] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.20 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.531 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.286 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.286/0.674/1.207/0.390 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:37:27,349] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:27,451] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:37:27,451] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:37:27,452] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:37:32,456] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter17 [2019-08-22 18:37:32,456] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:32,457] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:32,457] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:34,474] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:37:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:34,474] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:34,578] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:34,578] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:34,578] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:34,578] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:34,578] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:36,635] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:37:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:36,636] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:36,739] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:36,739] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:36,740] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:37:36,740] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:37:36,740] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:36,740] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:36,740] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:38,639] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:38,639] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:38,742] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:38,742] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:37:38,743] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:38,743] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:38,743] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:40,767] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:40,767] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:40,870] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:40,871] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:40,871] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:37:40,871] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:37:42,874] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:42,874] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:42,874] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:45,095] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:45,095] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:45,198] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:45,199] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:47,200] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:47,200] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:47,200] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:49,233] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:49,234] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:49,337] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:49,337] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:51,340] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:51,340] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:51,340] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:53,350] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:53,350] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:53,454] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:53,454] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:55,456] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:55,457] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:55,457] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:37:57,500] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:57,500] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:37:57,603] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:37:57,604] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:37:59,605] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:37:59,605] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:37:59,605] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:01,644] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:01,644] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:01,747] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:01,747] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:38:03,749] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:03,750] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:03,750] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:05,834] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:37:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:05,834] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:05,938] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:05,938] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:38:07,940] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:07,941] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:07,941] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:09,967] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:38:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:09,968] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:10,070] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:10,071] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:38:10,071] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:38:10,071] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:10,071] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:10,071] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:12,250] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:38:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:12,251] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:12,354] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:12,354] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:38:12,354] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 18:38:12,354] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:38:12,354] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:12,355] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:12,355] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:38:14,562] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:14,562] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:14,665] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:14,666] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:38:14,666] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:38:14,666] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:38:14,666] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:14,666] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:38:26,769] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:38:26,769] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:26,871] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:38:26,871] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:38:29,874] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:38:29,875] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:38:29,875] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:38:29,875] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:29,875] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:29,876] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:38:32,260] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:32,260] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:32,363] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:32,364] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:38:32,364] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:38:32,364] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:38:32,364] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:32,364] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:38:34,470] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.47 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.543 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.358 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.358/0.791/1.472/0.487 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:38:34,470] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:34,572] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:38:34,572] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:38:34,573] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:38:39,578] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter18 [2019-08-22 18:38:39,578] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:39,578] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:39,578] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:41,637] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:38:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:41,637] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:41,740] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:41,740] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:38:41,740] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:41,741] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:41,741] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:43,773] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:38:07Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:43,774] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:43,877] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:43,877] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:38:43,877] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:38:43,877] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:38:43,878] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:43,878] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:43,878] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:45,829] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:45,829] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:45,932] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:45,933] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:38:45,933] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:45,933] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:45,934] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:48,149] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:38:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:48,149] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:48,252] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:48,252] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:38:48,252] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:38:48,253] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:38:50,255] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:50,255] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:50,255] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:52,444] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:38:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:52,445] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:52,547] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:52,548] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:38:54,550] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:54,551] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:54,551] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:38:56,597] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:38:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:56,597] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:38:56,700] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:38:56,701] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:38:58,703] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:38:58,703] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:38:58,703] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:00,733] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:38:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:00,733] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:00,837] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:00,837] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:02,840] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:02,840] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:02,840] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:04,875] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 27 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:39:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:04,875] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:04,979] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:04,979] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:06,982] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:06,982] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:06,982] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:09,016] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 27 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:39:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:09,016] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:09,119] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:09,120] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:11,122] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:11,122] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:11,123] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:13,185] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 27 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:39:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:13,185] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:13,288] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:13,289] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:15,291] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:15,291] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:15,291] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:17,355] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 27 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:39:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:17,356] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:17,458] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:17,459] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:19,460] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:19,460] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:19,461] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:21,486] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:39:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:21,486] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:21,589] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:21,590] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:21,590] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:39:21,590] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:21,590] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:21,590] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:23,650] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:39:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:23,650] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:23,753] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:23,754] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:23,754] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 18:39:23,754] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:39:23,754] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:23,754] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:23,755] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:39:25,972] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:25,972] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:26,075] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:26,076] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:39:26,076] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:39:26,076] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:39:26,076] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:26,076] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:39:38,181] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2001ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:39:38,181] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:38,283] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:39:38,283] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:39:41,286] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:39:41,287] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:39:41,287] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:39:41,287] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:41,287] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:41,287] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:39:43,468] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:43,468] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:43,571] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:43,571] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:39:43,571] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:39:43,571] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:39:43,572] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:43,572] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:39:45,674] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.45 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.478 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.434 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.434/0.789/1.456/0.472 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:39:45,674] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:45,776] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:39:45,777] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:39:45,777] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:39:50,780] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter19 [2019-08-22 18:39:50,780] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:50,780] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:50,780] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:52,817] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:39:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:52,818] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:52,921] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:52,921] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:52,921] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:52,922] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:52,922] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:54,939] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:39:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:54,939] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:55,042] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:55,042] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:55,042] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:39:55,043] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:39:55,043] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:55,043] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:55,043] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:56,955] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:56,956] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:57,059] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:57,059] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:39:57,059] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:39:57,060] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:39:57,060] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:39:59,102] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:39:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:59,102] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:39:59,205] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:39:59,205] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:39:59,206] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:39:59,206] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:40:01,208] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:01,208] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:01,209] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:03,232] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:39:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:03,232] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:03,335] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:03,336] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:05,337] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:05,337] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:05,338] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:07,675] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:39:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:07,676] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:07,779] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:07,779] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:09,781] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:09,782] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:09,782] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:11,840] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:39:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:11,840] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:11,943] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:11,944] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:13,946] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:13,946] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:13,946] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:16,017] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:40:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:16,017] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:16,120] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:16,120] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:18,122] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:18,123] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:18,123] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:20,208] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:40:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:20,208] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:20,311] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:20,311] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:22,314] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:22,314] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:22,314] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:24,458] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:40:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:24,458] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:24,561] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:24,562] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:26,564] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:26,564] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:26,564] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:28,621] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:40:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:28,621] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:28,724] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:28,725] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:30,726] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:30,726] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:30,726] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:32,779] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:40:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:32,780] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:32,883] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:32,883] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:34,885] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:34,886] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:34,886] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:36,933] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:40:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:36,933] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:37,036] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:37,037] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:39,039] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:39,039] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:39,039] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:41,291] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:40:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:41,291] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:41,394] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:41,395] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:43,397] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:43,398] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:43,398] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:45,433] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:40:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:45,434] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:45,537] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:45,537] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:45,537] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:40:45,537] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:45,538] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:45,538] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:47,569] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:40:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:47,570] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:47,672] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:47,673] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:47,673] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-0 [2019-08-22 18:40:47,673] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:40:47,673] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:47,674] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:47,674] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:40:49,875] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:49,875] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:49,978] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:49,979] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:40:49,979] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:40:49,979] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:40:49,979] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:49,979] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:40:52,086] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.927 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=1.00 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=3.16 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.927/1.700/3.166/1.037 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:40:52,087] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:52,189] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:40:52,189] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:40:52,189] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:40:57,193] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter20 [2019-08-22 18:40:57,194] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:57,194] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:57,194] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:40:59,219] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:40:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:59,219] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:40:59,323] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:40:59,323] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:40:59,323] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:40:59,324] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:40:59,324] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:01,339] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:40:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:01,339] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:01,442] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:01,443] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:01,443] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:41:01,443] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:41:01,443] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:01,443] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:01,443] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:03,367] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:03,367] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:03,470] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:03,470] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:41:03,470] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:03,470] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:03,471] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:05,547] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:05,547] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:05,650] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:05,651] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:05,651] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:41:05,651] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:41:07,653] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:07,653] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:07,654] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:09,799] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:09,799] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:09,902] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:09,903] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:11,905] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:11,905] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:11,905] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:13,935] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:13,935] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:14,038] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:14,038] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:16,041] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:16,041] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:16,041] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:18,091] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:18,091] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:18,194] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:18,194] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:20,197] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:20,197] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:20,197] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:22,235] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:22,236] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:22,338] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:22,339] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:24,342] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:24,342] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:24,342] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:26,567] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:26,567] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:26,671] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:26,671] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:28,674] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:28,674] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:28,674] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:31,010] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:31,010] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:31,113] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:31,114] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:33,116] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:33,116] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:33,117] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:35,138] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:41:33Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:35,139] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:35,245] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:35,245] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:35,245] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:41:35,245] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:35,246] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:35,246] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:37,261] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:41:33Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:37,262] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:37,365] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:37,365] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:37,365] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 18:41:37,365] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:41:37,365] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:37,366] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:37,366] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:41:39,578] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:39,579] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:39,682] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:39,682] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:41:39,682] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:41:39,683] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:41:39,683] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:39,683] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:41:41,789] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=7.76 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.473 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.310 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.310/2.850/7.768/3.478 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:41:41,789] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:41,891] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:41:41,891] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:41:41,892] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:41:46,897] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter21 [2019-08-22 18:41:46,897] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:46,897] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:46,898] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:48,912] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:41:33Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:48,912] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:49,015] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:49,016] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:49,017] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:49,017] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:49,017] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:51,053] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:41:33Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:51,053] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:51,156] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:51,156] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:51,157] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:41:51,157] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:41:51,157] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:51,157] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:51,157] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:53,091] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:53,091] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:53,194] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:53,194] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:41:53,195] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:53,195] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:53,195] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:55,289] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:55,289] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:55,392] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:55,393] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:41:55,393] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:41:55,393] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:41:57,395] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:41:57,396] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:41:57,396] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:41:59,431] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:59,431] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:41:59,534] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:41:59,534] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:01,536] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:01,536] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:01,536] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:03,612] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:41:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:03,612] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:03,715] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:03,715] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:05,718] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:05,718] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:05,718] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:07,771] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:42:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:07,771] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:07,874] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:07,874] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:09,877] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:09,877] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:09,877] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:11,960] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:42:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:11,960] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:12,064] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:12,064] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:14,066] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:14,066] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:14,066] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:16,067] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:42:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:16,068] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:16,171] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:16,172] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:18,174] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:18,174] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:18,174] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:20,305] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:42:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:20,305] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:20,408] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:20,408] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:22,411] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:22,411] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:22,411] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:24,528] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:42:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:24,528] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:24,632] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:24,632] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:24,632] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:42:24,632] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:24,633] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:24,633] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:26,674] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:42:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:26,674] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:26,777] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:26,778] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:26,778] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-0 [2019-08-22 18:42:26,778] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:42:26,778] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:26,779] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:26,779] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:42:28,937] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:28,937] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:29,041] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:29,041] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:42:29,041] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:42:29,041] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:42:29,042] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:29,042] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:42:31,149] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=2.60 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.889 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.431 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.431/1.306/2.600/0.934 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:42:31,149] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:31,251] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:42:31,251] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:42:31,251] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:42:36,257] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter22 [2019-08-22 18:42:36,257] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:36,257] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:36,257] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:38,300] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:42:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:38,301] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:38,404] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:38,404] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:38,405] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:38,405] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:38,405] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:40,445] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:42:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:40,445] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:40,549] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:40,549] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:40,549] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:42:40,549] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:42:40,549] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:40,550] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:40,550] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:42,661] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:42,661] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:42,763] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:42,764] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:42:42,764] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:42,764] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:42,764] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:44,903] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:42:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:44,903] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:45,006] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:45,007] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:45,007] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:42:45,007] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:42:47,009] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:47,010] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:47,010] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:49,073] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:42:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:49,073] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:49,176] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:49,176] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:51,179] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:51,179] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:51,179] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:53,220] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:42:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:53,220] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:53,323] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:53,323] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:55,326] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:55,326] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:55,326] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:42:57,379] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:42:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:57,379] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:42:57,482] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:42:57,483] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:42:59,485] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:42:59,485] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:42:59,485] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:01,526] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:42:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:01,527] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:01,630] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:01,630] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:03,633] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:03,633] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:03,633] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:05,675] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:43:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:05,675] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:05,778] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:05,779] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:07,781] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:07,781] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:07,781] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:09,814] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:43:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:09,814] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:09,917] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:09,917] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:11,919] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:11,920] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:11,920] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:14,089] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:43:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:14,090] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:14,193] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:14,193] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:16,196] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:16,196] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:16,196] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:18,219] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:43:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:18,220] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:18,323] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:18,323] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:20,325] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:20,326] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:20,326] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:22,336] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:43:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:22,336] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:22,439] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:22,440] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:24,442] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:24,442] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:24,443] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:26,508] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:43:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:26,509] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:26,612] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:26,612] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:26,612] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:43:26,612] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:26,613] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:26,613] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:28,634] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:43:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:28,634] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:28,737] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:28,738] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:28,738] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 18:43:28,738] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:43:28,738] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:28,738] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:28,739] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:43:30,936] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:30,936] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:31,039] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:31,039] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:43:31,040] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:43:31,040] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:43:31,040] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:31,040] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:43:33,145] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.09 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.435 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.352 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.352/0.627/1.094/0.331 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:43:33,145] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:33,247] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:43:33,247] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:43:33,248] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:43:38,253] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter23 [2019-08-22 18:43:38,253] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:38,253] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:38,254] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:40,486] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:43:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:40,486] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:40,589] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:40,589] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:40,589] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:40,590] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:40,590] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:42,626] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:43:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:42,627] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:42,730] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:42,730] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:42,730] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:43:42,731] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:43:42,731] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:42,731] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:42,731] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:44,610] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:44,610] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:44,713] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:44,713] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:43:44,714] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:44,714] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:44,714] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:46,759] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:43:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:46,759] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:46,862] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:46,863] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:46,863] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:43:46,863] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:43:48,865] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:48,866] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:48,866] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:51,438] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:43:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:51,438] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:51,541] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:51,541] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:53,544] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:53,544] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:53,544] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:55,582] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:43:55Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:55,583] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:55,685] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:55,686] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:43:57,688] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:43:57,688] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:43:57,689] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:43:59,894] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:43:55Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:59,895] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:43:59,998] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:43:59,998] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:44:02,001] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:02,001] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:02,001] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:04,036] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:44:00Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:04,036] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:04,139] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:04,139] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:44:06,141] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:06,141] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:06,142] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:08,262] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:44:00Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:08,262] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:08,365] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:08,365] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:44:10,368] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:10,368] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:10,368] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:12,416] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:44:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:12,416] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:12,519] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:12,519] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:44:12,520] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:44:12,520] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:12,520] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:12,520] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:14,494] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:44:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:14,495] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:14,598] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:14,599] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:44:14,599] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 18:44:14,599] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:44:14,600] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:14,600] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:14,600] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:44:16,808] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:16,808] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:16,911] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:16,911] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:44:16,912] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:44:16,912] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:44:16,912] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:16,912] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:44:29,024] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2008ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:44:29,024] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:29,126] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:44:29,126] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:44:32,129] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:44:32,129] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:44:32,130] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:44:32,130] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:32,130] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:32,130] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:44:34,313] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:34,313] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:34,416] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:34,417] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:44:34,417] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:44:34,417] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:44:34,417] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:34,417] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:44:36,522] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.41 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.484 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.378 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.378/0.758/1.413/0.465 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:44:36,522] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:36,624] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:44:36,624] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:44:36,624] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:44:41,629] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter24 [2019-08-22 18:44:41,630] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:41,630] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:41,630] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:43,677] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:44:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:43,677] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:43,780] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:43,781] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:44:43,781] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:43,781] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:43,781] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:45,808] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:44:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:45,808] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:45,911] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:45,911] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:44:45,912] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:44:45,912] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:44:45,912] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:45,912] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:45,912] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:47,803] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:47,804] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:47,907] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:47,907] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:44:47,907] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:47,907] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:47,908] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:49,970] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:44:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:49,970] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:50,073] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:50,073] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:44:50,073] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:44:50,073] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:44:52,076] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:52,076] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:52,076] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:54,080] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:44:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:54,080] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:54,183] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:54,184] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:44:56,186] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:44:56,186] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:44:56,186] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:44:58,275] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:44:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:58,275] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:44:58,378] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:44:58,378] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:00,381] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:00,381] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:00,381] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:02,413] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:44:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:02,413] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:02,516] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:02,517] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:04,517] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:04,518] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:04,518] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:06,586] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:45:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:06,586] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:06,689] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:06,690] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:08,691] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:08,691] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:08,691] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:10,718] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:45:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:10,718] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:10,821] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:10,822] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:12,824] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:12,824] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:12,824] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:15,234] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:45:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:15,234] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:15,337] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:15,338] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:17,340] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:17,340] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:17,340] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:19,382] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:45:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:19,382] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:19,485] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:19,485] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:21,488] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:21,488] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:21,488] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:23,539] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:45:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:23,539] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:23,642] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:23,643] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:25,645] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:25,645] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:25,645] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:27,708] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:45:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:27,709] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:27,812] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:27,812] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:27,812] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:45:27,813] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:27,813] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:27,813] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:29,867] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:45:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:29,868] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:29,971] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:29,971] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:29,971] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 18:45:29,971] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:45:29,972] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:29,972] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:29,972] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:45:32,221] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:32,221] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:32,324] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:32,325] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:45:32,325] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:45:32,325] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:45:32,325] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:32,325] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:45:34,432] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=5.42 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.284 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.284 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.284/1.996/5.420/2.421 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:45:34,432] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:34,534] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:45:34,534] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:45:34,534] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:45:39,539] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter25 [2019-08-22 18:45:39,539] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:39,540] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:39,540] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:41,594] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:45:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:41,594] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:41,697] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:41,697] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:41,698] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:41,698] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:41,698] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:43,740] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:45:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:43,740] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:43,843] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:43,843] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:43,843] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:45:43,844] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:45:43,844] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:43,844] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:43,844] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:46,180] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:46,180] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:46,283] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:46,283] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:45:46,284] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:46,284] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:46,284] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:48,492] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:45:46Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:48,492] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:48,596] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:48,596] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:48,596] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:45:48,597] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:45:50,599] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:50,599] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:50,600] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:53,108] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:45:46Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:53,108] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:53,211] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:53,211] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:55,214] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:55,214] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:55,214] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:45:57,252] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:45:46Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:57,253] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:45:57,356] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:45:57,356] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:45:59,358] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:45:59,359] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:45:59,359] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:01,396] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:45:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:01,396] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:01,499] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:01,500] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:03,502] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:03,502] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:03,502] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:05,540] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:02Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:05,541] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:05,643] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:05,644] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:07,646] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:07,646] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:07,647] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:09,702] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:02Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:09,703] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:09,805] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:09,806] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:11,808] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:11,808] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:11,809] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:13,889] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:02Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:13,889] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:13,992] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:13,993] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:15,994] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:15,994] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:15,994] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:18,021] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:46:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:18,022] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:18,125] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:18,125] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:18,125] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:46:18,125] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:18,125] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:18,126] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:20,169] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:46:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:20,169] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:20,273] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:20,273] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:20,273] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 18:46:20,273] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:46:20,274] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:20,274] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:20,274] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:46:22,466] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:22,467] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:22,570] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:22,570] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:46:22,570] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:46:22,571] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:46:22,571] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:22,571] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:46:24,675] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=9.92 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.399 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.374 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.374/3.566/9.927/4.497 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:46:24,675] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:24,777] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:46:24,777] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:46:24,777] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:46:29,783] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter26 [2019-08-22 18:46:29,783] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:29,783] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:29,783] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:31,801] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:46:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:31,801] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:31,905] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:31,905] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:31,905] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:31,906] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:31,906] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:33,926] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:46:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:33,926] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:34,030] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:34,030] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:34,030] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:46:34,030] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:46:34,031] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:34,031] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:34,031] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:35,958] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:35,958] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:36,061] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:36,062] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:46:36,062] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:36,062] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:36,062] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:38,080] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:38,080] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:38,183] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:38,184] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:38,184] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:46:38,184] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:46:40,186] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:40,186] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:40,186] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:42,451] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:42,451] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:42,554] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:42,554] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:44,556] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:44,557] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:44,557] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:46,828] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:46,829] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:46,932] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:46,932] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:48,934] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:48,934] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:48,935] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:50,920] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:50,921] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:51,024] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:51,024] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:53,026] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:53,027] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:53,027] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:55,048] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 61 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:55,048] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:55,151] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:55,152] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:46:57,153] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:46:57,153] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:46:57,153] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:46:59,191] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 61 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:59,191] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:46:59,295] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:46:59,295] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:47:01,297] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:01,298] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:01,298] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:03,525] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 61 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:46:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:03,525] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:03,628] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:03,629] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:47:05,631] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:05,631] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:05,631] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:08,083] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:47:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:08,083] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:08,186] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:08,186] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:47:08,186] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:47:08,187] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:08,187] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:08,187] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:10,226] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:47:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:10,226] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:10,329] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:10,330] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:47:10,330] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 18:47:10,330] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:47:10,330] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:10,330] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:10,330] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:47:12,615] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:12,615] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:12,718] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:12,718] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:47:12,718] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:47:12,718] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:47:12,719] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:12,719] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:47:24,821] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1998ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:47:24,821] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:24,923] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:47:24,923] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:47:27,926] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:47:27,927] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:47:27,927] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:47:27,927] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:27,927] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:27,927] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:47:30,179] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:30,179] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:30,282] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:30,282] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:47:30,282] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:47:30,282] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:47:30,283] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:30,283] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:47:32,386] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.777 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.472 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.376 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.376/0.541/0.777/0.173 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:47:32,387] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:32,489] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:47:32,489] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:47:32,489] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:47:37,494] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter27 [2019-08-22 18:47:37,495] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:37,495] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:37,495] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:39,546] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:47:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:39,547] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:39,649] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:39,650] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:47:39,650] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:39,651] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:39,651] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:41,681] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:47:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:41,681] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:41,784] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:41,785] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:47:41,785] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:47:41,785] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:47:41,785] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:41,786] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:41,786] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:43,701] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:43,701] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:43,804] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:43,804] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:47:43,804] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:43,804] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:43,805] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:45,884] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:47:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:45,884] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:45,987] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:45,987] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:47:45,987] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:47:45,987] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:47:47,990] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:47,990] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:47,990] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:50,182] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:47:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:50,182] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:50,285] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:50,286] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:47:52,288] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:52,288] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:52,289] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:54,327] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:47:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:54,327] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:54,430] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:54,431] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:47:56,433] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:47:56,434] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:47:56,434] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:47:58,472] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:47:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:58,472] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:47:58,575] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:47:58,576] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:00,578] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:00,578] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:00,579] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:02,620] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:47:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:02,620] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:02,723] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:02,723] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:04,726] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:04,726] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:04,726] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:06,775] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:48:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:06,776] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:06,879] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:06,879] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:08,882] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:08,882] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:08,882] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:10,947] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:48:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:10,948] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:11,050] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:11,051] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:13,053] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:13,053] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:13,053] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:15,124] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:48:03Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:15,125] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:15,228] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:15,228] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:17,230] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:17,231] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:17,231] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:19,262] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:48:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:19,262] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:19,365] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:19,365] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:19,366] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:48:19,366] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:19,366] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:19,366] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:21,413] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:48:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:21,413] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:21,517] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:21,517] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:21,517] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 18:48:21,518] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:48:21,518] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:21,518] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:21,518] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:48:23,708] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:23,708] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:23,811] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:23,812] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:48:23,812] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:48:23,812] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:48:23,812] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:23,813] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:48:35,922] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2006ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:48:35,923] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:36,024] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:48:36,025] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:48:39,028] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:48:39,028] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:48:39,028] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:48:39,029] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:39,029] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:39,029] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:48:41,229] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:41,229] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:41,332] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:41,333] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:48:41,333] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:48:41,333] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:48:41,333] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:41,333] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:48:43,438] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=11.6 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.441 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.320 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.320/4.133/11.639/5.307 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:48:43,438] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:43,540] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:48:43,540] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:48:43,541] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:48:48,546] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter28 [2019-08-22 18:48:48,546] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:48,546] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:48,547] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:50,627] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:48:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:50,627] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:50,730] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:50,731] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:50,731] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:50,731] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:50,731] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:52,733] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:48:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:52,733] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:52,836] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:52,837] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:52,837] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:48:52,837] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:48:52,837] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:52,838] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:52,838] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:54,695] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:54,695] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:54,798] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:54,799] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:48:54,799] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:54,799] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:54,799] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:48:56,882] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:48:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:56,882] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:48:56,985] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:48:56,985] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:48:56,986] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:48:56,986] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:48:58,988] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:48:58,988] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:48:58,988] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:00,986] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:48:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:00,987] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:01,089] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:01,090] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:03,092] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:03,092] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:03,093] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:05,126] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:48:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:05,127] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:05,229] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:05,230] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:07,232] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:07,232] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:07,232] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:09,265] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:09,265] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:09,368] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:09,369] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:11,371] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:11,371] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:11,371] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:13,381] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:13,382] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:13,485] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:13,485] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:15,487] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:15,488] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:15,488] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:17,512] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:17,512] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:17,615] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:17,616] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:19,618] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:19,618] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:19,619] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:21,695] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:21,696] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:21,799] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:21,799] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:23,802] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:23,802] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:23,802] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:25,916] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:49:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:25,916] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:26,019] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:26,020] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:26,020] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:49:26,020] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:26,020] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:26,020] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:28,033] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:49:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:28,033] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:28,136] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:28,137] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:28,137] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-0 [2019-08-22 18:49:28,137] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:49:28,137] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:28,137] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:28,138] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:49:30,314] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:30,314] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:30,417] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:30,417] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:49:30,417] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:49:30,418] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:49:30,418] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:30,418] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:49:32,522] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=3.83 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.390 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.296 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.296/1.506/3.834/1.646 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:49:32,523] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:32,624] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:49:32,625] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:49:32,625] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:49:37,626] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter29 [2019-08-22 18:49:37,626] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:37,627] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:37,627] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:39,726] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:49:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:39,727] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:39,830] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:39,830] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:39,830] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:39,830] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:39,831] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:41,862] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:49:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:41,862] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:41,965] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:41,966] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:41,966] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:49:41,966] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:49:41,966] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:41,966] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:41,966] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:43,865] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:43,865] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:43,968] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:43,968] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:49:43,968] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:43,969] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:43,969] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:46,584] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:46,584] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:46,688] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:46,688] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:46,688] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:49:46,688] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:49:48,691] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:48,691] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:48,691] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:51,159] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:51,160] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:51,262] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:51,263] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:53,265] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:53,266] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:53,266] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:55,323] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:55,324] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:55,427] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:55,427] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:49:57,430] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:49:57,430] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:49:57,430] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:49:59,495] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:59,495] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:49:59,598] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:49:59,599] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:50:01,601] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:01,601] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:01,601] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:03,633] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:03,633] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:03,736] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:03,736] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:50:05,739] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:05,739] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:05,739] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:07,779] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:07,779] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:07,882] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:07,882] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:50:09,885] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:09,885] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:09,885] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:11,926] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:49:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:11,926] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:12,029] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:12,029] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:50:14,032] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:14,032] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:14,032] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:16,086] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:50:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:16,086] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:16,189] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:16,189] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:50:16,190] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:50:16,190] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:16,190] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:16,190] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:18,230] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:50:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:18,231] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:18,333] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:18,334] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:50:18,334] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 18:50:18,334] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:50:18,334] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:18,335] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:18,335] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:50:20,567] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:20,568] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:20,671] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:20,671] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:50:20,671] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:50:20,671] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:50:20,672] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:20,672] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:50:32,783] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2008ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:50:32,783] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:32,885] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:50:32,885] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:50:35,888] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:50:35,889] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:50:35,889] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:50:35,889] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:35,889] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:35,889] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:50:38,060] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:38,060] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:38,163] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:38,163] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:50:38,164] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:50:38,164] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:50:38,164] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:38,164] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:50:40,270] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.39 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.424 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.317 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.317/0.711/1.392/0.483 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:50:40,270] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:40,372] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:50:40,372] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:50:40,372] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:50:45,378] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter30 [2019-08-22 18:50:45,378] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:45,378] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:45,378] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:47,382] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:50:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:47,382] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:47,485] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:47,486] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:50:47,486] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:47,486] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:47,486] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:49,528] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:50:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:49,528] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:49,631] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:49,632] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:50:49,632] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:50:49,632] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:50:49,632] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:49,632] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:49,632] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:51,573] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:51,573] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:51,676] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:51,676] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:50:51,676] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:51,677] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:51,677] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:53,719] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:50:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:53,719] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:53,822] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:53,823] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:50:53,823] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:50:53,823] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:50:55,825] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:50:55,825] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:50:55,825] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:50:57,994] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:50:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:57,994] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:50:58,097] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:50:58,098] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:00,100] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:00,100] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:00,100] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:02,111] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:50:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:02,112] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:02,215] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:02,215] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:04,217] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:04,217] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:04,218] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:06,272] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:50:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:06,272] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:06,375] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:06,375] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:08,378] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:08,378] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:08,378] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:10,382] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:51:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:10,382] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:10,485] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:10,485] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:12,488] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:12,488] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:12,488] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:14,520] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:51:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:14,520] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:14,623] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:14,623] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:16,626] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:16,626] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:16,626] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:18,672] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:51:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:18,672] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:18,775] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:18,776] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:20,778] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:20,778] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:20,779] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:22,945] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:51:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:22,945] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:23,048] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:23,048] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:25,050] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:25,050] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:25,051] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:27,080] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:51:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:27,081] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:27,183] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:27,184] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:27,184] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:51:27,184] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:27,184] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:27,184] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:29,353] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:51:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:29,354] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:29,456] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:29,457] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:29,457] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 18:51:29,457] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:51:29,457] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:29,458] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:29,458] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:51:31,640] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:31,640] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:31,744] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:31,744] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:51:31,744] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:51:31,744] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:51:31,745] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:31,745] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:51:33,847] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=6.42 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.757 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.318 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.318/2.501/6.428/2.782 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:51:33,847] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:33,949] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:51:33,949] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:51:33,949] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:51:38,955] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter31 [2019-08-22 18:51:38,955] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:38,955] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:38,955] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:41,009] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:51:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:41,009] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:41,113] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:41,113] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:41,113] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:41,113] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:41,113] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:43,156] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:51:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:43,156] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:43,259] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:43,260] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:43,260] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:51:43,260] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:51:43,260] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:43,261] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:43,261] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:45,217] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:45,217] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:45,321] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:45,321] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:51:45,321] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:45,321] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:45,321] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:47,383] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:51:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:47,383] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:47,487] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:47,487] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:47,487] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:51:47,488] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:51:49,489] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:49,489] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:49,489] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:51,812] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:51:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:51,813] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:51,916] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:51,916] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:53,918] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:53,918] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:53,918] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:51:55,983] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:51:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:55,983] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:51:56,086] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:51:56,087] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:51:58,088] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:51:58,088] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:51:58,089] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:52:00,149] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:51:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:00,149] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:00,252] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:00,252] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:52:02,254] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:52:02,255] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:02,255] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:52:04,468] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:52:01Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:04,469] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:04,572] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:04,572] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:52:06,574] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:52:06,575] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:06,575] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:52:08,614] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:52:01Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:08,614] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:08,717] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:08,718] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:52:10,720] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:52:10,720] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:10,721] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:52:12,797] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:52:01Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:12,798] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:12,901] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:12,901] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:52:14,904] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:52:14,904] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:14,904] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:52:17,012] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:52:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:17,012] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:17,115] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:17,116] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:52:17,116] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:52:17,116] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:52:17,116] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:17,116] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:52:19,150] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:52:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:19,150] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:19,253] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:19,254] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:52:19,254] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 18:52:19,254] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:52:19,254] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:52:19,254] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:19,254] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:52:21,426] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:21,427] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:21,530] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:21,530] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:52:21,530] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:52:21,530] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:52:21,530] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:21,531] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:52:33,651] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2016ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:52:33,651] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:33,753] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:52:33,753] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:52:36,756] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:52:36,756] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:52:36,756] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:52:36,757] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:52:36,757] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:36,757] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:52:38,897] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:38,898] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:39,001] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:39,001] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:52:39,001] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:52:39,001] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:52:39,002] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:39,002] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:52:51,105] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2000ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:52:51,105] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:51,207] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:52:51,207] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:52:54,210] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:52:54,211] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:52:54,211] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:52:54,211] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:52:54,211] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:54,211] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:52:56,428] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:56,428] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:56,531] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:52:56,531] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:52:56,532] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:52:56,532] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:52:56,532] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:52:56,532] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:52:58,637] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.36 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.453 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.303 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.303/0.706/1.363/0.468 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:52:58,637] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:52:58,739] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:52:58,739] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:52:58,740] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:53:03,745] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter32 [2019-08-22 18:53:03,745] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:03,745] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:03,745] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:05,799] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:52:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:05,799] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:05,902] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:05,902] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:05,903] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:05,903] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:05,903] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:08,112] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:52:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:08,113] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:08,216] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:08,216] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:08,216] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:53:08,217] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:53:08,217] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:08,217] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:08,217] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:10,095] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:10,096] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:10,198] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:10,199] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:53:10,199] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:10,199] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:10,199] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:12,231] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:53:09Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:12,232] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:12,335] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:12,335] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:12,336] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:53:12,336] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:53:14,338] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:14,338] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:14,339] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:16,388] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:53:09Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:16,388] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:16,492] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:16,492] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:18,494] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:18,495] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:18,495] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:20,549] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:53:09Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:20,549] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:20,652] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:20,653] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:22,653] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:22,654] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:22,654] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:24,702] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:53:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:24,703] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:24,806] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:24,806] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:26,808] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:26,809] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:26,809] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:28,794] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:53:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:28,795] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:28,898] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:28,898] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:30,900] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:30,901] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:30,901] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:32,959] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:53:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:32,960] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:33,063] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:33,063] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:35,065] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:35,065] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:35,066] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:37,146] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:53:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:37,147] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:37,250] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:37,250] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:39,252] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:39,253] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:39,253] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:41,298] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:53:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:41,298] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:41,402] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:41,402] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:41,402] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:53:41,403] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:41,403] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:41,403] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:53:43,430] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:53:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:43,430] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:43,533] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:43,534] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:53:43,534] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 18:53:43,534] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:53:43,534] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:53:43,534] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:43,535] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:53:45,725] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:45,725] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:45,828] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:53:45,828] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:53:45,828] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:53:45,829] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:53:45,829] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:53:45,829] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:53:57,947] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:53:57,947] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:53:58,049] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:53:58,049] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:54:01,053] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:54:01,053] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:54:01,053] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:54:01,053] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:01,053] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:01,054] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:54:03,250] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:03,250] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:03,353] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:03,353] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:54:03,353] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:54:03,354] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:54:03,354] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:03,354] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:54:05,458] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.61 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.551 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.349 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.349/0.839/1.617/0.556 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:54:05,459] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:05,560] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:54:05,561] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:54:05,561] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:54:10,566] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter33 [2019-08-22 18:54:10,566] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:10,567] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:10,567] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:12,592] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:53:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:12,592] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:12,695] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:12,695] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:12,696] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:12,696] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:12,696] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:14,720] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:53:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:14,720] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:14,823] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:14,823] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:14,824] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:54:14,824] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:54:14,824] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:14,824] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:14,824] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:16,701] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:16,701] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:16,804] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:16,804] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:54:16,804] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:16,804] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:16,805] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:18,827] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:54:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:18,827] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:18,931] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:18,931] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:18,932] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:54:18,932] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:54:20,933] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:20,933] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:20,933] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:23,225] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:54:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:23,225] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:23,328] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:23,329] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:25,331] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:25,331] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:25,331] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:27,364] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:54:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:27,364] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:27,467] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:27,468] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:29,470] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:29,470] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:29,470] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:31,526] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:54:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:31,527] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:31,630] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:31,630] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:33,633] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:33,633] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:33,633] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:35,708] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:54:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:35,708] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:35,811] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:35,811] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:37,813] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:37,813] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:37,814] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:39,858] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:54:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:39,858] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:39,961] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:39,961] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:41,963] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:41,964] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:41,964] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:44,116] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:54:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:44,116] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:44,219] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:44,220] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:46,220] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:46,221] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:46,221] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:48,484] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:54:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:48,485] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:48,587] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:48,588] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:50,590] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:50,590] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:50,590] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:52,675] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:54:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:52,675] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:52,779] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:52,779] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:52,779] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:54:52,779] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:52,779] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:52,780] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:54:54,850] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:54:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:54,850] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:54,953] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:54,954] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:54:54,954] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 18:54:54,954] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:54:54,954] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:54:54,955] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:54,955] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:54:57,152] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:57,153] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:57,256] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:54:57,256] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:54:57,256] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:54:57,256] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:54:57,257] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:54:57,257] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:54:59,361] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.07 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.354 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.253 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.253/0.560/1.075/0.367 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:54:59,361] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:54:59,463] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:54:59,463] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:54:59,463] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:55:04,469] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter34 [2019-08-22 18:55:04,469] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:04,469] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:04,469] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:06,519] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:54:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:06,519] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:06,622] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:06,622] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:06,623] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:06,623] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:06,623] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:08,628] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:54:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:08,628] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:08,731] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:08,731] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:08,731] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 18:55:08,732] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:55:08,732] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:08,732] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:08,732] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:10,818] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:10,818] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:10,921] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:10,921] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:55:10,922] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:10,922] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:10,922] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:12,996] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:55:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:12,997] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:13,100] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:13,100] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:13,100] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:55:13,101] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:55:15,103] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:15,103] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:15,103] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:17,105] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:55:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:17,105] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:17,208] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:17,209] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:19,210] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:19,210] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:19,210] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:21,232] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:55:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:21,232] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:21,336] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:21,336] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:23,338] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:23,338] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:23,338] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:25,404] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:55:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:25,404] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:25,507] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:25,508] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:27,510] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:27,510] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:27,510] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:29,556] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:55:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:29,557] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:29,659] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:29,660] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:31,662] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:31,662] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:31,663] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:34,001] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:55:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:34,001] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:34,104] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:34,104] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:36,106] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:36,106] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:36,106] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:38,176] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:55:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:38,176] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:38,279] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:38,279] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:40,281] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:40,281] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:40,281] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:42,322] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:55:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:42,322] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:42,426] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:42,426] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:42,426] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:55:42,426] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:42,426] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:42,427] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:55:44,429] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:55:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:44,429] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:44,532] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:44,533] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:55:44,533] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 18:55:44,533] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:55:44,534] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:55:44,534] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:44,534] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:55:46,735] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:46,735] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:46,837] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:55:46,838] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:55:46,838] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:55:46,838] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:55:46,838] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:55:46,838] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:55:58,941] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:55:58,941] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:55:59,043] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:55:59,043] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:56:02,046] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:56:02,047] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:56:02,047] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:56:02,047] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:02,047] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:02,047] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:56:04,203] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:04,204] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:04,306] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:04,307] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:56:04,307] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:56:04,307] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:56:04,307] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:04,307] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:56:06,413] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.49 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.487 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.341 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.341/0.774/1.494/0.512 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:56:06,413] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:06,515] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:56:06,515] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:56:06,516] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:56:11,520] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter35 [2019-08-22 18:56:11,520] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:11,520] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:11,521] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:13,769] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:55:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:13,770] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:13,873] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:13,873] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:13,873] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:13,873] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:13,873] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:15,885] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:55:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:15,885] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:15,988] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:15,989] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:15,989] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:56:15,989] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:56:15,989] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:15,989] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:15,989] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:17,905] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:17,905] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:18,008] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:18,008] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:56:18,009] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:18,009] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:18,009] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:20,120] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:56:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:20,120] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:20,224] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:20,224] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:20,224] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:56:20,224] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:56:22,225] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:22,225] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:22,225] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:24,582] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:56:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:24,582] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:24,685] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:24,685] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:26,688] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:26,688] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:26,688] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:28,735] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:56:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:28,735] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:28,838] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:28,839] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:30,841] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:30,841] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:30,841] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:32,873] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:56:29Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:32,873] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:32,976] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:32,976] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:34,979] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:34,979] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:34,979] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:37,040] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:56:34Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:37,040] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:37,143] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:37,144] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:39,146] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:39,147] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:39,147] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:41,172] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:56:34Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:41,172] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:41,275] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:41,276] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:43,278] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:43,278] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:43,278] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:45,483] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:56:34Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:45,483] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:45,587] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:45,587] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:47,589] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:47,590] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:47,590] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:49,679] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:56:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:49,679] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:49,782] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:49,782] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:49,783] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:56:49,783] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:49,783] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:49,783] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:56:51,803] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:56:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:51,804] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:51,907] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:51,907] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:56:51,907] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 18:56:51,907] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:56:51,908] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:56:51,908] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:51,908] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:56:54,081] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:54,081] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:56:54,184] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:56:54,184] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:56:54,185] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:56:54,185] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:56:54,185] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:56:54,185] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:57:06,303] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:57:06,303] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:06,405] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:57:06,405] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:57:09,408] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:57:09,409] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:57:09,409] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:57:09,409] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:09,409] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:09,410] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:57:11,613] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:11,613] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:11,717] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:11,717] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:57:11,717] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:57:11,717] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:57:11,718] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:11,718] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:57:13,822] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.41 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.586 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.356 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.356/0.785/1.413/0.453 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:57:13,822] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:13,924] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:57:13,924] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:57:13,925] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:57:18,930] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter36 [2019-08-22 18:57:18,930] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:18,930] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:18,930] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:21,083] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:56:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:21,083] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:21,186] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:21,186] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:21,186] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:21,187] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:21,187] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:23,217] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:56:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:23,217] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:23,320] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:23,321] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:23,321] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:57:23,321] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:57:23,321] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:23,321] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:23,321] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:25,223] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:25,224] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:25,326] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:25,327] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:57:25,327] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:25,327] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:25,327] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:27,684] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:57:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:27,684] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:27,787] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:27,788] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:27,788] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:57:27,788] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:57:29,790] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:29,791] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:29,791] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:32,199] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:57:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:32,200] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:32,303] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:32,303] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:34,306] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:34,306] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:34,306] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:36,378] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:57:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:36,378] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:36,481] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:36,481] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:38,483] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:38,483] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:38,484] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:40,519] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:57:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:40,519] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:40,623] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:40,623] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:42,625] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:42,626] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:42,626] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:44,696] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:57:41Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:44,697] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:44,799] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:44,800] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:46,802] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:46,802] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:46,802] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:48,863] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:57:41Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:48,863] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:48,966] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:48,967] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:50,969] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:50,969] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:50,969] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:53,261] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:57:50Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:53,262] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:53,365] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:53,365] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:55,367] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:55,368] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:55,368] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:57:57,584] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:57:50Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:57,584] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:57:57,687] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:57:57,688] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:57:59,690] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:57:59,690] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:57:59,690] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:01,732] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:57:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:01,732] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:01,835] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:01,835] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:58:01,835] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:58:01,836] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:01,836] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:01,836] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:03,945] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:57:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:03,946] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:04,048] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:04,049] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:58:04,049] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 18:58:04,049] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:58:04,049] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:04,049] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:04,050] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:58:06,259] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:06,259] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:06,363] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:06,364] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:58:06,364] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:58:06,364] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:58:06,364] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:06,364] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:58:18,483] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:58:18,483] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:18,585] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:58:18,585] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:58:21,588] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:58:21,589] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:58:21,589] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:58:21,589] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:21,589] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:21,590] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:58:23,785] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:23,785] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:23,888] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:23,888] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:58:23,888] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:58:23,889] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:58:23,889] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:23,889] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:58:25,993] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.685 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.587 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.327 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.327/0.533/0.685/0.151 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:58:25,993] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:26,095] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:58:26,095] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:58:26,095] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:58:31,101] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter37 [2019-08-22 18:58:31,101] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:31,101] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:31,101] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:33,130] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:57:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:33,130] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:33,233] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:33,233] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:58:33,234] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:33,234] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:33,234] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:35,277] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:57:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:35,277] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:35,380] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:35,380] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:58:35,381] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 18:58:35,381] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:58:35,381] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:35,381] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:35,381] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:37,276] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:37,276] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:37,379] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:37,379] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:58:37,379] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:37,379] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:37,380] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:39,485] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:58:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:39,486] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:39,589] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:39,589] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:58:39,589] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:58:39,589] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:58:41,592] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:41,592] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:41,592] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:43,634] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:58:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:43,634] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:43,737] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:43,738] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:58:45,740] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:45,740] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:45,741] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:47,803] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:58:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:47,804] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:47,906] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:47,907] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:58:49,909] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:49,909] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:49,910] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:52,000] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:58:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:52,000] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:52,103] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:52,103] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:58:54,106] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:54,106] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:54,106] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:58:56,117] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:58:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:56,117] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:58:56,220] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:58:56,221] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:58:58,223] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:58:58,223] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:58:58,224] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:00,266] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:58:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:00,266] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:00,369] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:00,370] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:59:02,370] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:02,370] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:02,370] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:04,599] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:58:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:04,599] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:04,702] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:04,703] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:59:06,705] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:06,705] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:06,705] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:08,809] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:59:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:08,810] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:08,913] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:08,913] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:59:08,914] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 18:59:08,914] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:08,914] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:08,914] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:10,931] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:59:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:10,931] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:11,034] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:11,035] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:59:11,035] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 18:59:11,035] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:59:11,035] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:11,035] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:11,035] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:59:13,215] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:13,215] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:13,318] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:13,319] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:59:13,319] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:59:13,319] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:59:13,319] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:13,319] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:59:25,439] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2016ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:59:25,439] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:25,541] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:59:25,541] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 18:59:28,544] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 18:59:28,545] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 18:59:28,545] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 18:59:28,545] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:28,545] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:28,545] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 18:59:30,726] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:30,726] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:30,829] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:30,829] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 18:59:30,829] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 18:59:30,830] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 18:59:30,830] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:30,830] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 18:59:32,934] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.09 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.538 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.365 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.365/0.664/1.091/0.311 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:59:32,934] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:33,036] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 18:59:33,036] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 18:59:33,036] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 18:59:38,041] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter38 [2019-08-22 18:59:38,042] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:38,042] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:38,042] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:40,067] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:59:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:40,067] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:40,170] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:40,171] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:59:40,171] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:40,171] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:40,171] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:42,182] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T18:59:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:42,182] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:42,285] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:42,286] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:59:42,286] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 18:59:42,286] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 18:59:42,286] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:42,286] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:42,286] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:44,271] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:44,272] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:44,374] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:44,375] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 18:59:44,375] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:44,375] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:44,375] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:46,424] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:59:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:46,424] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:46,527] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:46,528] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:59:46,528] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 18:59:46,528] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 18:59:48,530] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:48,531] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:48,531] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:50,544] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:59:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:50,545] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:50,648] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:50,648] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:59:52,650] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:52,651] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:52,651] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:54,984] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:59:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:54,985] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:55,087] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:55,088] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 18:59:57,090] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 18:59:57,090] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 18:59:57,090] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 18:59:59,105] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:59:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:59,105] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 18:59:59,208] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 18:59:59,208] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:01,211] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:01,211] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:01,211] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:03,259] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T18:59:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:03,260] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:03,362] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:03,363] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:05,365] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:05,365] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:05,366] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:07,413] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:07,414] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:07,516] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:07,517] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:09,519] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:09,519] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:09,519] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:11,545] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:11,545] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:11,649] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:11,649] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:13,651] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:13,652] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:13,652] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:15,932] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:15,932] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:16,036] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:16,036] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:18,038] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:18,039] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:18,039] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:20,264] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:06Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:20,264] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:20,367] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:20,368] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:22,370] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:22,370] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:22,371] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:24,444] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:00:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:24,444] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:24,547] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:24,547] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:24,548] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:00:24,548] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:24,548] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:24,548] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:26,601] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:00:24Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:26,601] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:26,704] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:26,705] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:26,705] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 19:00:26,705] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:00:26,705] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:26,705] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:26,706] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:00:28,897] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:28,897] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:29,000] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:29,001] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:00:29,001] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:00:29,001] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:00:29,001] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:29,002] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:00:31,106] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=2.52 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.422 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.305 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.305/1.085/2.529/1.022 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:00:31,106] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:31,208] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:00:31,208] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:00:31,208] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:00:36,213] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter39 [2019-08-22 19:00:36,213] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:36,214] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:36,214] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:38,425] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:00:24Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:38,425] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:38,528] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:38,529] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:38,529] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:38,529] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:38,529] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:40,572] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:00:24Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:40,572] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:40,675] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:40,676] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:40,676] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 19:00:40,676] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:00:40,676] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:40,676] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:40,676] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:42,591] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:42,591] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:42,694] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:42,694] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:00:42,694] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:42,694] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:42,695] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:44,856] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:44,857] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:44,959] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:44,960] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:44,960] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:00:44,960] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:00:46,962] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:46,962] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:46,962] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:49,138] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:49,138] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:49,241] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:49,242] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:51,244] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:51,244] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:51,244] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:53,290] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:53,290] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:53,393] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:53,394] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:55,396] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:55,396] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:55,397] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:00:57,373] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:57,373] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:00:57,476] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:00:57,476] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:00:59,478] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:00:59,478] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:00:59,478] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:01,518] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 95 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:01,519] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:01,622] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:01,622] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:03,624] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:03,625] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:03,625] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:05,641] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 95 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:05,642] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:05,744] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:05,745] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:07,747] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:07,747] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:07,748] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:09,822] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 95 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:00:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:09,823] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:09,925] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:09,926] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:11,928] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:11,929] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:11,929] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:13,960] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:01:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:13,960] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:14,063] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:14,063] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:14,064] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:01:14,064] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:14,064] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:14,064] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:16,103] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:01:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:16,103] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:16,206] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:16,206] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:16,206] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 19:01:16,207] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:01:16,207] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:16,207] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:16,207] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:01:18,409] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:18,409] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:18,512] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:18,513] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:01:18,513] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:01:18,513] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:01:18,513] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:18,513] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:01:20,620] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.16 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=4.52 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.347 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.347/2.013/4.527/1.809 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:01:20,620] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:20,722] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:01:20,722] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:01:20,722] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:01:25,727] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter40 [2019-08-22 19:01:25,728] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:25,728] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:25,728] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:27,706] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:01:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:27,706] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:27,810] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:27,810] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:27,811] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:27,811] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:27,811] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:29,867] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:01:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:29,867] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:29,970] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:29,971] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:29,971] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:01:29,971] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:01:29,971] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:29,971] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:29,972] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:31,887] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:31,888] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:31,991] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:31,991] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:01:31,991] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:31,991] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:31,991] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:34,065] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:01:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:34,065] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:34,168] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:34,169] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:34,169] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:01:34,169] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:01:36,172] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:36,172] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:36,172] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:38,260] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:01:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:38,260] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:38,363] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:38,363] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:40,366] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:40,366] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:40,366] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:42,396] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:01:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:42,396] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:42,499] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:42,499] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:44,501] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:44,501] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:44,502] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:46,782] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:01:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:46,782] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:46,885] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:46,886] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:48,888] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:48,888] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:48,889] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:50,876] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:01:48Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:50,877] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:50,980] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:50,980] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:52,983] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:52,983] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:52,983] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:54,980] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 29 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:01:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:54,981] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:55,084] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:55,084] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:01:57,086] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:01:57,087] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:01:57,087] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:01:59,164] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 29 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:01:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:59,164] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:01:59,267] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:01:59,267] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:02:01,269] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:01,269] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:01,270] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:03,272] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 29 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:01:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:03,273] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:03,376] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:03,376] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:02:05,378] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:05,379] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:05,379] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:07,604] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:02:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:07,604] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:07,707] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:07,708] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:02:07,708] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:02:07,708] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:07,708] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:07,708] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:09,746] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:02:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:09,746] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:09,849] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:09,850] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:02:09,850] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 19:02:09,851] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:02:09,851] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:09,851] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:09,851] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:02:12,033] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:12,033] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:12,136] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:12,136] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:02:12,137] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:02:12,137] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:02:12,137] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:12,137] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:02:24,255] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:02:24,255] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:24,357] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:02:24,357] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:02:27,360] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:02:27,361] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:02:27,361] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:02:27,361] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:27,361] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:27,361] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:02:29,620] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:29,621] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:29,724] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:29,724] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:02:29,724] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:02:29,725] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:02:29,725] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:29,725] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:02:31,829] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.659 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.416 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.396 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.396/0.490/0.659/0.120 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:02:31,829] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:31,931] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:02:31,931] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:02:31,932] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:02:36,937] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter41 [2019-08-22 19:02:36,937] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:36,937] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:36,937] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:39,005] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:02:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:39,006] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:39,108] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:39,109] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:02:39,109] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:39,109] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:39,109] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:41,193] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:02:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:41,193] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:41,296] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:41,297] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:02:41,297] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 19:02:41,297] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:02:41,297] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:41,297] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:41,297] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:43,209] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:43,209] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:43,312] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:43,313] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:02:43,313] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:43,313] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:43,313] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:45,343] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:02:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:45,343] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:45,446] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:45,447] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:02:45,447] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:02:45,447] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:02:47,450] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:47,450] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:47,450] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:49,502] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:02:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:49,503] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:49,605] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:49,606] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:02:51,608] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:51,608] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:51,609] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:53,629] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:02:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:53,630] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:53,733] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:53,733] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:02:55,736] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:02:55,736] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:02:55,736] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:02:58,014] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:02:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:58,015] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:02:58,118] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:02:58,118] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:00,121] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:00,121] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:00,121] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:02,314] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:02:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:02,315] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:02,418] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:02,418] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:04,421] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:04,421] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:04,421] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:06,547] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:02:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:06,548] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:06,651] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:06,651] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:08,654] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:08,654] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:08,654] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:10,842] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:02:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:10,843] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:10,946] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:10,946] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:12,948] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:12,948] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:12,948] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:14,956] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:03:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:14,957] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:15,060] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:15,060] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:15,061] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:03:15,061] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:15,061] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:15,061] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:17,078] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:03:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:17,079] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:17,182] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:17,182] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:17,183] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 19:03:17,183] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:03:17,183] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:17,184] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:17,184] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:03:19,367] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:19,368] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:19,471] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:19,471] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:03:19,471] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:03:19,471] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:03:19,472] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:19,472] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:03:31,583] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2008ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:03:31,583] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:31,685] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:03:31,685] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:03:34,688] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:03:34,689] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:03:34,689] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:03:34,689] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:34,689] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:34,690] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:03:36,873] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:36,873] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:36,976] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:36,976] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:03:36,977] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:03:36,977] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:03:36,977] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:36,977] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:03:39,081] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.762 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.688 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.345 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.345/0.598/0.762/0.182 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:03:39,082] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:39,183] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:03:39,184] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:03:39,184] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:03:44,189] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter42 [2019-08-22 19:03:44,189] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:44,190] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:44,190] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:46,263] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:03:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:46,263] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:46,366] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:46,367] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:46,367] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:46,367] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:46,367] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:48,398] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:03:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:48,399] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:48,502] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:48,502] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:48,502] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:03:48,502] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:03:48,503] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:48,503] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:48,503] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:50,602] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:50,603] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:50,706] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:50,706] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:03:50,706] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:50,706] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:50,706] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:52,811] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:03:50Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:52,812] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:52,915] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:52,915] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:52,915] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:03:52,915] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:03:54,918] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:54,918] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:54,918] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:03:57,284] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:03:50Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:57,284] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:03:57,387] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:03:57,388] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:03:59,390] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:03:59,390] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:03:59,390] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:01,419] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:03:50Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:01,419] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:01,522] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:01,523] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:03,525] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:03,525] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:03,525] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:05,542] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:03:50Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:05,542] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:05,645] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:05,645] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:07,648] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:07,648] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:07,648] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:09,671] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:04:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:09,671] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:09,774] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:09,775] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:11,777] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:11,777] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:11,777] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:13,811] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:04:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:13,811] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:13,914] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:13,915] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:15,917] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:15,917] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:15,918] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:17,958] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:04:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:17,958] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:18,061] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:18,061] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:20,064] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:20,064] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:20,064] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:22,111] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:04:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:22,112] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:22,215] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:22,215] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:22,216] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:04:22,216] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:22,216] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:22,216] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:24,253] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:04:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:24,254] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:24,356] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:24,357] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:24,357] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 19:04:24,357] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:04:24,357] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:24,358] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:24,358] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:04:26,564] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:26,564] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:26,667] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:26,668] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:04:26,668] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:04:26,668] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:04:26,668] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:26,668] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:04:38,787] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2015ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:04:38,787] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:38,889] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:04:38,889] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:04:41,892] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:04:41,892] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:04:41,893] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:04:41,893] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:41,893] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:41,893] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:04:44,091] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:44,091] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:44,194] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:44,194] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:04:44,194] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:04:44,195] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:04:44,195] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:44,195] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:04:46,299] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.59 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.502 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.312 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.312/0.803/1.596/0.566 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:04:46,299] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:46,401] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:04:46,401] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:04:46,401] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:04:51,406] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter43 [2019-08-22 19:04:51,406] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:51,407] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:51,407] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:53,532] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:04:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:53,533] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:53,635] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:53,636] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:53,636] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:53,636] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:53,636] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:55,682] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:04:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:55,682] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:55,785] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:55,786] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:55,786] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 19:04:55,786] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:04:55,786] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:55,786] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:55,786] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:57,743] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:57,743] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:57,845] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:57,846] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:04:57,846] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:04:57,846] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:04:57,846] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:04:59,855] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:04:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:59,855] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:04:59,958] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:04:59,959] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:04:59,959] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:04:59,959] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:05:01,961] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:01,962] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:01,962] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:05:04,030] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:04:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:04,031] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:04,133] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:04,134] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:05:06,136] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:06,136] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:06,136] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:05:08,194] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:04:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:08,194] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:08,297] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:08,297] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:05:10,299] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:10,300] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:10,300] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:05:12,358] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:05:08Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:12,359] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:12,462] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:12,462] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:05:14,464] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:14,465] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:14,465] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:05:16,499] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:05:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:16,500] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:16,603] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:16,604] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:05:18,605] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:18,605] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:18,605] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:05:20,655] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:05:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:20,655] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:20,758] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:20,759] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:05:22,761] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:22,762] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:22,762] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:05:25,048] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:05:13Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:25,048] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:25,151] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:25,151] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:05:27,154] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:27,154] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:27,154] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:05:29,167] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:05:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:29,167] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:29,270] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:29,271] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:05:29,271] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:05:29,271] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:29,271] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:29,272] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:05:31,290] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:05:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:31,291] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:31,393] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:31,394] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:05:31,394] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 19:05:31,394] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:05:31,394] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:31,395] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:31,395] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:05:33,592] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:33,593] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:33,696] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:33,696] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:05:33,696] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:05:33,697] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:05:33,697] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:33,697] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:05:45,807] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2006ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:05:45,807] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:45,909] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:05:45,909] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:05:48,912] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:05:48,912] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:05:48,913] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:05:48,913] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:48,913] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:48,913] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:05:51,190] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:51,190] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:51,293] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:05:51,293] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:05:51,293] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:05:51,294] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:05:51,294] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:51,294] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:05:53,397] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.766 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.528 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.454 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.454/0.582/0.766/0.135 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:05:53,397] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:05:53,499] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:05:53,499] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:05:53,499] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:05:58,504] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter44 [2019-08-22 19:05:58,504] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:05:58,505] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:05:58,505] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:00,548] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:05:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:00,548] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:00,651] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:00,651] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:00,651] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:00,652] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:00,652] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:02,840] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:05:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:02,840] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:02,943] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:02,943] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:02,943] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:06:02,944] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:06:02,944] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:02,944] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:02,944] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:04,871] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:04,871] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:04,973] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:04,974] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:06:04,974] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:04,974] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:04,974] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:07,244] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:07,244] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:07,348] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:07,348] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:07,348] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:06:07,348] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:06:09,351] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:09,351] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:09,351] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:11,557] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:11,557] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:11,660] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:11,661] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:13,663] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:13,663] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:13,663] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:15,696] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:15,696] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:15,799] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:15,800] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:17,802] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:17,802] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:17,803] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:19,845] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:19,846] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:19,949] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:19,949] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:21,951] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:21,952] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:21,952] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:23,988] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:20Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:23,988] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:24,091] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:24,091] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:26,094] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:26,094] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:26,094] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:28,102] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:20Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:28,103] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:28,205] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:28,206] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:30,208] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:30,209] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:30,209] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:32,225] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:29Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:32,225] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:32,328] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:32,328] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:34,331] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:34,331] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:34,331] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:36,466] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:29Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:36,467] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:36,570] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:36,570] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:38,573] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:38,573] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:38,573] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:40,648] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:06:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:40,648] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:40,751] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:40,752] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:40,752] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:06:40,752] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:40,752] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:40,752] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:42,767] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:06:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:42,768] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:42,871] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:42,871] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:42,871] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 19:06:42,872] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:06:42,872] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:42,872] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:42,872] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:06:45,065] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:45,065] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:45,168] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:45,169] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:06:45,169] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:06:45,169] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:06:45,169] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:45,169] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:06:47,284] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.942 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.307 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.951 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.307/0.733/0.951/0.302 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:06:47,284] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:47,385] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:06:47,386] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:06:47,386] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:06:52,391] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter45 [2019-08-22 19:06:52,392] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:52,392] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:52,392] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:54,637] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:06:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:54,637] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:54,740] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:54,741] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:54,741] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:54,741] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:54,741] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:56,775] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:06:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:56,775] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:56,878] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:56,878] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:06:56,879] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 19:06:56,879] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:06:56,879] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:56,879] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:56,879] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:06:58,821] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:58,821] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:06:58,924] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:06:58,925] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:06:58,925] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:06:58,925] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:06:58,925] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:00,916] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:00,917] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:01,020] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:01,020] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:01,020] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:07:01,020] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:07:03,022] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:03,022] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:03,023] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:05,367] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:05,367] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:05,470] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:05,471] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:07,473] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:07,473] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:07,473] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:09,510] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:09,511] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:09,613] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:09,614] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:11,616] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:11,616] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:11,617] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:13,795] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:13,795] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:13,898] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:13,899] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:15,901] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:15,901] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:15,902] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:18,453] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:06:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:18,454] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:18,557] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:18,558] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:20,560] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:20,560] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:20,560] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:22,626] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:07:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:22,626] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:22,729] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:22,730] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:24,732] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:24,732] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:24,733] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:26,795] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 20 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:07:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:26,795] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:26,898] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:26,898] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:28,901] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:28,901] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:28,901] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:31,040] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 20 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:07:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:31,040] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:31,143] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:31,143] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:33,146] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:33,146] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:33,146] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:35,349] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 20 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:07:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:35,349] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:35,452] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:35,453] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:37,453] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:37,453] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:37,454] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:39,525] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:07:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:39,525] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:39,628] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:39,629] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:39,629] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:07:39,629] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:39,629] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:39,629] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:07:41,973] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:07:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:41,973] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:42,076] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:42,077] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:07:42,077] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 19:07:42,077] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:07:42,077] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:42,077] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:42,078] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:07:44,260] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:44,260] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:44,363] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:07:44,363] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:07:44,363] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:07:44,364] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:07:44,364] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:44,364] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:07:56,475] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2007ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:07:56,475] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:07:56,577] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:07:56,577] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:07:59,581] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:07:59,581] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:07:59,581] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:07:59,581] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:07:59,582] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:07:59,582] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:08:01,760] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:01,760] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:01,863] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:01,863] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:08:01,863] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:08:01,864] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:08:01,864] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:01,864] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:08:03,977] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.783 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.665 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.379 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.379/0.609/0.783/0.169 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:08:03,978] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:04,079] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:08:04,079] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:08:04,080] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:08:09,085] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter46 [2019-08-22 19:08:09,085] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:09,085] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:09,085] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:11,199] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:07:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:11,200] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:11,303] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:11,303] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:11,303] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:11,304] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:11,304] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:13,317] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:07:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:13,317] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:13,420] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:13,420] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:13,420] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 19:08:13,421] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:08:13,421] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:13,421] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:13,421] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:15,296] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:15,296] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:15,399] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:15,399] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:08:15,400] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:15,400] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:15,400] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:17,484] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:08:15Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:17,485] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:17,588] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:17,588] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:17,588] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:08:17,588] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:08:19,591] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:19,591] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:19,591] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:21,849] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:08:15Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:21,849] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:21,952] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:21,952] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:23,955] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:23,955] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:23,955] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:26,032] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:08:15Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:26,033] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:26,136] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:26,136] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:28,139] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:28,139] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:28,139] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:30,304] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:08:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:30,304] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:30,407] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:30,407] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:32,410] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:32,410] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:32,410] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:34,499] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:08:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:34,499] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:34,603] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:34,603] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:36,605] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:36,606] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:36,606] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:38,648] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:08:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:38,649] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:38,752] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:38,752] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:40,754] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:40,754] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:40,755] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:43,164] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:08:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:43,164] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:43,267] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:43,267] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:45,269] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:45,270] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:45,270] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:47,268] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:08:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:47,269] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:47,372] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:47,373] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:47,373] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:08:47,373] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:47,373] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:47,373] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:08:49,402] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:08:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:49,402] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:49,505] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:49,505] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:08:49,506] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-0 [2019-08-22 19:08:49,506] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:08:49,506] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:49,506] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:49,506] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:08:51,691] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:51,691] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:51,794] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:08:51,795] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:08:51,795] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:08:51,795] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:08:51,795] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:51,795] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:08:53,901] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.50 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.407 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.217 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.217/0.710/1.508/0.569 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:08:53,901] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:08:54,003] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:08:54,003] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:08:54,004] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:08:59,009] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter47 [2019-08-22 19:08:59,009] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:08:59,009] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:08:59,010] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:01,084] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:08:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:01,084] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:01,187] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:01,187] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:01,187] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:01,188] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:01,188] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:03,195] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:08:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:03,195] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:03,298] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:03,298] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:03,299] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 19:09:03,299] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:09:03,299] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:03,299] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:03,299] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:05,846] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:05,846] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:05,950] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:05,950] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:09:05,950] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:05,950] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:05,950] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:08,005] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:08,005] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:08,109] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:08,109] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:08,109] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:09:08,109] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:09:10,112] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:10,112] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:10,112] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:12,145] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:12,145] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:12,248] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:12,249] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:14,251] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:14,251] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:14,252] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:16,279] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:16,279] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:16,382] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:16,383] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:18,385] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:18,385] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:18,385] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:20,563] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:20,563] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:20,666] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:20,667] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:22,668] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:22,668] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:22,668] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:24,671] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:24,671] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:24,774] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:24,774] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:26,776] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:26,776] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:26,776] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:28,802] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:28,802] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:28,905] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:28,906] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:30,908] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:30,908] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:30,908] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:32,900] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:32,900] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:33,003] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:33,004] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:35,006] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:35,006] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:35,007] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:37,040] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:37,041] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:37,144] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:37,144] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:39,146] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:39,147] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:39,147] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:41,159] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:41,160] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:41,263] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:41,263] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:43,265] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:43,266] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:43,266] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:45,495] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:45,496] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:45,599] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:45,599] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:47,601] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:47,601] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:47,601] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:49,668] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:09:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:49,668] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:49,771] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:49,771] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:51,774] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:51,774] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:51,774] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:53,792] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:09:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:53,793] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:53,896] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:53,896] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:53,896] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:09:53,896] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:53,897] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:53,897] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:09:56,079] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:09:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:56,079] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:56,183] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:56,183] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:09:56,183] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 19:09:56,183] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:09:56,183] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:09:56,184] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:56,184] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:09:58,320] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:58,320] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:09:58,423] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:09:58,423] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:09:58,424] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:09:58,424] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:09:58,424] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:09:58,424] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:10:01,529] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.542 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.274 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 2 received, 33% packet loss, time 2001ms rtt min/avg/max/mdev = 0.274/0.408/0.542/0.134 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:10:01,529] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:01,631] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:10:01,631] 2558 WARNING MainThread network_helper.ping_server:: Some packets dropped when ping from 128.224.186.181 ssh session to 192.168.104.70. Packet loss rate: 33% [2019-08-22 19:10:01,631] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 33} [2019-08-22 19:10:06,636] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter48 [2019-08-22 19:10:06,636] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:06,637] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:06,637] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:08,684] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:09:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:08,684] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:08,787] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:08,788] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:08,788] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:08,788] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:08,788] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:10,845] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:09:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:10,845] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:10,948] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:10,948] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:10,948] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:10:10,948] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:10:10,949] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:10,949] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:10,949] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:12,829] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:12,830] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:12,933] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:12,933] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:10:12,933] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:12,934] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:12,934] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:15,023] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:10:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:15,024] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:15,127] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:15,127] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:15,127] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:10:15,127] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:10:17,130] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:17,130] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:17,130] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:19,371] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:10:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:19,371] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:19,475] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:19,475] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:21,477] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:21,478] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:21,478] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:23,522] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:10:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:23,522] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:23,625] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:23,625] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:25,628] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:25,628] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:25,628] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:27,640] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:10:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:27,640] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:27,743] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:27,743] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:29,745] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:29,746] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:29,746] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:31,804] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:10:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:31,804] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:31,908] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:31,908] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:33,910] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:33,911] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:33,911] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:35,959] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:10:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:35,960] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:36,062] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:36,063] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:38,065] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:38,065] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:38,065] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:40,263] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:10:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:40,263] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:40,367] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:40,367] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:42,369] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:42,370] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:42,370] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:44,387] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:10:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:44,387] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:44,490] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:44,491] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:44,491] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:10:44,491] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:44,492] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:44,492] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:10:46,569] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:10:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:46,569] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:46,672] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:46,672] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:10:46,672] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 19:10:46,672] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:10:46,673] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:10:46,673] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:46,673] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:10:48,841] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:48,841] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:10:48,944] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:10:48,945] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:10:48,945] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:10:48,945] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:10:48,945] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:10:48,945] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:11:01,063] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:11:01,063] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:01,165] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:11:01,165] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:11:04,168] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:11:04,168] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:11:04,169] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:11:04,169] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:04,169] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:04,169] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:11:06,366] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:06,366] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:06,469] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:06,470] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:11:06,470] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:11:06,470] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:11:06,471] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:06,471] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:11:08,576] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.15 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.518 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.382 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.382/0.683/1.151/0.336 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:11:08,576] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:08,680] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:11:08,680] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:11:08,680] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:11:13,685] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter49 [2019-08-22 19:11:13,686] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:13,686] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:13,686] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:15,851] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:10:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:15,852] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:15,955] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:15,955] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:15,955] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:15,955] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:15,956] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:17,961] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:10:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:17,961] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:18,064] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:18,064] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:18,064] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 19:11:18,064] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:11:18,065] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:18,065] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:18,065] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:19,993] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:19,993] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:20,096] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:20,096] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:11:20,096] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:20,097] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:20,097] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:22,148] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:11:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:22,149] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:22,252] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:22,252] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:22,252] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:11:22,252] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:11:24,255] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:24,255] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:24,255] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:26,303] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:11:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:26,303] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:26,406] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:26,407] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:28,409] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:28,409] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:28,409] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:30,450] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:11:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:30,450] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:30,553] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:30,554] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:32,556] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:32,556] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:32,557] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:34,583] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:11:30Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:34,583] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:34,686] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:34,687] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:36,689] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:36,689] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:36,689] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:38,711] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:11:36Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:38,711] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:38,814] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:38,814] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:40,816] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:40,817] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:40,817] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:42,841] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:11:36Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:42,841] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:42,944] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:42,944] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:44,947] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:44,947] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:44,947] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:47,367] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:11:36Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:47,367] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:47,470] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:47,470] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:49,472] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:49,472] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:49,473] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:51,508] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:11:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:51,508] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:51,611] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:51,612] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:51,612] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:11:51,612] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:51,612] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:51,612] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:11:53,652] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:11:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:53,653] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:53,756] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:53,756] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:11:53,756] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 19:11:53,757] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:11:53,757] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:11:53,757] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:53,757] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:11:55,967] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:55,968] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:11:56,071] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:11:56,071] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:11:56,072] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:11:56,072] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:11:56,072] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:11:56,072] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:12:08,183] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2008ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:12:08,184] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:08,285] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:12:08,285] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:12:11,289] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:12:11,289] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:12:11,289] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:12:11,289] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:11,290] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:11,290] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:12:13,629] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:13,630] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:13,732] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:13,733] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:12:13,733] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:12:13,733] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:12:13,733] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:13,733] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:12:15,837] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.670 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.515 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.428 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.428/0.537/0.670/0.103 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:12:15,837] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:15,939] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:12:15,939] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:12:15,939] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:12:20,944] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter50 [2019-08-22 19:12:20,944] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:20,945] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:20,945] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:23,124] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:11:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:23,125] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:23,228] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:23,228] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:12:23,229] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:23,229] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:23,229] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:25,294] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:11:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:25,295] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:25,397] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:25,398] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:12:25,398] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:12:25,398] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:12:25,398] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:25,398] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:25,399] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:27,290] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:27,290] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:27,393] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:27,393] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:12:27,394] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:27,394] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:27,394] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:29,461] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:12:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:29,461] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:29,564] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:29,564] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:12:29,564] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:12:29,565] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:12:31,567] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:31,567] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:31,567] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:33,895] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:12:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:33,895] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:33,998] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:33,998] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:12:36,000] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:36,000] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:36,000] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:38,191] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:12:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:38,191] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:38,293] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:38,294] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:12:40,296] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:40,296] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:40,297] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:42,332] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:12:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:42,332] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:42,435] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:42,435] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:12:44,438] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:44,438] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:44,438] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:46,479] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:12:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:46,479] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:46,582] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:46,583] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:12:48,585] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:48,585] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:48,585] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:50,619] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 83 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:12:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:50,619] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:50,722] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:50,723] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:12:52,725] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:52,725] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:52,725] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:54,750] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 83 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:12:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:54,751] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:54,853] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:54,854] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:12:56,856] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:12:56,856] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:12:56,857] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:12:58,913] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 83 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:12:49Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:58,913] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:12:59,016] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:12:59,016] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:01,018] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:01,019] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:01,019] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:03,025] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:13:00Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:03,026] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:03,129] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:03,129] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:03,129] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:13:03,130] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:03,130] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:03,130] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:05,185] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:13:00Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:05,185] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:05,290] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:05,290] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:05,291] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 19:13:05,291] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:13:05,291] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:05,291] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:05,291] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:13:07,482] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:07,482] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:07,585] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:07,585] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:13:07,586] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:13:07,586] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:13:07,586] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:07,586] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:13:09,691] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.66 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.719 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.393 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.393/0.927/1.669/0.541 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:13:09,691] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:09,793] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:13:09,793] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:13:09,794] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:13:14,799] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter51 [2019-08-22 19:13:14,799] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:14,799] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:14,800] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:16,836] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:13:00Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:16,836] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:16,939] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:16,940] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:16,940] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:16,940] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:16,940] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:18,974] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:13:00Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:18,975] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:19,078] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:19,078] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:19,078] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 19:13:19,079] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:13:19,079] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:19,079] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:19,079] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:20,992] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:20,992] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:21,095] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:21,095] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:13:21,096] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:21,096] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:21,096] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:23,198] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:13:20Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:23,198] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:23,301] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:23,301] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:23,302] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:13:23,302] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:13:25,304] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:25,304] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:25,305] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:27,535] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:13:20Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:27,536] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:27,639] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:27,639] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:29,642] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:29,642] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:29,642] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:31,690] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:13:20Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:31,690] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:31,793] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:31,794] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:33,795] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:33,795] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:33,795] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:35,863] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:13:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:35,863] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:35,966] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:35,967] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:37,969] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:37,969] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:37,969] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:39,995] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:13:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:39,996] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:40,099] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:40,099] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:42,101] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:42,102] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:42,102] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:44,311] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:13:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:44,311] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:44,414] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:44,415] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:46,417] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:46,418] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:46,418] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:48,638] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:13:37Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:48,638] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:48,741] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:48,742] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:50,744] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:50,744] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:50,744] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:52,806] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:13:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:52,806] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:52,909] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:52,910] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:52,910] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:13:52,910] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:52,910] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:52,910] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:13:54,947] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:13:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:54,947] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:55,050] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:55,050] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:13:55,050] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 19:13:55,051] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:13:55,051] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:13:55,051] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:55,051] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:13:57,249] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:57,250] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:13:57,353] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:13:57,353] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:13:57,353] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:13:57,353] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:13:57,354] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:13:57,354] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:14:09,457] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:14:09,457] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:09,559] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:14:09,559] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:14:12,562] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:14:12,563] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:14:12,563] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:14:12,563] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:12,563] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:12,563] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:14:14,885] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:14,886] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:14,989] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:14,989] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:14:14,989] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:14:14,989] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:14:14,990] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:14,990] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:14:17,094] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.37 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.475 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.364 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.364/0.739/1.379/0.455 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:14:17,094] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:17,196] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:14:17,196] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:14:17,197] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:14:22,202] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter52 [2019-08-22 19:14:22,202] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:22,202] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:22,202] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:24,250] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:13:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:24,250] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:24,353] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:24,353] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:24,354] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:24,354] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:24,354] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:26,415] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:13:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:26,415] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:26,518] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:26,518] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:26,519] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 19:14:26,519] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:14:26,519] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:26,519] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:26,519] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:28,442] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:28,443] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:28,546] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:28,546] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:14:28,546] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:28,547] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:28,547] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:30,696] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:14:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:30,697] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:30,800] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:30,801] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:30,801] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:14:30,801] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:14:32,803] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:32,804] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:32,804] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:34,857] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:14:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:34,857] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:34,960] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:34,960] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:36,962] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:36,963] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:36,963] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:39,012] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:14:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:39,012] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:39,115] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:39,116] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:41,118] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:41,118] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:41,118] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:43,150] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:14:39Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:43,151] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:43,254] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:43,255] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:45,257] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:45,257] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:45,257] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:47,272] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:14:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:47,273] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:47,376] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:47,376] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:49,378] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:49,378] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:49,378] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:51,432] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:14:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:51,432] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:51,535] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:51,536] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:53,538] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:53,538] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:53,539] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:55,702] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:14:44Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:55,702] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:55,805] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:55,806] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:57,807] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:57,807] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:57,808] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:14:59,842] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:14:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:59,842] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:14:59,945] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:14:59,946] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:14:59,946] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:14:59,946] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:14:59,946] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:14:59,946] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:02,009] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:14:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:02,009] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:02,113] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:02,113] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:15:02,113] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 19:15:02,113] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:15:02,114] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:02,114] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:02,114] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:15:04,334] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:04,334] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:04,437] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:04,437] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:15:04,437] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:15:04,438] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:15:04,438] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:04,438] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:15:16,541] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:15:16,541] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:16,643] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:15:16,643] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:15:19,647] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:15:19,647] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:15:19,647] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:15:19,647] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:19,648] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:19,648] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:15:21,842] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:21,842] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:21,945] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:21,946] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:15:21,946] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:15:21,946] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:15:21,946] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:21,946] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:15:24,051] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.29 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.564 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.277 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.277/0.712/1.295/0.428 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:15:24,051] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:24,153] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:15:24,153] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:15:24,153] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:15:29,159] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter53 [2019-08-22 19:15:29,159] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:29,159] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:29,159] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:31,193] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:14:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:31,193] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:31,296] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:31,297] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:15:31,297] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:31,297] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:31,297] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:33,318] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:14:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:33,319] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:33,422] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:33,422] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:15:33,423] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:15:33,423] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:15:33,423] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:33,423] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:33,423] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:35,427] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:35,427] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:35,531] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:35,532] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:15:35,532] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:35,532] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:35,533] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:37,679] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:15:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:37,679] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:37,782] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:37,782] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:15:37,783] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:15:37,783] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:15:39,785] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:39,785] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:39,786] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:41,995] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:15:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:41,995] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:42,098] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:42,099] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:15:44,101] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:44,102] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:44,102] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:46,294] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:15:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:46,294] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:46,397] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:46,398] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:15:48,400] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:48,401] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:48,401] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:50,426] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:15:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:50,426] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:50,529] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:50,530] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:15:52,532] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:52,532] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:52,533] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:54,555] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:15:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:54,555] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:54,658] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:54,659] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:15:56,661] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:15:56,662] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:15:56,662] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:15:58,694] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 15 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:15:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:58,695] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:15:58,799] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:15:58,800] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:00,802] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:00,802] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:00,802] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:02,881] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 15 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:15:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:02,881] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:02,984] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:02,984] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:04,987] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:04,987] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:04,987] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:07,016] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 15 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:15:57Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:07,016] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:07,119] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:07,119] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:09,122] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:09,122] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:09,122] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:11,161] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:16:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:11,161] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:11,264] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:11,265] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:11,265] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:16:11,265] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:11,265] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:11,265] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:13,312] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:16:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:13,312] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:13,415] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:13,416] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:13,416] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 19:16:13,416] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:16:13,416] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:13,416] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:13,417] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:16:15,606] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:15,606] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:15,709] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:15,709] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:16:15,709] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:16:15,710] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:16:15,710] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:15,710] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:16:17,814] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.07 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.569 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.322 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.322/0.654/1.073/0.313 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:16:17,814] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:17,916] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:16:17,916] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:16:17,916] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:16:22,920] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter54 [2019-08-22 19:16:22,921] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:22,921] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:22,921] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:24,960] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:16:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:24,960] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:25,063] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:25,064] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:25,064] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:25,064] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:25,064] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:27,102] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:16:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:27,103] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:27,206] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:27,206] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:27,207] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 19:16:27,207] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:16:27,207] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:27,207] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:27,207] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:29,082] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:29,082] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:29,185] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:29,185] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:16:29,185] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:29,185] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:29,185] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:31,238] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:16:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:31,238] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:31,342] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:31,342] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:31,342] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:16:31,342] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:16:33,345] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:33,345] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:33,345] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:35,618] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:16:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:35,618] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:35,721] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:35,721] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:37,723] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:37,723] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:37,723] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:39,741] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:16:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:39,742] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:39,844] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:39,845] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:41,845] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:41,845] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:41,845] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:43,849] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:16:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:43,850] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:43,953] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:43,953] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:45,956] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:45,956] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:45,956] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:47,982] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:16:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:47,983] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:48,086] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:48,086] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:50,088] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:50,088] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:50,089] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:52,355] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:16:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:52,355] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:52,458] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:52,459] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:54,461] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:54,461] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:54,461] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:16:56,530] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:16:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:56,530] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:16:56,633] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:16:56,634] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:16:58,636] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:16:58,636] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:16:58,636] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:00,690] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:16:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:00,690] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:00,793] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:00,794] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:00,794] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:17:00,794] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:00,794] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:00,794] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:02,803] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:16:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:02,803] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:02,906] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:02,907] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:02,907] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 19:17:02,907] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:17:02,907] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:02,907] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:02,907] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:17:05,091] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:05,091] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:05,194] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:05,195] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:17:05,195] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:17:05,195] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:17:05,195] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:05,195] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:17:07,301] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=2.45 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=1.89 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.346 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.346/1.564/2.452/0.890 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:17:07,301] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:07,403] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:17:07,403] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:17:07,403] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:17:12,409] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter55 [2019-08-22 19:17:12,409] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:12,409] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:12,409] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:14,595] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:16:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:14,595] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:14,698] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:14,699] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:14,699] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:14,699] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:14,700] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:16,783] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:16:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:16,783] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:16,886] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:16,886] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:16,886] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:17:16,887] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:17:16,887] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:16,887] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:16,887] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:18,757] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:18,757] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:18,860] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:18,861] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:17:18,861] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:18,861] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:18,861] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:20,903] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:17:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:20,903] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:21,006] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:21,007] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:21,007] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:17:21,007] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:17:23,009] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:23,009] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:23,010] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:25,025] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:17:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:25,025] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:25,128] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:25,129] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:27,131] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:27,132] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:27,132] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:29,558] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:17:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:29,558] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:29,661] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:29,662] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:31,663] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:31,663] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:31,663] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:33,723] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:17:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:33,723] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:33,826] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:33,826] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:35,828] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:35,828] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:35,828] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:37,914] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:17:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:37,914] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:38,017] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:38,018] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:40,020] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:40,020] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:40,020] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:42,059] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:17:39Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:42,059] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:42,162] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:42,163] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:44,164] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:44,164] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:44,164] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:46,203] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:17:39Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:46,204] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:46,307] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:46,307] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:48,310] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:48,310] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:48,310] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:50,382] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:17:39Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:50,382] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:50,485] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:50,486] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:52,488] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:52,488] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:52,488] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:54,551] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:17:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:54,551] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:54,654] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:54,655] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:54,655] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:17:54,655] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:54,655] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:54,655] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:17:56,663] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:17:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:56,663] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:56,766] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:56,767] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:17:56,767] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 19:17:56,767] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:17:56,767] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:17:56,767] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:56,767] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:17:58,962] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:58,962] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:17:59,065] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:17:59,065] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:17:59,066] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:17:59,066] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:17:59,066] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:17:59,066] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:18:01,171] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.25 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.419 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.299 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.299/0.658/1.256/0.425 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:18:01,171] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:01,273] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:18:01,273] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:18:01,273] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:18:06,278] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter56 [2019-08-22 19:18:06,279] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:06,279] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:06,279] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:08,288] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:17:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:08,289] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:08,392] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:08,392] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:08,392] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:08,393] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:08,393] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:10,565] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:17:53Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:10,565] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:10,669] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:10,669] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:10,669] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 19:18:10,669] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:18:10,669] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:10,670] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:10,670] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:12,914] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:12,914] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:13,017] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:13,017] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:18:13,018] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:13,018] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:13,018] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:15,063] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:18:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:15,063] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:15,166] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:15,167] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:15,167] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:18:15,167] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:18:17,169] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:17,169] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:17,170] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:19,245] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:18:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:19,245] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:19,348] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:19,348] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:21,351] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:21,351] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:21,351] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:23,391] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:18:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:23,391] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:23,494] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:23,494] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:25,496] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:25,497] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:25,497] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:27,528] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:18:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:27,528] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:27,631] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:27,631] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:29,634] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:29,634] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:29,634] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:31,817] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:18:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:31,817] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:31,920] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:31,920] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:33,923] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:33,923] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:33,923] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:35,972] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:18:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:35,972] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:36,075] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:36,076] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:38,078] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:38,078] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:38,079] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:40,174] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:18:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:40,174] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:40,277] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:40,277] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:42,280] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:42,280] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:42,280] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:44,309] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:18:41Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:44,309] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:44,412] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:44,413] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:44,413] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:18:44,413] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:44,413] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:44,413] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:18:46,462] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:18:41Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:46,462] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:46,566] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:46,566] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:18:46,566] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 19:18:46,566] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:18:46,567] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:18:46,567] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:46,567] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:18:48,785] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:48,785] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:18:48,888] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:18:48,888] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:18:48,889] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:18:48,889] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:18:48,889] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:18:48,889] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:19:00,993] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=1.56 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 1 received, 66% packet loss, time 2000ms rtt min/avg/max/mdev = 1.561/1.561/1.561/0.000 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:19:00,993] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:01,095] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:19:01,095] 2558 WARNING MainThread network_helper.ping_server:: Some packets dropped when ping from 128.224.186.181 ssh session to 192.168.104.70. Packet loss rate: 66% [2019-08-22 19:19:01,095] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 66} [2019-08-22 19:19:06,101] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter57 [2019-08-22 19:19:06,101] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:06,101] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:06,101] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:08,178] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:18:41Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:08,179] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:08,282] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:08,282] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:08,282] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:08,283] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:08,283] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:10,309] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:18:41Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:10,309] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:10,412] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:10,413] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:10,413] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:19:10,413] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:19:10,413] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:10,413] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:10,413] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:12,332] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:12,332] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:12,435] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:12,435] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:19:12,436] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:12,436] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:12,436] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:14,675] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:14,675] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:14,778] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:14,778] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:14,779] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:19:14,779] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:19:16,781] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:16,781] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:16,781] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:18,940] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:18,940] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:19,043] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:19,044] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:21,046] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:21,047] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:21,047] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:23,054] 423 DEBUG MainThread ssh.expect :: Output: openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default -- os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:12Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:23,054] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:23,157] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:23,158] 128 DEBUG MainThread table_parser.__table :: skipping invalid table line: openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default -- os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:19:23,158] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:25,160] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:25,160] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:25,161] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:27,196] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:27,196] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:27,300] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:27,300] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:29,302] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:29,303] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:29,303] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:31,343] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:26Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:31,343] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:31,446] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:31,446] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:33,449] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:33,449] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:33,449] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:35,508] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:31Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:35,508] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:35,611] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:35,611] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:37,614] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:37,614] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:37,614] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:39,647] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:36Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:39,647] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:39,749] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:39,750] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:41,752] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:41,752] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:41,753] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:43,790] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:36Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:43,790] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:43,893] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:43,893] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:45,896] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:45,896] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:45,896] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:47,919] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:36Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:47,919] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:48,022] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:48,022] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:50,025] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:50,025] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:50,025] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:52,145] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:19:36Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:52,145] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:52,248] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:52,249] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:54,251] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:54,251] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:54,251] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:56,298] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:19:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:56,299] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:56,402] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:56,402] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:56,402] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:19:56,403] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:56,403] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:56,403] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:19:58,444] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:19:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:58,444] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:19:58,547] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:19:58,548] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:19:58,548] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 19:19:58,548] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:19:58,548] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:19:58,548] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:19:58,549] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:20:00,746] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:00,746] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:00,849] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:00,850] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:20:00,850] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:20:00,850] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:20:00,850] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:00,850] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:20:02,953] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.948 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.516 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.390 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1999ms rtt min/avg/max/mdev = 0.390/0.618/0.948/0.238 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:20:02,954] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:03,055] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:20:03,056] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:20:03,056] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:20:08,061] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter58 [2019-08-22 19:20:08,061] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:08,062] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:08,062] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:10,107] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:19:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:10,107] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:10,210] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:10,210] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:10,211] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:10,211] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:10,211] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:12,258] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:19:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:12,258] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:12,361] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:12,361] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:12,362] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 19:20:12,362] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:20:12,362] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:12,362] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:12,363] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:14,471] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:14,471] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:14,574] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:14,575] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:20:14,575] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:14,575] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:14,575] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:16,646] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:20:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:16,646] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:16,749] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:16,750] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:16,750] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:20:16,750] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:20:18,752] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:18,753] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:18,753] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:20,801] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:20:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:20,801] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:20,904] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:20,904] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:22,907] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:22,907] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:22,907] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:24,955] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:20:14Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:24,956] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:25,059] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:25,059] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:27,061] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:27,062] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:27,062] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:29,089] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:20:25Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:29,089] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:29,192] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:29,192] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:31,195] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:31,195] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:31,195] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:33,213] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:20:30Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:33,214] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:33,317] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:33,317] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:35,320] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:35,320] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:35,320] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:37,364] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:20:30Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:37,364] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:37,467] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:37,468] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:39,470] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:39,470] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:39,470] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:41,739] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:20:30Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:41,739] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:41,842] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:41,842] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:43,845] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:43,845] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:43,845] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:45,868] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:20:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:45,868] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:45,971] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:45,972] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:45,972] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:20:45,972] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:45,972] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:45,972] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:20:48,005] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:20:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:48,005] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:48,108] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:48,109] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:20:48,109] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 19:20:48,109] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:20:48,109] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:20:48,109] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:48,109] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:20:50,336] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:50,336] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:20:50,439] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:20:50,439] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:20:50,440] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:20:50,440] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:20:50,440] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:20:50,440] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:21:02,559] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2015ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:21:02,559] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:02,661] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:21:02,662] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:21:05,665] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:21:05,665] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:21:05,666] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:21:05,666] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:05,666] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:05,667] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:21:07,844] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:07,844] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:07,947] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:07,947] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:21:07,948] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:21:07,948] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:21:07,948] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:07,948] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:21:10,050] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.825 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.583 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.398 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.398/0.602/0.825/0.174 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:21:10,050] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:10,152] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:21:10,152] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:21:10,152] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:21:15,157] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter59 [2019-08-22 19:21:15,158] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:15,158] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:15,158] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:17,208] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:20:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:17,209] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:17,312] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:17,312] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:17,312] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:17,313] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:17,313] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:19,338] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:20:45Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:19,338] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:19,441] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:19,442] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:19,442] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:21:19,442] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:21:19,442] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:19,443] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:19,443] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:21,331] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:21,331] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:21,434] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:21,434] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:21:21,435] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:21,435] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:21,435] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:23,518] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:21:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:23,519] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:23,622] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:23,622] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:23,622] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:21:23,623] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:21:25,625] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:25,625] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:25,625] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:27,853] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:21:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:27,853] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:27,956] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:27,956] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:29,957] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:29,957] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:29,958] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:31,967] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:21:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:31,967] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:32,071] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:32,071] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:34,073] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:34,074] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:34,074] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:36,122] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:21:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:36,123] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:36,225] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:36,226] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:38,227] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:38,227] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:38,227] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:40,263] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:21:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:40,264] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:40,367] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:40,367] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:42,369] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:42,370] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:42,370] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:44,443] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:21:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:44,444] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:44,547] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:44,547] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:46,549] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:46,550] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:46,550] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:48,620] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:21:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:48,621] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:48,723] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:48,724] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:50,726] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:50,726] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:50,727] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:52,865] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:21:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:52,866] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:52,968] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:52,969] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:52,969] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:21:52,969] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:52,969] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:52,969] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:21:54,990] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:21:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:54,991] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:55,094] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:55,094] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:21:55,094] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 19:21:55,094] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:21:55,094] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:21:55,095] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:55,095] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:21:57,304] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:57,304] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:57,407] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:21:57,407] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:21:57,408] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:21:57,408] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:21:57,408] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:21:57,408] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:21:59,510] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.751 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.343 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.227 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.227/0.440/0.751/0.225 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:21:59,511] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:21:59,612] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:21:59,613] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:21:59,613] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:22:04,618] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter60 [2019-08-22 19:22:04,619] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:04,619] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:04,619] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:06,704] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:21:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:06,704] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:06,807] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:06,807] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:06,807] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:06,808] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:06,808] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:08,802] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:21:52Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:08,803] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:08,906] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:08,906] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:08,906] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 19:22:08,906] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:22:08,906] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:08,907] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:08,907] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:10,787] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:10,787] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:10,890] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:10,890] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:22:10,890] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:10,891] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:10,891] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:13,163] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:22:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:13,164] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:13,267] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:13,267] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:13,267] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:22:13,268] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:22:15,270] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:15,270] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:15,270] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:17,489] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:22:10Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:17,489] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:17,592] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:17,593] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:19,595] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:19,595] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:19,595] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:21,645] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:22:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:21,645] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:21,748] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:21,748] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:23,750] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:23,751] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:23,751] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:25,798] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:22:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:25,798] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:25,901] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:25,901] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:27,903] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:27,904] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:27,904] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:29,919] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:22:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:29,920] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:30,022] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:30,023] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:32,025] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:32,025] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:32,025] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:34,037] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:22:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:34,037] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:34,140] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:34,141] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:36,142] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:36,142] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:36,142] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:38,162] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:22:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:38,163] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:38,266] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:38,266] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:40,268] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:40,269] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:40,269] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:42,477] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:22:28Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:42,477] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:42,580] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:42,580] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:44,583] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:44,583] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:44,583] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:46,662] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:22:46Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:46,662] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:46,765] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:46,765] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:46,766] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:22:46,766] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:46,766] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:46,766] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:22:48,815] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:22:46Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:48,816] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:48,918] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:48,919] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:22:48,919] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-1 [2019-08-22 19:22:48,919] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:22:48,919] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:22:48,920] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:48,920] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:22:51,097] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:51,097] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:22:51,200] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:22:51,201] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:22:51,201] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:22:51,201] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:22:51,201] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:22:51,201] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:23:03,319] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2014ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:23:03,319] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:03,421] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:23:03,421] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:23:06,424] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:23:06,425] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:23:06,425] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:23:06,425] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:06,425] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:06,426] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:23:08,611] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:08,612] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:08,715] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:08,715] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:23:08,715] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:23:08,716] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:23:08,716] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:08,716] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:23:10,817] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.663 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.377 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.274 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.274/0.438/0.663/0.164 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:23:10,818] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:10,919] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:23:10,920] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:23:10,920] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:23:15,925] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter61 [2019-08-22 19:23:15,925] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:15,926] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:15,926] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:17,956] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:22:46Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:17,956] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:18,059] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:18,059] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:18,060] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:18,060] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:18,060] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:20,086] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:22:46Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:20,087] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:20,189] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:20,190] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:20,190] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 19:23:20,190] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:23:20,190] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:20,190] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:20,190] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:22,208] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:22,208] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:22,311] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:22,311] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:23:22,312] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:22,312] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:22,312] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:24,342] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:23:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:24,342] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:24,445] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:24,446] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:24,446] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:23:24,446] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:23:26,448] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:26,448] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:26,449] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:28,731] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:23:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:28,732] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:28,835] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:28,835] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:30,838] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:30,838] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:30,838] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:32,891] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:23:21Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:32,892] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:32,995] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:32,995] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:34,997] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:34,998] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:34,998] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:37,027] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:23:33Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:37,027] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:37,130] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:37,130] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:39,133] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:39,133] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:39,133] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:41,198] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:23:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:41,198] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:41,301] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:41,302] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:43,304] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:43,305] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:43,305] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:45,447] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:23:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:45,447] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:45,550] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:45,550] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:47,552] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:47,553] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:47,553] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:49,809] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:23:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:49,810] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:49,913] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:49,913] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:51,915] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:51,916] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:51,916] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:54,316] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 100 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:23:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:54,316] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:54,419] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:54,420] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:56,422] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:56,422] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:56,422] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:23:58,452] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:23:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:58,453] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:23:58,556] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:23:58,556] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:23:58,556] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:23:58,557] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:23:58,557] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:23:58,557] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:00,576] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:23:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:00,577] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:00,680] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:00,680] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:00,681] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-0 [2019-08-22 19:24:00,681] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:24:00,681] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:00,681] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:00,681] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:24:02,875] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:02,875] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:02,978] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:02,978] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:24:02,979] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:24:02,979] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:24:02,979] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:02,979] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:24:05,083] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=0.767 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.460 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.235 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.235/0.487/0.767/0.218 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:24:05,083] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:05,184] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:24:05,185] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:24:05,185] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:24:10,190] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter62 [2019-08-22 19:24:10,190] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:10,191] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:10,191] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:12,220] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:23:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:12,220] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:12,323] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:12,323] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:12,324] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:12,324] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:12,324] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:14,337] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:23:54Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:14,337] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:14,440] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:14,441] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:14,441] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 19:24:14,441] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:24:14,441] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:14,441] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:14,442] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:16,376] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:16,376] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:16,479] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:16,479] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:24:16,479] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:16,479] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:16,480] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:18,499] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:24:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:18,499] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:18,602] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:18,603] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:18,603] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:24:18,603] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:24:20,605] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:20,606] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:20,606] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:22,744] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:24:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:22,745] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:22,847] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:22,848] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:24,850] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:24,850] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:24,851] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:26,923] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:24:16Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:26,923] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:27,026] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:27,026] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:29,029] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:29,029] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:29,029] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:31,017] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:24:27Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:31,017] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:31,120] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:31,120] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:33,123] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:33,123] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:33,123] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:35,180] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:24:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:35,181] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:35,284] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:35,284] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:37,286] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:37,286] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:37,286] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:39,458] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:24:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:39,458] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:39,561] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:39,561] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:41,564] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:41,564] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:41,564] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:43,931] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:24:32Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:43,931] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:44,034] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:44,034] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:46,036] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:46,037] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:46,037] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:48,107] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:24:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:48,107] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:48,210] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:48,210] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:48,210] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:24:48,211] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:48,211] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:48,211] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:24:50,239] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:24:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:50,239] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:50,342] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:50,342] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:24:50,342] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 19:24:50,343] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:24:50,343] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:24:50,343] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:50,343] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:24:52,740] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:52,740] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:24:52,843] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:24:52,843] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:24:52,844] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:24:52,844] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:24:52,844] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:24:52,844] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:25:04,955] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2007ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:25:04,955] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:05,057] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:25:05,057] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:25:08,061] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:25:08,061] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:25:08,061] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:25:08,061] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:08,062] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:08,062] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:25:10,227] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:10,227] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:10,330] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:10,330] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:25:10,330] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:25:10,331] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:25:10,331] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:10,331] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:25:12,438] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=5.56 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.495 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.344 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2002ms rtt min/avg/max/mdev = 0.344/2.135/5.566/2.426 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:25:12,438] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:12,540] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:25:12,540] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:25:12,540] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:25:17,545] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter63 [2019-08-22 19:25:17,545] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:17,545] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:17,546] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:19,605] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:24:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:19,605] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:19,709] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:19,709] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:19,709] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:19,709] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:19,710] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:21,772] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:24:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:21,772] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:21,875] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:21,875] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:21,875] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:25:21,875] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:25:21,876] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:21,876] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:21,876] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:23,752] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:23,753] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:23,855] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:23,856] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:25:23,856] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:23,856] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:23,856] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:25,959] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:25:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:25,959] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:26,062] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:26,063] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:26,063] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:25:26,063] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:25:28,065] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:28,066] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:28,066] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:30,265] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:25:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:30,265] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:30,368] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:30,369] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:32,371] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:32,372] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:32,372] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:34,397] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:25:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:34,397] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:34,501] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:34,501] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:36,503] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:36,503] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:36,503] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:38,714] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:25:23Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:38,715] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:38,818] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:38,818] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:40,820] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:40,820] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:40,820] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:42,831] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:25:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:42,831] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:42,934] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:42,934] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:44,937] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:44,937] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:44,938] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:46,985] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:25:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:46,985] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:47,088] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:47,088] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:49,090] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:49,091] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:49,091] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:51,415] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:25:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:51,416] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:51,519] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:51,519] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:53,521] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:53,522] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:53,522] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:55,562] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:25:47Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:55,563] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:55,666] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:55,666] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:57,668] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:57,669] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:57,669] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:25:59,724] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:25:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:59,724] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:25:59,827] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:25:59,828] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:25:59,828] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:25:59,828] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:25:59,828] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:25:59,828] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:01,845] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:25:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:01,846] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:01,949] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:01,949] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:01,949] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-0 [2019-08-22 19:26:01,949] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:26:01,950] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:01,950] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:01,950] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:26:04,400] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:04,400] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:04,502] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:04,503] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:26:04,503] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:26:04,503] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:26:04,503] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:04,504] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:26:06,611] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.66 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.256 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.277 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2003ms rtt min/avg/max/mdev = 0.256/0.732/1.665/0.660 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:26:06,611] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:06,713] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:26:06,713] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:26:06,713] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:26:11,718] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter64 [2019-08-22 19:26:11,719] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:11,719] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:11,719] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:13,757] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:25:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:13,757] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:13,860] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:13,861] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:13,861] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:13,862] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:13,862] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:15,898] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:25:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:15,898] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:16,001] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:16,001] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:16,001] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-0 started. [2019-08-22 19:26:16,001] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:26:16,002] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:16,002] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:16,002] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:18,177] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:18,177] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:18,280] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:18,280] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:26:18,280] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:18,281] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:18,281] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:20,325] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:20,325] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:20,428] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:20,428] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:20,429] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:26:20,429] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:26:22,431] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:22,431] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:22,431] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:24,469] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:24,469] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:24,572] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:24,573] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:26,575] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:26,575] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:26,575] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:28,591] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:28,591] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:28,694] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:28,695] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:30,697] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:30,697] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:30,697] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:32,726] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:32,727] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:32,830] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:32,830] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:34,832] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:34,832] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:34,833] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:36,881] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:36,881] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:36,984] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:36,984] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:38,987] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:38,987] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:38,987] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:41,001] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:38Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:41,001] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:41,104] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:41,105] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:43,107] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:43,108] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:43,108] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:45,142] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:45,143] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:45,246] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:45,246] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:47,248] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:47,249] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:47,249] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:49,296] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:49,296] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:49,399] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:49,399] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:51,402] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:51,402] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:51,402] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:53,445] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:53,446] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:53,549] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:53,549] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:55,549] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:55,550] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:55,550] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:26:57,579] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-0 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-0 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 0bfa4339b0de53971593e13843dcd27c2972cd358fff434994a5319c | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:26:43Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:57,579] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:26:57,682] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:26:57,682] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:26:59,684] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:26:59,684] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:26:59,684] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:01,733] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:26:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:01,734] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:01,837] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:01,837] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:01,837] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:27:01,838] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:01,838] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:01,838] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:03,887] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:26:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:03,888] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:03,990] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:03,991] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:03,991] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-0 to compute-2 [2019-08-22 19:27:03,991] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:27:03,991] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:03,991] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:03,991] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:27:06,222] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:06,222] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:06,325] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:06,326] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:27:06,326] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:27:06,326] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:27:06,326] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:06,326] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:27:08,431] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.21 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.398 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.839 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.398/0.817/1.214/0.333 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:27:08,431] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:08,533] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:27:08,533] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:27:08,534] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:27:13,539] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter65 [2019-08-22 19:27:13,539] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:13,539] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:13,540] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:15,588] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:26:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:15,588] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:15,691] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:15,691] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:15,692] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:15,692] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:15,692] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:17,724] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:26:59Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:17,725] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:17,828] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:17,829] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:17,829] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:27:17,829] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:27:17,829] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:17,830] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:17,830] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:19,744] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:19,744] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:19,847] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:19,847] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:27:19,847] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:19,848] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:19,848] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:22,062] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:27:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:22,062] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:22,165] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:22,165] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:22,165] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:27:22,165] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:27:24,168] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:24,168] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:24,168] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:26,353] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:27:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:26,353] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:26,456] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:26,456] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:28,459] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:28,459] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:28,459] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:30,477] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:27:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:30,477] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:30,580] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:30,580] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:32,582] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:32,583] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:32,583] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:34,627] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:27:19Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:34,628] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:34,730] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:34,731] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:36,733] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:36,733] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:36,734] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:38,834] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:27:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:38,835] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:38,938] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:38,938] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:40,940] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:40,940] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:40,940] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:43,080] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 12 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:27:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:43,080] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:43,183] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:43,184] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:45,186] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:45,186] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:45,186] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:47,220] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 12 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:27:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:47,220] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:47,323] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:47,323] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:49,326] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:49,326] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:49,326] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:51,399] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 12 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:27:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:51,400] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:51,503] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:51,503] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:53,505] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:53,506] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:53,506] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:27:55,705] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 12 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:27:40Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:55,706] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:27:55,809] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:27:55,809] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:27:57,812] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:27:57,812] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:27:57,812] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:00,131] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:27:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:00,131] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:00,234] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:00,234] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:28:00,235] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:28:00,235] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:00,235] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:00,235] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:02,293] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:27:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:02,293] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:02,397] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:02,397] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:28:02,397] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 19:28:02,397] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:28:02,398] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:02,398] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:02,398] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:28:04,559] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:04,559] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:04,662] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:04,662] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:28:04,662] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:28:04,662] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:28:04,663] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:04,663] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:28:16,775] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2008ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:28:16,775] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:16,877] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:28:16,877] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:28:19,880] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:28:19,881] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:28:19,881] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:28:19,881] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:19,881] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:19,881] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:28:22,086] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:22,087] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:22,190] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:22,190] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:28:22,190] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:28:22,191] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:28:22,191] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:22,191] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:28:24,295] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.51 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.540 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.383 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.383/0.813/1.517/0.502 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:28:24,295] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:24,397] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:28:24,397] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:28:24,398] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:28:29,403] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter66 [2019-08-22 19:28:29,403] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:29,403] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:29,404] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:31,445] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:27:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:31,446] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:31,549] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:31,549] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:28:31,549] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:31,549] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:31,550] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:33,591] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:27:56Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:33,592] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:33,695] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:33,695] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:28:33,695] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-1 started. [2019-08-22 19:28:33,695] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:28:33,695] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:33,696] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:33,696] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:35,593] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:35,593] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:35,696] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:35,696] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:28:35,696] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:35,696] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:35,697] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:37,733] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:28:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:37,733] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:37,836] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:37,836] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:28:37,836] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:28:37,836] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:28:39,839] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:39,839] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:39,839] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:41,919] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:28:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:41,919] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:42,023] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:42,023] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:28:44,026] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:44,026] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:44,026] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:46,143] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:28:35Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:46,143] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:46,246] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:46,247] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:28:48,249] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:48,249] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:48,249] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:50,289] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:28:46Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:50,289] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:50,393] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:50,393] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:28:52,396] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:52,396] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:52,396] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:54,444] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:28:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:54,444] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:54,548] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:54,548] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:28:56,551] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:28:56,551] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:28:56,552] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:28:58,577] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:28:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:58,577] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:28:58,681] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:28:58,681] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:00,684] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:00,684] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:00,684] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:02,887] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:28:51Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:02,888] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:02,991] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:02,991] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:04,994] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:04,994] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:04,994] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:07,032] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:29:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:07,033] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:07,136] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:07,136] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:07,136] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:29:07,137] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:07,137] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:07,137] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:09,210] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:29:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:09,211] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:09,314] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:09,314] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:09,314] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-1 to compute-2 [2019-08-22 19:29:09,314] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:29:09,314] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:09,315] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:09,315] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:29:11,522] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:11,523] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:11,626] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:11,627] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:29:11,627] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:29:11,627] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:29:11,628] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:11,628] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:29:23,740] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2008ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:29:23,741] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:23,842] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:29:23,843] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:29:26,846] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:29:26,846] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:29:26,846] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:29:26,846] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:26,847] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:26,847] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:29:29,044] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:29,044] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:29,148] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:29,148] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:29:29,148] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:29:29,149] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:29:29,149] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:29,149] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:29:31,254] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. 64 bytes from 192.168.104.70: icmp_seq=1 ttl=63 time=1.30 ms 64 bytes from 192.168.104.70: icmp_seq=2 ttl=63 time=0.523 ms 64 bytes from 192.168.104.70: icmp_seq=3 ttl=63 time=0.385 ms --- 192.168.104.70 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2001ms rtt min/avg/max/mdev = 0.385/0.737/1.303/0.404 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:29:31,255] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:31,356] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:29:31,357] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.70 [2019-08-22 19:29:31,357] 1913 INFO MainThread vm_helper._ping_vms:: Ping successful from 128.224.186.181: {'192.168.104.70': 0} [2019-08-22 19:29:36,362] 164 INFO MainThread test_migrate.test_migrate_stress:: Live migration iter67 [2019-08-22 19:29:36,362] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:36,363] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:36,363] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:38,381] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:29:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:38,381] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:38,484] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:38,485] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:38,485] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:38,485] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:38,485] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:40,514] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:29:05Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:40,514] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:40,616] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:40,617] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:40,617] 1288 INFO MainThread vm_helper.live_migrate_vm:: Live migrating VM 698586e7-0ff0-4f52-a9f5-76ff1693539c from compute-2 started. [2019-08-22 19:29:40,617] 1289 INFO MainThread vm_helper.live_migrate_vm:: nova live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c [2019-08-22 19:29:40,617] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:40,618] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:40,618] 301 DEBUG MainThread ssh.send :: Send 'nova --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne live-migration 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:42,666] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:42,666] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:42,769] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:42,770] 1300 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to MIGRATING with best effort [2019-08-22 19:29:42,770] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:42,770] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:42,770] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:44,790] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:29:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:44,791] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:44,893] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:44,894] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:44,894] 1761 INFO MainThread vm_helper.wait_for_vm_status:: VM status has reached MIGRATING [2019-08-22 19:29:44,894] 1309 INFO MainThread vm_helper.live_migrate_vm:: Waiting for VM status change to original state ACTIVE [2019-08-22 19:29:46,896] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:46,896] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:46,896] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:49,064] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:29:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:49,065] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:49,167] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:49,168] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:51,170] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:51,171] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:51,171] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:53,208] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:29:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:53,208] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:53,311] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:53,312] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:55,314] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:55,314] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:55,314] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:29:57,353] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:29:42Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:57,354] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:29:57,456] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:29:57,457] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:29:59,459] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:29:59,459] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:29:59,459] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:30:01,504] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:29:58Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:01,504] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:01,608] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:01,608] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:30:03,611] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:30:03,611] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:03,611] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:30:05,654] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:30:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:05,654] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:05,757] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:05,757] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:30:07,759] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:30:07,759] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:07,759] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:30:09,793] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:30:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:09,794] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:09,897] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:09,897] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:30:11,899] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:30:11,900] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:11,900] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:30:14,205] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-2 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-2 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | migrating | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 5f361256695e97e91aed590ffc19599290ec783ddd6ee9328184a74d | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | MIGRATING | | updated | 2019-08-22T19:30:04Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:14,205] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:14,308] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:14,309] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:30:16,310] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:30:16,310] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:16,310] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:30:18,375] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:30:17Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:18,375] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:18,478] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:18,479] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:30:18,479] 1316 INFO MainThread vm_helper.live_migrate_vm:: Live migrate vm 698586e7-0ff0-4f52-a9f5-76ff1693539c completed [2019-08-22 19:30:18,479] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:30:18,479] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:18,479] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:30:20,528] 423 DEBUG MainThread ssh.expect :: Output: +-------------------------------------+-------------------------------------------------------------+ | Field | Value | +-------------------------------------+-------------------------------------------------------------+ | OS-DCF:diskConfig | MANUAL | | OS-EXT-AZ:availability_zone | nova | | OS-EXT-SRV-ATTR:host | compute-1 | | OS-EXT-SRV-ATTR:hypervisor_hostname | compute-1 | | OS-EXT-SRV-ATTR:instance_name | instance-00000023 | | OS-EXT-STS:power_state | Running | | OS-EXT-STS:task_state | None | | OS-EXT-STS:vm_state | active | | OS-SRV-USG:launched_at | 2019-08-22T18:20:42.000000 | | OS-SRV-USG:terminated_at | None | | accessIPv4 | | | accessIPv6 | | | addresses | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | config_drive | | | created | 2019-08-22T18:20:26Z | | flavor | flavor-default-size2 (b8f20ebe-4c60-4d9d-81e7-f30332e01049) | | hostId | 211103613e4ac03ab8ddea069c9d5a844206be00ec9c9186237554ce | | id | 698586e7-0ff0-4f52-a9f5-76ff1693539c | | image | | | key_name | keypair-tenant1 | | name | tenant1-volume-stress-1 | | progress | 0 | | project_id | 71e4950fb29c40d8ac13742b4990dfad | | properties | | | security_groups | name='default' | | | name='default' | | status | ACTIVE | | updated | 2019-08-22T19:30:18Z | | user_id | 2a2aefc01a8a4503b3ed8775f3443efe | | volumes_attached | id='753449a4-494b-42f5-aa19-768f8fc0d60e' | +-------------------------------------+-------------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:20,528] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:20,631] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:20,631] 202 DEBUG MainThread table_parser.__convert_multilines_values:: Multi-row entry found for: security_groups [2019-08-22 19:30:20,631] 1368 INFO MainThread vm_helper.live_migrate_vm:: VM 698586e7-0ff0-4f52-a9f5-76ff1693539c successfully migrated from compute-2 to compute-1 [2019-08-22 19:30:20,632] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:30:20,632] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:30:20,632] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:20,632] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:30:22,883] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:22,883] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:22,986] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:22,987] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:30:22,987] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:30:22,987] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:30:22,987] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:22,987] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:30:35,107] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2016ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:30:35,107] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:35,209] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:30:35,209] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:30:38,213] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:30:38,213] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:30:38,213] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:30:38,213] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:30:38,213] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:38,214] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:30:40,443] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:40,444] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:40,547] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:40,547] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:30:40,547] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:30:40,547] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:30:40,548] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:40,548] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:30:43,661] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. From 192.168.4.1 icmp_seq=1 Destination Host Unreachable From 192.168.4.1 icmp_seq=2 Destination Host Unreachable From 192.168.4.1 icmp_seq=3 Destination Host Unreachable --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 1999ms pipe 3 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:30:43,661] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:43,763] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:30:43,763] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:30:46,767] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:30:46,767] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:30:46,767] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:30:46,767] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:30:46,768] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:46,768] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:30:49,000] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:49,000] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:49,103] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:49,104] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:30:49,104] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:30:49,104] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:30:49,104] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:49,104] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:30:52,205] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. From 192.168.4.1 icmp_seq=1 Destination Host Unreachable From 192.168.4.1 icmp_seq=2 Destination Host Unreachable From 192.168.4.1 icmp_seq=3 Destination Host Unreachable --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2001ms pipe 3 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:30:52,205] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:52,307] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:30:52,307] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:30:55,310] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:30:55,311] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:30:55,311] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:30:55,311] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:30:55,311] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:55,311] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:30:57,495] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:57,496] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:30:57,598] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:30:57,599] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:30:57,599] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:30:57,599] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:30:57,599] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:30:57,599] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:31:00,701] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. From 192.168.4.1 icmp_seq=1 Destination Host Unreachable From 192.168.4.1 icmp_seq=2 Destination Host Unreachable From 192.168.4.1 icmp_seq=3 Destination Host Unreachable --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2008ms pipe 3 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:00,701] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:00,803] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:00,803] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:31:03,806] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:31:03,806] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:31:03,807] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:31:03,807] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:31:03,807] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:03,807] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:31:06,007] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:06,008] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:06,111] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:06,111] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:31:06,111] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:31:06,111] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:31:06,112] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:06,112] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:31:09,213] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. From 192.168.4.1 icmp_seq=1 Destination Host Unreachable From 192.168.4.1 icmp_seq=2 Destination Host Unreachable From 192.168.4.1 icmp_seq=3 Destination Host Unreachable --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2008ms pipe 3 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:09,213] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:09,315] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:09,315] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:31:12,318] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:31:12,319] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:31:12,319] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:31:12,319] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:31:12,319] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:12,319] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:31:14,699] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:14,699] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:14,802] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:14,803] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:31:14,803] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:31:14,803] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:31:14,803] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:14,803] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:31:17,905] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. From 192.168.4.1 icmp_seq=1 Destination Host Unreachable From 192.168.4.1 icmp_seq=2 Destination Host Unreachable From 192.168.4.1 icmp_seq=3 Destination Host Unreachable --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, +3 errors, 100% packet loss, time 2007ms pipe 3 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:17,905] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:18,007] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:18,007] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:31:21,010] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:31:21,011] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:31:21,011] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:31:21,011] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:31:21,011] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:21,011] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:31:23,197] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:23,197] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:23,300] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:23,301] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:31:23,301] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:31:23,301] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:31:23,301] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:23,302] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:31:35,405] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2000ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:35,405] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:35,507] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:35,507] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:31:38,510] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:31:38,511] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:31:38,511] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:31:38,511] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:31:38,511] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:38,512] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:31:40,688] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:40,689] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:40,792] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:40,792] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:31:40,792] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:31:40,792] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:31:40,793] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:40,793] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:31:52,923] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2017ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:52,923] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:53,025] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:31:53,025] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:31:56,028] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:31:56,028] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:31:56,029] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:31:56,029] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:31:56,029] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:56,029] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:31:58,163] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:58,164] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:31:58,267] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:31:58,267] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:31:58,267] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:31:58,268] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:31:58,268] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:31:58,268] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:32:10,373] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2001ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:32:10,373] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:32:10,475] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:32:10,475] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:32:13,478] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:32:13,479] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:32:13,479] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:32:13,479] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:32:13,479] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:32:13,479] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:32:16,113] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:32:16,113] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:32:16,216] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:32:16,217] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:32:16,217] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:32:16,217] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:32:16,217] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:32:16,217] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:32:28,331] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2010ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:32:28,331] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:32:28,433] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:32:28,433] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:32:31,437] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:32:31,437] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:32:31,437] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:32:31,437] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:32:31,437] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:32:31,438] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:32:33,622] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:32:33,622] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:32:33,725] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:32:33,725] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:32:33,726] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:32:33,726] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:32:33,726] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:32:33,726] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:32:45,829] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:32:45,829] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:32:45,931] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:32:45,931] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:32:48,934] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:32:48,935] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:32:48,935] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:32:48,935] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:32:48,935] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:32:48,936] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:32:51,170] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:32:51,171] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:32:51,274] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:32:51,274] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:32:51,274] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:32:51,275] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:32:51,275] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:32:51,275] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:33:03,388] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2010ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:03,388] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:03,490] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:03,491] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:33:06,493] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:33:06,493] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:33:06,494] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:33:06,494] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:33:06,494] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:06,494] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:33:08,699] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:08,699] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:08,803] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:08,803] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:33:08,803] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:33:08,804] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:33:08,804] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:08,804] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:33:20,915] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 2007ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:20,915] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:21,016] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:21,017] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:33:24,019] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:33:24,019] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:33:24,020] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:33:24,020] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:33:24,020] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:24,020] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --a' [2019-08-22 19:33:26,218] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:26,218] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:26,321] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:26,321] 1654 DEBUG MainThread network_helper._get_net_ips_for_vms:: targeted ips for vm: ['192.168.104.70'] [2019-08-22 19:33:26,321] 1666 INFO MainThread network_helper._get_net_ips_for_vms:: IPs dict: {'698586e7-0ff0-4f52-a9f5-76ff1693539c': ['192.168.104.70']} [2019-08-22 19:33:26,322] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.70 from host 128.224.186.181 [2019-08-22 19:33:26,322] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:26,322] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.70' [2019-08-22 19:33:38,425] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.70 (192.168.104.70) 56(84) bytes of data. --- 192.168.104.70 ping statistics --- 3 packets transmitted, 0 received, 100% packet loss, time 1999ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:38,425] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:38,527] 423 DEBUG MainThread ssh.expect :: Output: 1 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:38,528] 2563 INFO MainThread network_helper.ping_server:: retry in 3 seconds [2019-08-22 19:33:41,531] 2567 WARNING MainThread network_helper.ping_server:: Ping from 128.224.186.181 to 192.168.104.70 failed. [2019-08-22 19:33:41,532] 1926 INFO MainThread vm_helper._ping_vms:: Ping unsuccessful from vm (logged in via 128.224.186.181): {'192.168.104.70': 100} [2019-08-22 19:33:41,532] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:33:41,532] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:41,532] 301 DEBUG MainThread ssh.send :: Send 'date +'%Y%m%d_%H-%M'' [2019-08-22 19:33:41,637] 423 DEBUG MainThread ssh.expect :: Output: 20190822_19-33 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:41,637] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:41,740] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:41,741] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:33:41,741] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:41,741] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne console log show 698586e7-0ff0-4f52-a9f5-76ff1693539c' [2019-08-22 19:33:44,137] 423 DEBUG MainThread ssh.expect :: Output: [ 4248.518530] chronyd[520]: Can't synchronise: no selectable sources [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:44,137] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:44,240] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:44,241] 3295 INFO MainThread network_helper.collect_networking_info:: Ping tenant(s) router's external and internal gateway IPs [2019-08-22 19:33:44,241] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:33:44,241] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:44,241] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne router list' [2019-08-22 19:33:46,308] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+----------------+--------+-------+----------------------------------+-------------+-------+ | ID | Name | Status | State | Project | Distributed | HA | +--------------------------------------+----------------+--------+-------+----------------------------------+-------------+-------+ | 599452f7-c7b7-45a0-b81e-b1dcd3aff849 | tenant2-router | ACTIVE | UP | 5fad34de5c5b4397b8e0fc2a83d94854 | False | False | | f224e410-8643-47ae-9f7a-16ccb1f6e1e7 | tenant1-router | ACTIVE | UP | 71e4950fb29c40d8ac13742b4990dfad | False | False | +--------------------------------------+----------------+--------+-------+----------------------------------+-------------+-------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:46,308] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:46,411] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:46,412] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:33:46,412] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:46,412] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --project=5fad34de5c5b4397b8e0fc2a83d94854' [2019-08-22 19:33:48,353] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:48,354] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:48,457] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:48,457] 225 DEBUG MainThread table_parser.table :: No table returned [2019-08-22 19:33:48,457] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:33:48,457] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:48,458] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne server list --project=71e4950fb29c40d8ac13742b4990dfad' [2019-08-22 19:33:50,663] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | ID | Name | Status | Networks | Image | Flavor | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ | 698586e7-0ff0-4f52-a9f5-76ff1693539c | tenant1-volume-stress-1 | ACTIVE | tenant1-mgmt-net=192.168.104.70; tenant1-net5=172.16.5.152 | | flavor-default-size2 | +--------------------------------------+-------------------------+--------+------------------------------------------------------------+-------+----------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:50,663] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:50,766] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:50,766] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:33:50,767] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:50,767] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne port list --router=f224e410-8643-47ae-9f7a-16ccb1f6e1e7 --long' [2019-08-22 19:33:52,709] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+------+-------------------+-------------------------------------------------------------------------------+--------+-----------------+--------------------------+------+ | ID | Name | MAC Address | Fixed IP Addresses | Status | Security Groups | Device Owner | Tags | +--------------------------------------+------+-------------------+-------------------------------------------------------------------------------+--------+-----------------+--------------------------+------+ | 81b0a8de-c568-4a3f-a049-a250c5f006cc | | fa:16:3e:35:c6:94 | ip_address='192.168.104.33', subnet_id='0ceac9b3-8c6a-4698-8cd0-bc31ec320822' | ACTIVE | | network:router_interface | | | ba454dce-87bf-4100-99ec-3547e6cc01c3 | | fa:16:3e:19:a9:57 | ip_address='192.168.104.1', subnet_id='089f4c0e-3b8e-4d02-9dc3-195f410cd99c' | ACTIVE | | network:router_interface | | | d9a47080-f509-4807-9ac6-226350636b47 | | fa:16:3e:71:da:b9 | ip_address='192.168.4.2', subnet_id='d074ae1e-7fcb-465c-b8f4-96641156ba84' | ACTIVE | | network:router_gateway | | | f7038ae5-0129-4598-9946-c592189c5431 | | fa:16:3e:e6:fa:bb | ip_address='192.168.104.65', subnet_id='065469c8-d801-47fb-a4f6-5fa45dbb2df7' | ACTIVE | | network:router_interface | | +--------------------------------------+------+-------------------+-------------------------------------------------------------------------------+--------+-----------------+--------------------------+------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:52,709] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:52,812] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:33:52,813] 1419 INFO MainThread ssh.get_natbox_client:: Getting NatBox Client... [2019-08-22 19:33:52,813] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.33 from host 128.224.186.181 [2019-08-22 19:33:52,813] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:52,813] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.33' [2019-08-22 19:33:54,914] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.33 (192.168.104.33) 56(84) bytes of data. 64 bytes from 192.168.104.33: icmp_seq=1 ttl=64 time=0.100 ms 64 bytes from 192.168.104.33: icmp_seq=2 ttl=64 time=0.086 ms 64 bytes from 192.168.104.33: icmp_seq=3 ttl=64 time=0.083 ms --- 192.168.104.33 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 1998ms rtt min/avg/max/mdev = 0.083/0.089/0.100/0.013 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:54,915] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:55,016] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:55,017] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.33 [2019-08-22 19:33:55,017] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.1 from host 128.224.186.181 [2019-08-22 19:33:55,017] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:55,017] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.1' [2019-08-22 19:33:57,121] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.1 (192.168.104.1) 56(84) bytes of data. 64 bytes from 192.168.104.1: icmp_seq=1 ttl=64 time=0.153 ms 64 bytes from 192.168.104.1: icmp_seq=2 ttl=64 time=0.121 ms 64 bytes from 192.168.104.1: icmp_seq=3 ttl=64 time=0.115 ms --- 192.168.104.1 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.115/0.129/0.153/0.021 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:57,121] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:57,223] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:57,223] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.1 [2019-08-22 19:33:57,223] 2525 INFO MainThread network_helper.ping_server:: Ping 192.168.104.65 from host 128.224.186.181 [2019-08-22 19:33:57,224] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:57,224] 301 DEBUG MainThread ssh.send :: Send 'ping -c 3 192.168.104.65' [2019-08-22 19:33:59,327] 423 DEBUG MainThread ssh.expect :: Output: PING 192.168.104.65 (192.168.104.65) 56(84) bytes of data. 64 bytes from 192.168.104.65: icmp_seq=1 ttl=64 time=0.120 ms 64 bytes from 192.168.104.65: icmp_seq=2 ttl=64 time=0.152 ms 64 bytes from 192.168.104.65: icmp_seq=3 ttl=64 time=0.172 ms --- 192.168.104.65 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 2000ms rtt min/avg/max/mdev = 0.120/0.148/0.172/0.021 ms svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:59,327] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:33:59,429] 423 DEBUG MainThread ssh.expect :: Output: 0 svc-cgcsauto@tis-lab-nat-box:~$ [2019-08-22 19:33:59,430] 2560 INFO MainThread network_helper.ping_server:: All packets received by 192.168.104.65 [2019-08-22 19:33:59,430] 3394 INFO MainThread network_helper.ping_ips_from_natbox:: Ping successful from NatBox: ['192.168.104.33', '192.168.104.1', '192.168.104.65'] [2019-08-22 19:33:59,430] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:33:59,431] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:33:59,431] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne hypervisor list' [2019-08-22 19:34:01,280] 423 DEBUG MainThread ssh.expect :: Output: +----+---------------------+-----------------+-----------------+-------+ | ID | Hypervisor Hostname | Hypervisor Type | Host IP | State | +----+---------------------+-----------------+-----------------+-------+ | 5 | compute-2 | QEMU | 192.168.206.156 | up | | 8 | compute-0 | QEMU | 192.168.206.194 | down | | 11 | compute-3 | QEMU | 192.168.206.126 | down | | 14 | compute-1 | QEMU | 192.168.206.139 | up | +----+---------------------+-----------------+-----------------+-------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:01,280] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:01,383] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:01,383] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:34:01,384] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:01,384] 301 DEBUG MainThread ssh.send :: Send 'openstack --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://keystone.openstack.svc.cluster.local/v3 --os-user-domain-name Default --os-project-domain-name Default --os-identity-api-version 3 --os-interface internal --os-region-name RegionOne network agent list --router=f224e410-8643-47ae-9f7a-16ccb1f6e1e7' [2019-08-22 19:34:03,372] 423 DEBUG MainThread ssh.expect :: Output: +--------------------------------------+------------+-----------+-------------------+-------+-------+------------------+ | ID | Agent Type | Host | Availability Zone | Alive | State | Binary | +--------------------------------------+------------+-----------+-------------------+-------+-------+------------------+ | 26faf905-e6ab-40b0-926f-ef049ed779ee | L3 agent | compute-2 | nova | :-) | UP | neutron-l3-agent | +--------------------------------------+------------+-----------+-------------------+-------+-------+------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:03,372] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:03,475] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:03,476] 3332 INFO MainThread network_helper.collect_networking_info:: Router f224e410-8643-47ae-9f7a-16ccb1f6e1e7 is hosted on compute-2 [2019-08-22 19:34:03,476] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:34:03,476] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:03,476] 301 DEBUG MainThread ssh.send :: Send 'system --os-username 'admin' --os-password 'Li69nux*' --os-project-name admin --os-auth-url http://192.168.204.2:5000/v3 --os-user-domain-name Default --os-project-domain-name Default --os-endpoint-type internalURL --os-region-name RegionOne show' [2019-08-22 19:34:05,097] 423 DEBUG MainThread ssh.expect :: Output: +----------------------+------------------------------------------------------+ | Property | Value | +----------------------+------------------------------------------------------+ | contact | None | | created_at | 2019-08-21T18:54:39.677916+00:00 | | description | yow-cgcs-ironpass-20_27: setup by deployment manager | | https_enabled | False | | location | None | | name | yow-cgcs-ironpass-20-27 | | region_name | RegionOne | | sdn_enabled | False | | security_feature | spectre_meltdown_v1 | | service_project_name | services | | software_version | 19.09 | | system_mode | duplex | | system_type | Standard | | timezone | UTC | | updated_at | 2019-08-21T19:04:28.806360+00:00 | | uuid | 3341a897-159f-4c0c-aa7f-ebff9b6c8d9f | | vswitch_type | none | +----------------------+------------------------------------------------------+ [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:05,098] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:05,201] 423 DEBUG MainThread ssh.expect :: Output: 0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:05,201] 3339 INFO MainThread network_helper.collect_networking_info:: Collect ovs.info for ['f224e410-8643-47ae-9f7a-16ccb1f6e1e7'] router(s) on router host(s): ['compute-2', 'compute-1'] [2019-08-22 19:34:05,201] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:34:05,201] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:05,201] 301 DEBUG MainThread ssh.send :: Send 'hostname' [2019-08-22 19:34:05,305] 423 DEBUG MainThread ssh.expect :: Output: controller-0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:05,305] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:05,306] 301 DEBUG MainThread ssh.send :: Send 'hostname' [2019-08-22 19:34:05,409] 423 DEBUG MainThread ssh.expect :: Output: controller-0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:05,410] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 19:34:05,513] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:05,513] 1185 INFO MainThread ssh.connect :: Attempt to connect to compute-2 from 128.224.151.49... [2019-08-22 19:34:05,513] 301 DEBUG MainThread ssh.send :: Send '/usr/bin/ssh -o RSAAuthentication=no -o PubkeyAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null sysadmin@compute-2' [2019-08-22 19:34:05,655] 423 DEBUG MainThread ssh.expect :: Output: Warning: Permanently added 'compute-2,192.168.204.58' (ECDSA) to the list of known hosts. Release 19.09 ------------------------------------------------------------------------ W A R N I N G *** W A R N I N G *** W A R N I N G *** W A R N I N G *** ------------------------------------------------------------------------ THIS IS A PRIVATE COMPUTER SYSTEM. This computer system including all related equipment, network devices (specifically including Internet access), are provided only for authorized use. All computer systems may be monitored for all lawful purposes, including to ensure that their use is authorized, for management of the system, to facilitate protection against unauthorized access, and to verify security procedures, survivability and operational security. Monitoring includes active attacks by authorized personnel and their entities to test or verify the security of the system. During monitoring, information may be examined, recorded, copied and used for authorized purposes. All information including personal information, placed on or sent over this system may be monitored. Uses of this system, authorized or unauthorized, constitutes consent to monitoring of this system. Unauthorized use may subject you to criminal prosecution. Evidence of any such unauthorized use collected during monitoring may be used for administrative, criminal or other adverse action. Use of this system constitutes consent to monitoring for these purposes. sysadmin@compute-2's password: [2019-08-22 19:34:05,655] 301 DEBUG MainThread ssh.send :: Send 'Li69nux*' [2019-08-22 19:34:06,083] 423 DEBUG MainThread ssh.expect :: Output: Last login: Thu Aug 22 18:14:31 2019 from 192.168.204.3 /etc/motd.d/00-header:  WARNING: Unauthorized access to this system is forbidden and will be prosecuted by law. By accessing this system, you agree that your actions may be monitored if unauthorized usage is suspected. [?1034hcompute-2:~$ [2019-08-22 19:34:06,083] 1220 INFO MainThread ssh.connect :: Successfully connected to compute-2 from 128.224.151.49! [2019-08-22 19:34:06,084] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:06,084] 301 DEBUG MainThread ssh.send :: Send 'export TMOUT=0' [2019-08-22 19:34:06,198] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:06,198] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:06,301] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-2:~$ [2019-08-22 19:34:06,301] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:06,302] 301 DEBUG MainThread ssh.send :: Send 'hostname' [2019-08-22 19:34:06,405] 423 DEBUG MainThread ssh.expect :: Output: compute-2 compute-2:~$ [2019-08-22 19:34:06,405] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:06,406] 301 DEBUG MainThread ssh.send :: Send 'touch /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:06,511] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:06,511] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:06,615] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-2:~$ [2019-08-22 19:34:06,615] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:06,616] 301 DEBUG MainThread ssh.send :: Send 'sudo /etc/collect.d/collect_ovs' [2019-08-22 19:34:06,739] 423 DEBUG MainThread ssh.expect :: Output: Password: [2019-08-22 19:34:06,739] 301 DEBUG MainThread ssh.send :: Send 'Li69nux*' [2019-08-22 19:34:06,881] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:06,882] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:06,984] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-2:~$ [2019-08-22 19:34:06,985] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:06,985] 301 DEBUG MainThread ssh.send :: Send 'echo -e "##### compute-2 ovs.info collected #####\n" >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:07,088] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:08,089] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:08,089] 301 DEBUG MainThread ssh.send :: Send 'sudo cat /scratch/var/extra/ovs.info >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:08,221] 423 DEBUG MainThread ssh.expect :: Output: cat: /scratch/var/extra/ovs.info: No such file or directory compute-2:~$ [2019-08-22 19:34:08,221] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:08,221] 301 DEBUG MainThread ssh.send :: Send 'sudo rm -f /scratch/var/extra/ovs.info' [2019-08-22 19:34:08,350] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:08,350] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:08,453] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-2:~$ [2019-08-22 19:34:08,453] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:08,454] 301 DEBUG MainThread ssh.send :: Send 'echo -e "\n\n#### Additional ovs cmds on compute-2 ####\n >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log"' [2019-08-22 19:34:08,557] 423 DEBUG MainThread ssh.expect :: Output: #### Additional ovs cmds on compute-2 #### >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log compute-2:~$ [2019-08-22 19:34:08,557] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:08,557] 301 DEBUG MainThread ssh.send :: Send 'echo -e "\n\n\n$ sudo ovs-ofctl show br-int" >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:08,660] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:08,660] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:08,764] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-2:~$ [2019-08-22 19:34:08,764] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:08,764] 301 DEBUG MainThread ssh.send :: Send 'sudo ovs-ofctl show br-int >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:08,909] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:08,909] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:08,910] 301 DEBUG MainThread ssh.send :: Send 'echo -e "\n\n\n$ sudo ovs-ofctl dump-flows br-int" >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:09,012] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:09,013] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:09,116] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-2:~$ [2019-08-22 19:34:09,116] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:09,116] 301 DEBUG MainThread ssh.send :: Send 'sudo ovs-ofctl dump-flows br-int >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:09,254] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:09,254] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:09,254] 301 DEBUG MainThread ssh.send :: Send 'echo -e "\n\n\n$ sudo ovs-appctl dpif/dump-flows br-int" >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:09,357] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:09,357] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:09,475] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-2:~$ [2019-08-22 19:34:09,475] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:09,475] 301 DEBUG MainThread ssh.send :: Send 'sudo ovs-appctl dpif/dump-flows br-int >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:09,723] 423 DEBUG MainThread ssh.expect :: Output: 2019-08-22T19:34:09Z|00001|daemon_unix|WARN|/var/run/openvswitch/ovs-vswitchd.pid: stale pidfile for pid 23 being deleted by pid 856747 ovs-appctl: cannot read pidfile "/var/run/openvswitch/ovs-vswitchd.pid" (No such process) compute-2:~$ [2019-08-22 19:34:09,723] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:09,724] 301 DEBUG MainThread ssh.send :: Send 'sudo chmod 777 /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:09,919] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:09,919] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:10,022] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-2:~$ [2019-08-22 19:34:10,023] 301 DEBUG MainThread ssh.send :: Send 'scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null /home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log sysadmin@controller-0:/home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log' [2019-08-22 19:34:10,246] 423 DEBUG MainThread ssh.expect :: Output: Warning: Permanently added 'controller-0,192.168.204.3' (ECDSA) to the list of known hosts. Release 19.09 ------------------------------------------------------------------------ W A R N I N G *** W A R N I N G *** W A R N I N G *** W A R N I N G *** ------------------------------------------------------------------------ THIS IS A PRIVATE COMPUTER SYSTEM. This computer system including all related equipment, network devices (specifically including Internet access), are provided only for authorized use. All computer systems may be monitored for all lawful purposes, including to ensure that their use is authorized, for management of the system, to facilitate protection against unauthorized access, and to verify security procedures, survivability and operational security. Monitoring includes active attacks by authorized personnel and their entities to test or verify the security of the system. During monitoring, information may be examined, recorded, copied and used for authorized purposes. All information including personal information, placed on or sent over this system may be monitored. Uses of this system, authorized or unauthorized, constitutes consent to monitoring of this system. Unauthorized use may subject you to criminal prosecution. Evidence of any such unauthorized use collected during monitoring may be used for administrative, criminal or other adverse action. Use of this system constitutes consent to monitoring for these purposes. sysadmin@controller-0's password: [2019-08-22 19:34:10,247] 301 DEBUG MainThread ssh.send :: Send 'Li69nux*' [2019-08-22 19:34:10,400] 423 DEBUG MainThread ssh.expect :: Output: 20190822_19-33-test_migrate_stress-compute-2-vswitch.log 0% 0 0.0KB/s --:-- ETA 20190822_19-33-test_migrate_stress-compute-2-vswitch.log 100% 26KB 27.3MB/s 00:00 compute-2:~$ [2019-08-22 19:34:10,400] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:10,511] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-2:~$ [2019-08-22 19:34:10,512] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 19:34:10,626] 423 DEBUG MainThread ssh.expect :: Output: compute-2:~$ [2019-08-22 19:34:10,626] 301 DEBUG MainThread ssh.send :: Send 'exit' [2019-08-22 19:34:10,731] 423 DEBUG MainThread ssh.expect :: Output: exit logout Connection to compute-2 closed. [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:10,731] 1257 INFO MainThread ssh.close :: ssh session to compute-2 is closed and returned to parent session 128.224.151.49 [2019-08-22 19:34:10,731] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:34:10,732] 309 DEBUG MainThread common._scp_on_local:: scp cmd: scp -oStrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null sysadmin@128.224.151.49:/home/sysadmin/20190822_19-33-test_migrate_stress-compute-2-vswitch.log /sandbox/AUTOMATION_LOGS/ip_20_27/201908221419/ping_failures/20190822_19-33-test_migrate_stress-compute-2-vswitch.log [2019-08-22 19:34:11,120] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:34:11,121] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:11,121] 301 DEBUG MainThread ssh.send :: Send 'hostname' [2019-08-22 19:34:11,225] 423 DEBUG MainThread ssh.expect :: Output: controller-0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:11,225] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:11,225] 301 DEBUG MainThread ssh.send :: Send 'hostname' [2019-08-22 19:34:11,329] 423 DEBUG MainThread ssh.expect :: Output: controller-0 [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:11,329] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 19:34:11,432] 423 DEBUG MainThread ssh.expect :: Output: [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:11,433] 1185 INFO MainThread ssh.connect :: Attempt to connect to compute-1 from 128.224.151.49... [2019-08-22 19:34:11,433] 301 DEBUG MainThread ssh.send :: Send '/usr/bin/ssh -o RSAAuthentication=no -o PubkeyAuthentication=no -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null sysadmin@compute-1' [2019-08-22 19:34:11,609] 423 DEBUG MainThread ssh.expect :: Output: Warning: Permanently added 'compute-1,192.168.204.77' (ECDSA) to the list of known hosts. Release 19.09 ------------------------------------------------------------------------ W A R N I N G *** W A R N I N G *** W A R N I N G *** W A R N I N G *** ------------------------------------------------------------------------ THIS IS A PRIVATE COMPUTER SYSTEM. This computer system including all related equipment, network devices (specifically including Internet access), are provided only for authorized use. All computer systems may be monitored for all lawful purposes, including to ensure that their use is authorized, for management of the system, to facilitate protection against unauthorized access, and to verify security procedures, survivability and operational security. Monitoring includes active attacks by authorized personnel and their entities to test or verify the security of the system. During monitoring, information may be examined, recorded, copied and used for authorized purposes. All information including personal information, placed on or sent over this system may be monitored. Uses of this system, authorized or unauthorized, constitutes consent to monitoring of this system. Unauthorized use may subject you to criminal prosecution. Evidence of any such unauthorized use collected during monitoring may be used for administrative, criminal or other adverse action. Use of this system constitutes consent to monitoring for these purposes. sysadmin@compute-1's password: [2019-08-22 19:34:11,609] 301 DEBUG MainThread ssh.send :: Send 'Li69nux*' [2019-08-22 19:34:11,953] 423 DEBUG MainThread ssh.expect :: Output: Last login: Thu Aug 22 18:14:29 2019 from 192.168.204.3 /etc/motd.d/00-header:  WARNING: Unauthorized access to this system is forbidden and will be prosecuted by law. By accessing this system, you agree that your actions may be monitored if unauthorized usage is suspected. [?1034hcompute-1:~$ [2019-08-22 19:34:11,953] 1220 INFO MainThread ssh.connect :: Successfully connected to compute-1 from 128.224.151.49! [2019-08-22 19:34:11,953] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:11,953] 301 DEBUG MainThread ssh.send :: Send 'export TMOUT=0' [2019-08-22 19:34:12,057] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:12,057] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:12,160] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-1:~$ [2019-08-22 19:34:12,161] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:12,161] 301 DEBUG MainThread ssh.send :: Send 'hostname' [2019-08-22 19:34:12,266] 423 DEBUG MainThread ssh.expect :: Output: compute-1 compute-1:~$ [2019-08-22 19:34:12,266] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:12,266] 301 DEBUG MainThread ssh.send :: Send 'touch /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:12,372] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:12,372] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:12,476] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-1:~$ [2019-08-22 19:34:12,476] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:12,476] 301 DEBUG MainThread ssh.send :: Send 'sudo /etc/collect.d/collect_ovs' [2019-08-22 19:34:12,611] 423 DEBUG MainThread ssh.expect :: Output: Password: [2019-08-22 19:34:12,611] 301 DEBUG MainThread ssh.send :: Send 'Li69nux*' [2019-08-22 19:34:12,886] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:12,886] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:12,989] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-1:~$ [2019-08-22 19:34:12,989] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:12,989] 301 DEBUG MainThread ssh.send :: Send 'echo -e "##### compute-1 ovs.info collected #####\n" >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:13,092] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:14,094] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:14,094] 301 DEBUG MainThread ssh.send :: Send 'sudo cat /scratch/var/extra/ovs.info >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:14,385] 423 DEBUG MainThread ssh.expect :: Output: cat: /scratch/var/extra/ovs.info: No such file or directory compute-1:~$ [2019-08-22 19:34:14,385] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:14,385] 301 DEBUG MainThread ssh.send :: Send 'sudo rm -f /scratch/var/extra/ovs.info' [2019-08-22 19:34:14,534] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:14,534] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:14,637] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-1:~$ [2019-08-22 19:34:14,638] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:14,638] 301 DEBUG MainThread ssh.send :: Send 'echo -e "\n\n#### Additional ovs cmds on compute-1 ####\n >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log"' [2019-08-22 19:34:14,741] 423 DEBUG MainThread ssh.expect :: Output: #### Additional ovs cmds on compute-1 #### >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log compute-1:~$ [2019-08-22 19:34:14,741] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:14,741] 301 DEBUG MainThread ssh.send :: Send 'echo -e "\n\n\n$ sudo ovs-ofctl show br-int" >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:14,845] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:14,845] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:14,948] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-1:~$ [2019-08-22 19:34:14,949] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:14,949] 301 DEBUG MainThread ssh.send :: Send 'sudo ovs-ofctl show br-int >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:15,160] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:15,160] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:15,160] 301 DEBUG MainThread ssh.send :: Send 'echo -e "\n\n\n$ sudo ovs-ofctl dump-flows br-int" >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:15,282] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:15,282] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:15,414] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-1:~$ [2019-08-22 19:34:15,414] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:15,414] 301 DEBUG MainThread ssh.send :: Send 'sudo ovs-ofctl dump-flows br-int >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:15,808] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:15,808] 466 DEBUG MainThread ssh.exec_cmd:: Executing command... [2019-08-22 19:34:15,808] 301 DEBUG MainThread ssh.send :: Send 'echo -e "\n\n\n$ sudo ovs-appctl dpif/dump-flows br-int" >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:15,934] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:15,934] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:16,052] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-1:~$ [2019-08-22 19:34:16,053] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:16,053] 301 DEBUG MainThread ssh.send :: Send 'sudo ovs-appctl dpif/dump-flows br-int >> /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:16,353] 423 DEBUG MainThread ssh.expect :: Output: 2019-08-22T19:34:16Z|00001|daemon_unix|WARN|/var/run/openvswitch/ovs-vswitchd.pid: stale pidfile for pid 24 being deleted by pid 845882 ovs-appctl: cannot read pidfile "/var/run/openvswitch/ovs-vswitchd.pid" (No such process) compute-1:~$ [2019-08-22 19:34:16,354] 752 DEBUG MainThread ssh.exec_sudo_cmd:: Executing sudo command... [2019-08-22 19:34:16,354] 301 DEBUG MainThread ssh.send :: Send 'sudo chmod 777 /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:16,622] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:16,622] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:16,750] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-1:~$ [2019-08-22 19:34:16,750] 301 DEBUG MainThread ssh.send :: Send 'scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null /home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log sysadmin@controller-0:/home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log' [2019-08-22 19:34:16,973] 423 DEBUG MainThread ssh.expect :: Output: Warning: Permanently added 'controller-0,192.168.204.3' (ECDSA) to the list of known hosts. Release 19.09 ------------------------------------------------------------------------ W A R N I N G *** W A R N I N G *** W A R N I N G *** W A R N I N G *** ------------------------------------------------------------------------ THIS IS A PRIVATE COMPUTER SYSTEM. This computer system including all related equipment, network devices (specifically including Internet access), are provided only for authorized use. All computer systems may be monitored for all lawful purposes, including to ensure that their use is authorized, for management of the system, to facilitate protection against unauthorized access, and to verify security procedures, survivability and operational security. Monitoring includes active attacks by authorized personnel and their entities to test or verify the security of the system. During monitoring, information may be examined, recorded, copied and used for authorized purposes. All information including personal information, placed on or sent over this system may be monitored. Uses of this system, authorized or unauthorized, constitutes consent to monitoring of this system. Unauthorized use may subject you to criminal prosecution. Evidence of any such unauthorized use collected during monitoring may be used for administrative, criminal or other adverse action. Use of this system constitutes consent to monitoring for these purposes. sysadmin@controller-0's password: [2019-08-22 19:34:16,973] 301 DEBUG MainThread ssh.send :: Send 'Li69nux*' [2019-08-22 19:34:17,125] 423 DEBUG MainThread ssh.expect :: Output: 20190822_19-33-test_migrate_stress-compute-1-vswitch.log 0% 0 0.0KB/s --:-- ETA 20190822_19-33-test_migrate_stress-compute-1-vswitch.log 100% 33KB 32.8MB/s 00:00 compute-1:~$ [2019-08-22 19:34:17,126] 301 DEBUG MainThread ssh.send :: Send 'echo $?' [2019-08-22 19:34:17,229] 423 DEBUG MainThread ssh.expect :: Output: 0 compute-1:~$ [2019-08-22 19:34:17,229] 301 DEBUG MainThread ssh.send :: Send '' [2019-08-22 19:34:17,333] 423 DEBUG MainThread ssh.expect :: Output: compute-1:~$ [2019-08-22 19:34:17,333] 301 DEBUG MainThread ssh.send :: Send 'exit' [2019-08-22 19:34:17,437] 423 DEBUG MainThread ssh.expect :: Output: exit logout Connection to compute-1 closed. [sysadmin@controller-0 ~(keystone_admin)]$ [2019-08-22 19:34:17,438] 1257 INFO MainThread ssh.close :: ssh session to compute-1 is closed and returned to parent session 128.224.151.49 [2019-08-22 19:34:17,438] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:34:17,438] 309 DEBUG MainThread common._scp_on_local:: scp cmd: scp -oStrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null sysadmin@128.224.151.49:/home/sysadmin/20190822_19-33-test_migrate_stress-compute-1-vswitch.log /sandbox/AUTOMATION_LOGS/ip_20_27/201908221419/ping_failures/20190822_19-33-test_migrate_stress-compute-1-vswitch.log [2019-08-22 19:34:18,069] 60 DEBUG MainThread conftest.update_results:: ***Failure at test call: /home/svc-cgcsauto/wassp-repos.new/testcases/cgcs/CGCSAuto/keywords/vm_helper.py:1076: utils.exceptions.VMNetworkError: VM network error. ***Details: check_hypervisors = None, boot_source = 'volume', count = 1000 @mark.parametrize(('boot_source', 'count'), [ ('volume', 1000), ('image', 1000) ]) def test_migrate_stress(check_hypervisors, boot_source, count): LOG.tc_step("Launch a VM from {}".format(boot_source)) vm = vm_helper.boot_vm(name='{}-stress'.format(boot_source), cleanup='function', source=boot_source)[1] vm_helper.wait_for_vm_pingable_from_natbox(vm_id=vm) block_mig = True if boot_source == 'image' else False if not block_mig: LOG.tc_step("Attempt to block migration on boot-from-volume VM and ensure if fails") code = vm_helper.live_migrate_vm(vm_id=vm, block_migrate=True)[0] assert code > 0, "Block migration passed unexpectedly for boot-from-volume vm" vm_helper.wait_for_vm_pingable_from_natbox(vm_id=vm) LOG.tc_step("Live migrate and ping vm 1000 times") for i in range(count): LOG.info('Live migration iter{}'.format(i+1)) vm_helper.live_migrate_vm(vm) > vm_helper.wait_for_vm_pingable_from_natbox(vm_id=vm) testcases/stress/nova/test_migrate.py:166: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ vm_id = '698586e7-0ff0-4f52-a9f5-76ff1693539c', timeout = 200, fail_ok = False con_ssh = None, use_fip = False def wait_for_vm_pingable_from_natbox(vm_id, timeout=200, fail_ok=False, con_ssh=None, use_fip=False): """ Wait for ping vm from natbox succeeds. Args: vm_id (str): id of the vm to ping timeout (int): max retry time for pinging vm fail_ok (bool): whether to raise exception if vm cannot be ping'd successfully from natbox within timeout con_ssh (SSHClient): TiS server ssh handle use_fip (bool): whether or not to ping floating ip only if any Returns (bool): True if ping vm succeeded, False otherwise. """ ping_end_time = time.time() + timeout while time.time() < ping_end_time: if ping_vms_from_natbox(vm_ids=vm_id, fail_ok=True, con_ssh=con_ssh, num_pings=3, use_fip=use_fip)[0]: # give it sometime to settle after vm booted and became pingable time.sleep(5) return True else: msg = "Ping from NatBox to vm {} failed for {} seconds.".format(vm_id, timeout) if fail_ok: LOG.warning(msg) return False else: time_stamp = common.get_date_in_format(ssh_client=con_ssh, date_format='%Y%m%d_%H-%M') f_path = '{}/{}-{}'.format(ProjVar.get_var('PING_FAILURE_DIR'), time_stamp, ProjVar.get_var('TEST_NAME')) common.write_to_file(f_path, "=================={}===============\n".format( msg)) ProjVar.set_var(PING_FAILURE=True) get_console_logs(vm_ids=vm_id, sep_file=f_path) network_helper.collect_networking_info(vms=vm_id, sep_file=f_path, time_stamp=time_stamp) > raise exceptions.VMNetworkError(msg) E utils.exceptions.VMNetworkError: VM network error. E Details: Ping from NatBox to vm 698586e7-0ff0-4f52-a9f5-76ff1693539c failed for 200 seconds. keywords/vm_helper.py:1076: VMNetworkError [2019-08-22 19:34:18,959] 793 DEBUG MainThread ssh.close :: connection closed. host: 128.224.186.181, user: svc-cgcsauto. Object ID: 140068246774000 [2019-08-22 19:34:18,959] 1069 INFO MainThread conftest.pytest_unconfigure:: Test Results saved to: /sandbox/AUTOMATION_LOGS/ip_20_27/201908221419/test_results.log [2019-08-22 19:34:18,959] 1534 DEBUG MainThread ssh.get_active_controller:: Getting active controller client for ip_20_27 [2019-08-22 19:34:19,589] 793 DEBUG MainThread ssh.close :: connection closed. host: 128.224.151.49, user: sysadmin. Object ID: 140068249155904