vmtp

nuttcp-t: v7.3.2: Error: connect: No route to host

Bug #1647021 reported by xuchao
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
vmtp
New
Undecided
Unassigned

Bug Description

1）Run VMTP test
# docker run --rm -it -v $PWD:/tmp/vmtp -v $HOME/.ssh:/tmp/ssh berrypatch/vmtp vmtp -d -c /tmp/vmtp/vmtp.cfg -r /tmp/vmtp/admin-openrc.sh -p admin --protocol T --json /tmp/vmtp/test.json

2016-12-03 06:41:56,900 INFO Using http://172.16.71.xxx:5000/v2.0/
2016-12-03 06:41:56,900 INFO VM public key: /tmp/ssh/id_rsa.pub
2016-12-03 06:41:56,900 INFO VM private key: /tmp/ssh/id_rsa
2016-12-03 06:41:57,755 INFO Found image centos7 to launch VM, will continue
/usr/local/lib/python2.7/dist-packages/keystoneclient/service_catalog.py:200: UserWarning: Providing attr without filter_value to get_urls() is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release. Either both should be provided or neither should be provided.
  'Providing attr without filter_value to get_urls() is '
2016-12-03 06:41:57,994 INFO Using external network: public_net
2016-12-03 06:41:58,200 INFO Created ext router router.
2016-12-03 06:41:58,377 INFO Created internal network: pns-internal-net.
2016-12-03 06:41:58,553 INFO Created internal network: pns-internal-net2.
2016-12-03 06:41:58,881 DEBUG Ext router associated to pns-internal-net
2016-12-03 06:41:59,127 DEBUG Ext router associated to pns-internal-net2
2016-12-03 06:41:59,349 INFO OpenStack agent: Open vSwitch agent
2016-12-03 06:41:59,349 INFO OpenStack network type: vlan
2016-12-03 06:42:00,067 INFO [TestServer1] Creating server VM...
2016-12-03 06:42:00,068 DEBUG [TestServer1] Starting on zone test_kvm:nc106.xxxx.com
2016-12-03 06:42:00,722 DEBUG [TestServer1] VM status=BUILD, retrying 1 of 50...
2016-12-03 06:42:03,204 DEBUG [TestServer1] VM status=BUILD, retrying 2 of 50...
2016-12-03 06:42:05,402 DEBUG [TestServer1] VM status=BUILD, retrying 3 of 50...
2016-12-03 06:42:07,505 DEBUG [TestServer1] VM status=BUILD, retrying 4 of 50...
2016-12-03 06:42:09,612 DEBUG [TestServer1] VM status=BUILD, retrying 5 of 50...
2016-12-03 06:42:11,735 DEBUG [TestServer1] VM status=BUILD, retrying 6 of 50...
2016-12-03 06:42:13,888 DEBUG [TestServer1] VM status=BUILD, retrying 7 of 50...
2016-12-03 06:42:15,991 DEBUG [TestServer1] VM status=BUILD, retrying 8 of 50...
2016-12-03 06:42:18,103 DEBUG [TestServer1] VM status=BUILD, retrying 9 of 50...
2016-12-03 06:42:20,207 DEBUG [TestServer1] VM status=BUILD, retrying 10 of 50...
2016-12-03 06:42:22,383 DEBUG [TestServer1] VM status=BUILD, retrying 11 of 50...
2016-12-03 06:42:24,488 DEBUG [TestServer1] VM status=BUILD, retrying 12 of 50...
2016-12-03 06:42:26,606 DEBUG [TestServer1] VM status=BUILD, retrying 13 of 50...
2016-12-03 06:42:28,715 DEBUG [TestServer1] VM status=BUILD, retrying 14 of 50...
2016-12-03 06:42:30,819 DEBUG [TestServer1] VM status=BUILD, retrying 15 of 50...
2016-12-03 06:42:32,932 DEBUG [TestServer1] VM status=BUILD, retrying 16 of 50...
2016-12-03 06:42:35,087 DEBUG [TestServer1] VM status=BUILD, retrying 17 of 50...
2016-12-03 06:42:37,193 DEBUG [TestServer1] VM status=BUILD, retrying 18 of 50...
2016-12-03 06:42:39,312 DEBUG [TestServer1] VM status=BUILD, retrying 19 of 50...
2016-12-03 06:42:41,418 DEBUG [TestServer1] VM status=BUILD, retrying 20 of 50...
2016-12-03 06:42:43,522 DEBUG [TestServer1] VM status=BUILD, retrying 21 of 50...
2016-12-03 06:42:45,652 DEBUG [TestServer1] VM status=BUILD, retrying 22 of 50...
2016-12-03 06:42:47,759 DEBUG [TestServer1] VM status=BUILD, retrying 23 of 50...
2016-12-03 06:42:49,867 DEBUG [TestServer1] VM status=BUILD, retrying 24 of 50...
2016-12-03 06:42:51,974 DEBUG [TestServer1] VM status=BUILD, retrying 25 of 50...
2016-12-03 06:42:54,080 DEBUG [TestServer1] VM status=BUILD, retrying 26 of 50...
2016-12-03 06:42:56,187 DEBUG [TestServer1] VM status=BUILD, retrying 27 of 50...
2016-12-03 06:42:58,296 DEBUG [TestServer1] VM status=BUILD, retrying 28 of 50...
2016-12-03 06:43:00,404 DEBUG [TestServer1] VM status=BUILD, retrying 29 of 50...
2016-12-03 06:43:02,537 DEBUG [TestServer1] VM status=BUILD, retrying 30 of 50...
2016-12-03 06:43:04,644 DEBUG [TestServer1] VM status=BUILD, retrying 31 of 50...
2016-12-03 06:43:06,851 DEBUG [TestServer1] VM status=BUILD, retrying 32 of 50...
2016-12-03 06:43:08,957 DEBUG [TestServer1] VM status=BUILD, retrying 33 of 50...
2016-12-03 06:43:11,062 DEBUG [TestServer1] VM status=BUILD, retrying 34 of 50...
2016-12-03 06:43:13,168 DEBUG [TestServer1] VM status=BUILD, retrying 35 of 50...
2016-12-03 06:43:15,278 DEBUG [TestServer1] VM status=BUILD, retrying 36 of 50...
2016-12-03 06:43:17,382 DEBUG [TestServer1] VM status=BUILD, retrying 37 of 50...
2016-12-03 06:43:19,490 DEBUG [TestServer1] VM status=BUILD, retrying 38 of 50...
2016-12-03 06:43:21,597 DEBUG [TestServer1] VM status=BUILD, retrying 39 of 50...
2016-12-03 06:43:23,853 INFO [TestServer1] Associating floating IP 172.16.71.xxx
2016-12-03 06:43:24,499 INFO [TestServer1] Internal network IP: 10.0.0.2
2016-12-03 06:43:24,499 INFO [TestServer1] SSH IP: 172.16.71.xxx
2016-12-03 06:43:24,499 INFO [TestServer1] Setup SSH for root@172.16.71.xxx
2016-12-03 06:44:01,364 INFO [TestServer1] Installing nuttcp-7.3.2...
2016-12-03 06:44:01,419 DEBUG [TestServer1] Copying nuttcp-7.3.2 to target...
2016-12-03 06:44:01,725 DEBUG [TestServer1] Starting nuttcp-7.3.2 server...
2016-12-03 06:44:01,812 DEBUG [TestServer1]
2016-12-03 06:44:01,896 INFO [TestClient1] Creating client VM...
2016-12-03 06:44:01,897 DEBUG [TestClient1] Starting on zone test_kvm:nc106.xxxx.com
2016-12-03 06:44:03,131 DEBUG [TestClient1] VM status=BUILD, retrying 1 of 50...
2016-12-03 06:44:05,240 DEBUG [TestClient1] VM status=BUILD, retrying 2 of 50...
2016-12-03 06:44:07,347 DEBUG [TestClient1] VM status=BUILD, retrying 3 of 50...
2016-12-03 06:44:09,451 DEBUG [TestClient1] VM status=BUILD, retrying 4 of 50...
2016-12-03 06:44:11,663 DEBUG [TestClient1] VM status=BUILD, retrying 5 of 50...
2016-12-03 06:44:13,771 DEBUG [TestClient1] VM status=BUILD, retrying 6 of 50...
2016-12-03 06:44:15,974 DEBUG [TestClient1] VM status=BUILD, retrying 7 of 50...
2016-12-03 06:44:18,077 DEBUG [TestClient1] VM status=BUILD, retrying 8 of 50...
2016-12-03 06:44:20,188 DEBUG [TestClient1] VM status=BUILD, retrying 9 of 50...
2016-12-03 06:44:22,293 DEBUG [TestClient1] VM status=BUILD, retrying 10 of 50...
2016-12-03 06:44:24,396 DEBUG [TestClient1] VM status=BUILD, retrying 11 of 50...
2016-12-03 06:44:26,500 DEBUG [TestClient1] VM status=BUILD, retrying 12 of 50...
2016-12-03 06:44:28,611 DEBUG [TestClient1] VM status=BUILD, retrying 13 of 50...
2016-12-03 06:44:30,715 DEBUG [TestClient1] VM status=BUILD, retrying 14 of 50...
2016-12-03 06:44:32,822 DEBUG [TestClient1] VM status=BUILD, retrying 15 of 50...
2016-12-03 06:44:34,925 DEBUG [TestClient1] VM status=BUILD, retrying 16 of 50...
2016-12-03 06:44:37,032 DEBUG [TestClient1] VM status=BUILD, retrying 17 of 50...
2016-12-03 06:44:39,136 DEBUG [TestClient1] VM status=BUILD, retrying 18 of 50...
2016-12-03 06:44:41,252 DEBUG [TestClient1] VM status=BUILD, retrying 19 of 50...
2016-12-03 06:44:43,359 DEBUG [TestClient1] VM status=BUILD, retrying 20 of 50...
2016-12-03 06:44:45,466 DEBUG [TestClient1] VM status=BUILD, retrying 21 of 50...
2016-12-03 06:44:47,570 DEBUG [TestClient1] VM status=BUILD, retrying 22 of 50...
2016-12-03 06:44:49,815 DEBUG [TestClient1] VM status=BUILD, retrying 23 of 50...
2016-12-03 06:44:51,920 DEBUG [TestClient1] VM status=BUILD, retrying 24 of 50...
2016-12-03 06:44:54,027 DEBUG [TestClient1] VM status=BUILD, retrying 25 of 50...
2016-12-03 06:44:56,133 DEBUG [TestClient1] VM status=BUILD, retrying 26 of 50...
2016-12-03 06:44:58,243 DEBUG [TestClient1] VM status=BUILD, retrying 27 of 50...
2016-12-03 06:45:00,401 DEBUG [TestClient1] VM status=BUILD, retrying 28 of 50...
2016-12-03 06:45:02,503 DEBUG [TestClient1] VM status=BUILD, retrying 29 of 50...
2016-12-03 06:45:04,630 DEBUG [TestClient1] VM status=BUILD, retrying 30 of 50...
2016-12-03 06:45:06,733 DEBUG [TestClient1] VM status=BUILD, retrying 31 of 50...
2016-12-03 06:45:08,938 DEBUG [TestClient1] VM status=BUILD, retrying 32 of 50...
2016-12-03 06:45:11,047 DEBUG [TestClient1] VM status=BUILD, retrying 33 of 50...
2016-12-03 06:45:13,151 DEBUG [TestClient1] VM status=BUILD, retrying 34 of 50...
2016-12-03 06:45:15,260 DEBUG [TestClient1] VM status=BUILD, retrying 35 of 50...
2016-12-03 06:45:17,372 DEBUG [TestClient1] VM status=BUILD, retrying 36 of 50...
2016-12-03 06:45:19,480 DEBUG [TestClient1] VM status=BUILD, retrying 37 of 50...
2016-12-03 06:45:21,583 DEBUG [TestClient1] VM status=BUILD, retrying 38 of 50...
2016-12-03 06:45:23,712 DEBUG [TestClient1] VM status=BUILD, retrying 39 of 50...
2016-12-03 06:45:25,824 DEBUG [TestClient1] VM status=BUILD, retrying 40 of 50...
2016-12-03 06:45:28,020 INFO [TestClient1] Associating floating IP 172.16.71.xxx
2016-12-03 06:45:28,592 INFO [TestClient1] Internal network IP: 10.0.0.4
2016-12-03 06:45:28,592 INFO [TestClient1] SSH IP: 172.16.71.xxx
2016-12-03 06:45:28,592 INFO [TestClient1] Setup SSH for root@172.16.71.xxx
2016-12-03 06:46:17,648 INFO [TestClient1] Installing nuttcp-7.3.2...
2016-12-03 06:46:17,701 DEBUG [TestClient1] Copying nuttcp-7.3.2 to target...
2016-12-03 06:46:17,879 INFO ============================================================
2016-12-03 06:46:17,879 INFO Flow 1: VM to VM same network fixed IP (intra-node)
2016-12-03 06:46:17,879 INFO [TestClient1] Measuring TCP Throughput (packet size=65536)...
2016-12-03 06:46:17,879 DEBUG [TestClient1] /tmp/nuttcp-7.3.2 -T10 -l65536 -p5001 -P5002 -fparse 10.0.0.2
2016-12-03 06:46:17,998 INFO [TestClient1] ERROR cmd=/tmp/nuttcp-7.3.2 -T10 -l65536 -p5001 -P5002 -fparse 10.0.0.2
2016-12-03 06:46:17,998 INFO [TestClient1] error=nuttcp-t: Info="attempting_to_switch_to_deprecated_classic_mode"
nuttcp-t: Info="will_use_less_reliable_transmitter_side_statistics"
nuttcp-t: v7.3.2: Error: connect: No route to host
errno=113

2016-12-03 06:46:18,001 ERROR Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/vmtp/vmtp.py", line 427, in run
    self.measure_vm_flows()
  File "/usr/local/lib/python2.7/dist-packages/vmtp/vmtp.py", line 378, in measure_vm_flows
    self.server.internal_ip)
  File "/usr/local/lib/python2.7/dist-packages/vmtp/vmtp.py", line 353, in measure_flow
    az_to=self.server.az)
  File "/usr/local/lib/python2.7/dist-packages/vmtp/perf_instance.py", line 80, in run_client
    bidirectional=bidirectional)
  File "/usr/local/lib/python2.7/dist-packages/vmtp/nuttcp_tool.py", line 66, in run_client
    length=pkt_size)
  File "/usr/local/lib/python2.7/dist-packages/vmtp/nuttcp_tool.py", line 206, in run_client_dir
    match = re.search(re_tcp, cmd_out)
  File "/usr/lib/python2.7/re.py", line 142, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or buffer

Revision history for this message
Yichen Wang (yicwang) wrote :

Hi, Xuchao,

Sometimes when traceback is shown, it doesn't really mean there is a bug... Regard of this, I would say there might be something wrong with your OpenStack.

What VMTP trying to do now is, creating two VMs on the same compute node and same network, and start the nuttcp between this two VM. One thing you could try is, run VMTP with --stop-on-error. Once the error occurs, everything will be remained as is without cleanup. Then you could try to ssh to one of the VM and try to ping and diagnose more on it.

Regards,
Yichen

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.