Failed job: http://jenkins-product.srt.mirantis.net:8080/job/6.1.ubuntu.smoke_neutron/159/
VERSION:
feature_groups:
- mirantis
production: "docker"
release: "6.1"
api: "1.0"
build_number: "235"
build_id: "2015-03-27_03-04-29"
nailgun_sha: "b163f6fc77d6639aaffd9dd992e1ad96951c3bbf"
python-fuelclient_sha: "e5e8389d8d481561a4d7107a99daae07c6ec5177"
astute_sha: "f595715750a2c4820722a96e0236f5c89ca6521c"
fuellib_sha: "9c7716bc2ce6075065d7d9dcf96f4c94662c0b56"
ostf_sha: "a4cf5f218c6aea98105b10c97a4aed8115c15867"
fuelmain_sha: "320b5f46fc1b2798f9e86ed7df51d3bda1686c10"
Deployment passed without errors, but after it in astute log we can see next errors:
2015-03-27T04:50:44 info: [673] Processing RPC call 'verify_networks'
2015-03-27T04:50:44 debug: [673] f6c022a6-c64b-4dda-9062-d565f8db3565: MC agent 'systemtype', method 'get_type', results: {:sender=>"3", :statuscode=>0, :statusmsg=>"OK", :data=>{:node_type=>"target"}}
2015-03-27T04:50:44 debug: [673] f6c022a6-c64b-4dda-9062-d565f8db3565: MC agent 'systemtype', method 'get_type', results: {:sender=>"1", :statuscode=>0, :statusmsg=>"OK", :data=>{:node_type=>"target"}}
2015-03-27T04:50:44 debug: [673] f6c022a6-c64b-4dda-9062-d565f8db3565: MC agent 'systemtype', method 'get_type', results: {:sender=>"2", :statuscode=>0, :statusmsg=>"OK", :data=>{:node_type=>"target"}}
2015-03-27T04:51:15 debug: [673] f6c022a6-c64b-4dda-9062-d565f8db3565: MC agent 'net_probe', method 'dhcp_discover', results: {:sender=>"1", :statuscode=>0, :statusmsg=>"OK", :data=>{:err=>"Spawning listener for eth1 failed.\nNetwork for iface eth1 is down.Network for iface eth1 is down.Network for iface eth1 is down.", :out=>"[{\"server_id\": \"10.109.0.2\", \"iface\": \"eth0\", \"yiaddr\": \"10.109.0.4\", \"mac\": \"64:02:4b:22:8c:23\", \"server_ip\": \"10.109.0.2\", \"dport\": 67, \"message\": \"offer\", \"gateway\": \"0.0.0.0\"}]\n", :status=>0}}
2015-03-27T04:51:15 debug: [673] f6c022a6-c64b-4dda-9062-d565f8db3565: MC agent 'net_probe', method 'dhcp_discover', results: {:sender=>"2", :statuscode=>0, :statusmsg=>"OK", :data=>{:err=>"", :out=>"[{\"server_id\": \"10.109.0.2\", \"iface\": \"eth0\", \"yiaddr\": \"10.109.0.3\", \"mac\": \"64:02:4b:22:8c:23\", \"server_ip\": \"10.109.0.2\", \"dport\": 67, \"message\": \"offer\", \"gateway\": \"0.0.0.0\"}]\n", :status=>0}}
2015-03-27T04:51:15 debug: [673] f6c022a6-c64b-4dda-9062-d565f8db3565: MC agent 'net_probe', method 'dhcp_discover', results: {:sender=>"3", :statuscode=>0, :statusmsg=>"OK", :data=>{:err=>"Spawning listener for eth1 failed.\nNetwork for iface eth1 is down.Network for iface eth1 is down.Network for iface eth1 is down.", :out=>"[{\"server_id\": \"10.109.0.2\", \"iface\": \"eth0\", \"yiaddr\": \"10.109.0.5\", \"mac\": \"64:02:4b:22:8c:23\", \"server_ip\": \"10.109.0.2\", \"dport\": 67, \"message\": \"offer\", \"gateway\": \"0.0.0.0\"}]\n", :status=>0}}
2015-03-27T04:51:15 debug: [673] DHCP checker received: #<MCollective::RPC::Result:0x000000027d44c8 @agent="net_probe", @action="dhcp_discover", @results={:sender=>"1", :statuscode=>0, :statusmsg=>"OK", :data=>{:err=>"Spawning listener for eth1 failed.\nNetwork for iface eth1 is down.Network for iface eth1 is down.Network for iface eth1 is down.", :out=>"[{\"server_id\": \"10.109.0.2\", \"iface\": \"eth0\", \"yiaddr\": \"10.109.0.4\", \"mac\": \"64:02:4b:22:8c:23\", \"server_ip\": \"10.109.0.2\", \"dport\": 67, \"message\": \"offer\", \"gateway\": \"0.0.0.0\"}]\n", :status=>0}}>
2015-03-27T04:51:15 debug: [673] DHCP checker received: #<MCollective::RPC::Result:0x000000027a9d18 @agent="net_probe", @action="dhcp_discover", @results={:sender=>"2", :statuscode=>0, :statusmsg=>"OK", :data=>{:err=>"", :out=>"[{\"server_id\": \"10.109.0.2\", \"iface\": \"eth0\", \"yiaddr\": \"10.109.0.3\", \"mac\": \"64:02:4b:22:8c:23\", \"server_ip\": \"10.109.0.2\", \"dport\": 67, \"message\": \"offer\", \"gateway\": \"0.0.0.0\"}]\n", :status=>0}}>
2015-03-27T04:51:15 debug: [673] DHCP checker received: #<MCollective::RPC::Result:0x0000000278afa8 @agent="net_probe", @action="dhcp_discover", @results={:sender=>"3", :statuscode=>0, :statusmsg=>"OK", :data=>{:err=>"Spawning listener for eth1 failed.\nNetwork for iface eth1 is down.Network for iface eth1 is down.Network for iface eth1 is down.", :out=>"[{\"server_id\": \"10.109.0.2\", \"iface\": \"eth0\", \"yiaddr\": \"10.109.0.5\", \"mac\": \"64:02:4b:22:8c:23\", \"server_ip\": \"10.109.0.2\", \"dport\": 67, \"message\": \"offer\", \"gateway\": \"0.0.0.0\"}]\n", :status=>0}}>
2015-03-27T04:51:15 info: [673] Casting message to Nailgun: {"method"=>"check_dhcp_resp", "args"=>{"task_uuid"=>"f6c022a6-c64b-4dda-9062-d565f8db3565", "status"=>"ready", "progress"=>100, "nodes"=>[{:uid=>"1", :status=>"ready", :data=>[{"server_id"=>"10.109.0.2", "iface"=>"eth0", "yiaddr"=>"10.109.0.4", "mac"=>"64:02:4b:22:8c:23", "server_ip"=>"10.109.0.2", "dport"=>67, "message"=>"offer", "gateway"=>"0.0.0.0"}]}, {:uid=>"2", :status=>"ready", :data=>[{"server_id"=>"10.109.0.2", "iface"=>"eth0", "yiaddr"=>"10.109.0.3", "mac"=>"64:02:4b:22:8c:23", "server_ip"=>"10.109.0.2", "dport"=>67, "message"=>"offer", "gateway"=>"0.0.0.0"}]}, {:uid=>"3", :status=>"ready", :data=>[{"server_id"=>"10.109.0.2", "iface"=>"eth0", "yiaddr"=>"10.109.0.5", "mac"=>"64:02:4b:22:8c:23", "server_ip"=>"10.109.0.2", "dport"=>67, "message"=>"offer", "gateway"=>"0.0.0.0"}]}]}}
2015-03-27T04:51:16 debug: [673] 5f313591-9a16-459f-9ba0-26a7817f61ba: MC agent 'systemtype', method 'get_type', results: {:sender=>"2", :statuscode=>0, :statusmsg=>"OK", :data=>{:node_type=>"target"}}
2015-03-27T04:51:16 debug: [673] 5f313591-9a16-459f-9ba0-26a7817f61ba: MC agent 'systemtype', method 'get_type', results: {:sender=>"1", :statuscode=>0, :statusmsg=>"OK", :data=>{:node_type=>"target"}}
2015-03-27T04:51:16 debug: [673] 5f313591-9a16-459f-9ba0-26a7817f61ba: MC agent 'systemtype', method 'get_type', results: {:sender=>"3", :statuscode=>0, :statusmsg=>"OK", :data=>{:node_type=>"target"}}
2015-03-27T04:51:16 debug: [673] 5f313591-9a16-459f-9ba0-26a7817f61ba: MC agent 'rpcutil', method 'inventory', results: {:sender=>"1", :statuscode=>0, :statusmsg=>"OK", :data=>{:agents=>["systemtype", "discovery", "net_probe", "fake", "puppetd", "puppetsync", "erase_node", "rpcutil", "execute_shell_command", "uploadfile"], :facts=>{"mcollective"=>"1"}, :classes=>[], :version=>"@DEVELOPMENT_VERSION@", :main_collective=>"mcollective", :collectives=>["mcollective"], :data_plugins=>["agent_data", "fstat_data"]}}
2015-03-27T04:51:16 debug: [673] 5f313591-9a16-459f-9ba0-26a7817f61ba: MC agent 'rpcutil', method 'inventory', results: {:sender=>"3", :statuscode=>0, :statusmsg=>"OK", :data=>{:agents=>["systemtype", "discovery", "net_probe", "fake", "puppetd", "puppetsync", "erase_node", "rpcutil", "execute_shell_command", "uploadfile"], :facts=>{"mcollective"=>"1"}, :classes=>[], :version=>"@DEVELOPMENT_VERSION@", :main_collective=>"mcollective", :collectives=>["mcollective"], :data_plugins=>["agent_data", "fstat_data"]}}
2015-03-27T04:51:16 debug: [673] 5f313591-9a16-459f-9ba0-26a7817f61ba: MC agent 'rpcutil', method 'inventory', results: {:sender=>"2", :statuscode=>0, :statusmsg=>"OK", :data=>{:agents=>["systemtype", "discovery", "net_probe", "fake", "puppetd", "puppetsync", "erase_node", "rpcutil", "execute_shell_command", "uploadfile"], :facts=>{"mcollective"=>"1"}, :classes=>[], :version=>"@DEVELOPMENT_VERSION@", :main_collective=>"mcollective", :collectives=>["mcollective"], :data_plugins=>["agent_data", "fstat_data"]}}
2015-03-27T04:51:16 err: [673] Retrying RPC client instantiation after exception: #<RuntimeError: Could not find any hosts in discovery data provided>
2015-03-27T04:51:21 err: [673] Retrying RPC client instantiation after exception: #<RuntimeError: Could not find any hosts in discovery data provided>
2015-03-27T04:51:26 err: [673] No more retries for MCollective client instantiation after exception:
["/usr/lib64/ruby/gems/2.1.0/gems/mcollective-client-2.4.1/lib/mcollective/rpc/client.rb:498:in `discover'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/mclient.rb:152:in `initialize_mclient'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/mclient.rb:41:in `initialize'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/version.rb:34:in `new'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/version.rb:34:in `get_versions'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/version.rb:51:in `split_on_version'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/network.rb:40:in `check_network'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/orchestrator.rb:105:in `verify_networks'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/dispatcher.rb:135:in `verify_networks'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:142:in `dispatch_message'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:103:in `block in dispatch'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:64:in `call'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:64:in `block in each'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:56:in `each'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:56:in `each'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `each_with_index'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `dispatch'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:85:in `block in perform_main_job'"]
2015-03-27T04:51:26 err: [673] Error running RPC method verify_networks: #<RuntimeError: Could not find any hosts in discovery data provided>
, trace:
["/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/mclient.rb:163:in `rescue in initialize_mclient'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/mclient.rb:147:in `initialize_mclient'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/mclient.rb:41:in `initialize'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/version.rb:34:in `new'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/version.rb:34:in `get_versions'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/version.rb:51:in `split_on_version'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/network.rb:40:in `check_network'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/orchestrator.rb:105:in `verify_networks'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/dispatcher.rb:135:in `verify_networks'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:142:in `dispatch_message'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:103:in `block in dispatch'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:64:in `call'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:64:in `block in each'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:56:in `each'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:56:in `each'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `each_with_index'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `dispatch'",
"/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:85:in `block in perform_main_job'"]
2015-03-27T04:51:26 info: [673] Casting message to Nailgun: {"method"=>"verify_networks_resp", "args"=>{"task_uuid"=>"5f313591-9a16-459f-9ba0-26a7817f61ba", "status"=>"error", "error"=>"Method verify_networks. #<RuntimeError: Could not find any hosts in discovery data provided>\n.\nInspect Astute logs for the details"}}
Seems the reason of failure is:
"Spawning listener for eth1 failed.\nNetwork for iface eth1 is down."
But message "Could not find any hosts in discovery data provided" is not related to this.