Error running RPC method granular_deploy: Deployment failed on nodes

[lev]

[473] Casting message to Nailgun: {"method"=>"deploy_resp", "args"=>{"task_uuid"=>"7ee16f44-a4ee-42b0-b2b1-fd83a18860d0", "status"=>"error", "error"=>"Method granular_deploy. Deployment failed on nodes 16.\nInspect Astute logs for the details"}}
2015-09-08 08:58:15 ERR
[473] Error running RPC method granular_deploy: Deployment failed on nodes 16, trace:
["/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:68:in `block (3 levels) in deploy'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:51:in `each_slice'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:51:in `block (2 levels) in deploy'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:49:in `each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:49:in `block in deploy'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:45:in `each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:45:in `deploy'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/orchestrator.rb:179:in `deploy_cluster'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/orchestrator.rb:56:in `granular_deploy'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/dispatcher.rb:111:in `granular_deploy'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:142:in `dispatch_message'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:103:in `block in dispatch'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/task_queue.rb:64:in `call'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/task_queue.rb:64:in `block in each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/task_queue.rb:56:in `each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/task_queue.rb:56:in `each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:101:in `each_with_index'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:101:in `dispatch'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:85:in `block in perform_main_job'"]
2015-09-08 08:58:15 INFO
[473] Casting message to Nailgun: {"method"=>"deploy_resp", "args"=>{"task_uuid"=>"7ee16f44-a4ee-42b0-b2b1-fd83a18860d0", "nodes"=>[{"uid"=>"16", "status"=>"error", "error_type"=>"deploy", "role"=>"primary-mongo", "task"=>{"priority"=>700, "type"=>"puppet", "uids"=>["16"], "parameters"=>{"puppet_modules"=>"/etc/puppet/modules", "puppet_manifest"=>"/etc/puppet/modules/osnailyfacter/modular/netconfig/connectivity_tests.pp", "timeout"=>3600, "cwd"=>"/"}}, "progress"=>100}]}}
2015-09-08 08:58:15 INFO
[473] 7ee16f44-a4ee-42b0-b2b1-fd83a18860d0: Finished deployment of nodes => roles: {"16"=>"primary-mongo"}
2015-09-08 08:58:15 ERR
[473] No more tasks will be executed on the node 16
2015-09-08 08:58:15 ERR
[473] Task '{"priority"=>700, "type"=>"puppet", "uids"=>["16"], "parameters"=>{"puppet_modules"=>"/etc/puppet/modules", "puppet_manifest"=>"/etc/puppet/modules/osnailyfacter/modular/netconfig/connectivity_tests.pp", "timeout"=>3600, "cwd"=>"/"}}' failed on node 16

[Nastya Urlapova (aurlapova)]

Lev, can you also attach diagnostic snapshot and fuel version?

[lev (levw)]

Hey Nastya,
What do you mean in diagnostic snapshot? How I get it?
The fuel version is 6.1.

Thanks a lot!
Lev Wainstein.

-----Original Message-----
From: <email address hidden> [mailto:<email address hidden>] On Behalf Of Nastya Urlapova
Sent: Tuesday, September 08, 2015 12:56 PM
To: Lev Wainstein
Subject: [Bug 1493290] Re: Error running RPC method granular_deploy: Deployment failed on nodes

Lev, can you also attach diagnostic snapshot and fuel version?

--
You received this bug notification because you are subscribed to the bug report.
https://bugs.launchpad.net/bugs/1493290

Title:
  Error running RPC method granular_deploy: Deployment failed on nodes

Status in Fuel for OpenStack:
  New

Bug description:

  [473] Casting message to Nailgun: {"method"=>"deploy_resp", "args"=>{"task_uuid"=>"7ee16f44-a4ee-42b0-b2b1-fd83a18860d0", "status"=>"error", "error"=>"Method granular_deploy. Deployment failed on nodes 16.\nInspect Astute logs for the details"}}
  2015-09-08 08:58:15 ERR
  [473] Error running RPC method granular_deploy: Deployment failed on nodes 16, trace:
  ["/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:68:in `block (3 levels) in deploy'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:51:in `each_slice'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:51:in `block (2 levels) in deploy'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:49:in `each'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:49:in `block in deploy'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:45:in `each'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/deployment_engine.rb:45:in `deploy'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/orchestrator.rb:179:in `deploy_cluster'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/orchestrator.rb:56:in `granular_deploy'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/dispatcher.rb:111:in `granular_deploy'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:142:in `dispatch_message'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:103:in `block in dispatch'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/task_queue.rb:64:in `call'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/task_queue.rb:64:in `block in each'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/task_queue.rb:56:in `each'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/task_queue.rb:56:in `each'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:101:in `each_with_index'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:101:in `dispatch'",
   "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.1.0/lib/astute/server/server.rb:85:in `block in perform_main_job'"]
  2015-09-08 08:58:15 INFO
  [473] Casting message to Nailgun: {"method"=>"deploy_resp", "args"=>...

[Matthew Mosesohn (raytrac3r)]

Lev, sorry for the delays. You can generate a diagnostic snapshot if you go to Support at the top of Fuel UI, then run "Generate diagnostic snapshot". Alternatively, you can do it using Fuel CLI: fuel snapshot

[lev (levw)]

thanks Andrew,
here is the snapshot.

[Kyrylo Galanov (kgalanov)]

Hello Lev,

It was temporary a network / availability issue. For some reason archive.ubuntu.com or it's mirror was offline. Nothing to do with fuel-library here.

---
2015-09-08 08:58:13 +0000 Puppet (err): ERROR: Unable to fetch url 'http://archive.ubuntu.com/ubuntu/', error 'Network is unreachable - connect(2)'. Please verify node connectivity to this URL, or remove it from the settings page if it is invalid. on node node-16.checkpoint.com
---

Best regards,
Kyrylo