[vbox and bare metal deployment] Zabbix Plugin 2.5.0 fails to deploy in granular_deployment for MOS 7.0

[Bruce Basil Mathews]

Steps to reproduce:

Install MOS 7.0 Fuel master
perform fuel-createmirror -U and fuel-createmirror -M
install zabbix_2.5.0 RPM
create environment installing zabbix with 3 controllers

Deployment fails with the following information:

2016-03-11 01:24:43
 INFO [634] Casting message to Nailgun: {"method"=>"deploy_resp", "args"=>{"task_uuid"=>"16db3ba6-33c0-46f2-94a1-78d6200d12d4", "nodes"=>[{"uid"=>"34", "status"=>"error", "role"=>"hook", "error_type"=>"deploy"}, {"uid"=>"35", "status"=>"error", "role"=>"hook", "error_type"=>"deploy"}, {"uid"=>"36", "status"=>"error", "role"=>"hook", "error_type"=>"deploy"}, {"uid"=>"38", "status"=>"error", "role"=>"hook", "error_type"=>"deploy"}, {"uid"=>"39", "status"=>"error", "role"=>"hook", "error_type"=>"deploy"}, {"uid"=>"40", "status"=>"error", "role"=>"hook", "error_type"=>"deploy"}, {"uid"=>"41", "status"=>"error", "role"=>"hook", "error_type"=>"deploy"}]}}
2016-03-11 01:24:43 ERR [634] Error running RPC method granular_deploy: Failed to execute hook 'zabbix_monitoring-2.5.0' Puppet run failed. Check puppet logs for details --- uids: - '37' parameters: puppet_modules: puppet/modules:/etc/puppet/modules puppet_manifest: puppet/manifests/primary_controller.pp timeout: 1200 cwd: "/etc/fuel/plugins/zabbix_monitoring-2.5/" priority: 2000 fail_on_error: true type: puppet diagnostic_name: zabbix_monitoring-2.5.0 , trace: ["/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/nailgun_hooks.rb:59:in `block in process'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/nailgun_hooks.rb:26:in `each'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/nailgun_hooks.rb:26:in `process'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/deployment_engine/granular_deployment.rb:221:in `post_deployment_actions'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/deployment_engine.rb:75:in `deploy'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/orchestrator.rb:179:in `deploy_cluster'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/orchestrator.rb:45:in `granular_deploy'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/dispatcher.rb:91:in `granular_deploy'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/server.rb:146:in `dispatch_message'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/server.rb:107:in `block in dispatch'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/task_queue.rb:64:in `call'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/task_queue.rb:64:in `block in each'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/task_queue.rb:56:in `each'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/task_queue.rb:56:in `each'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/server.rb:105:in `each_with_index'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/server.rb:105:in `dispatch'", "/usr/lib64/ruby/gems/2.1.0/gems/astute-7.0.0/lib/astute/server/server.rb:89:in `block in perform_main_job'"]
2016-03-11 01:24:43
 WARNING [634] Puppet run failed. Check puppet logs for details
2016-03-11 01:24:43 INFO [634] 16db3ba6-33c0-46f2-94a1-78d6200d12d4: Spent 271.429535061 seconds on puppet run for following nodes(uids): 37
2016-03-11 01:23:03 INFO [634] Retrying to run puppet for following error nodes: 37
2016-03-11 01:21:22 INFO [634] Retrying to run puppet for following error nodes: 37
2016-03-11 01:20:11 INFO [634] Run hook --- uids: - '37' parameters: puppet_modules: puppet/modules:/etc/puppet/modules puppet_manifest: puppet/manifests/primary_controller.pp timeout: 1200 cwd: "/etc/fuel/plugins/zabbix_monitoring-2.5/" priority: 2000 fail_on_error: true type: puppet diagnostic_name: zabbix_monitoring-2.5.0

Error
Deployment has failed. Method granular_deploy. Failed to execute hook 'zabbix_monitoring-2.5.0' Puppet run failed. Check puppet logs for details

---
uids:
- '37'
parameters:
  puppet_modules: puppet/modules:/etc/puppet/modules
  puppet_manifest: puppet/manifests/primary_controller.pp
  timeout: 1200
  cwd: "/etc/fuel/plugins/zabbix_monitoring-2.5/"
priority: 2000
fail_on_error: true
type: puppet
diagnostic_name: zabbix_monitoring-2.5.0
.
Inspect Astute logs for the details

[Swann Croiset (swann-w)]

would be very valuable to attach the puppet log as well, as mentionned in the error above the interresting part is:

2016-03-11 01:24:43 WARNING [634] Puppet run failed. Check puppet logs for details

The node should be the controller node-37 in your env.

thanks.

[Olivier Bourdon (olivierbourdon38)]

Will start looking into this one

[Olivier Bourdon (olivierbourdon38)]

Tried to reproduce this on libvirt and vbox environments

Deployment went perfectly OK on vbox

After patching the 7.0 ISO deployment with https://review.openstack.org/#/c/264457/2/deb-mirror so that the fuel-createmirror -U command can succeed due to https://bugs.launchpad.net/fuel/+bug/1528498, I also got an error but this seemed to be very different from the one mentioned in this bug report:

 ERROR: Unable to communicate with at least one of NTP server, checked the following host(s): ["0.fuel.pool.ntp.org", "1.fuel.pool.ntp.org", "2.fuel.pool.ntp.org"] on node node-1.test.domain.local

Please advise on what is your exact setup (which MU, what other plugins, ...) and provide logs as sugested

[Olivier Bourdon (olivierbourdon38)]

Any news about getting the infos requested in the previous comment ?

[Olivier Bourdon (olivierbourdon38)]

If no news are received on this issue, we will close this bug by the end of the month

[Olivier Bourdon (olivierbourdon38)]

Changing bug status