Deploy has failed.Could not find data item management_vrouter_vip in any Hiera data file and no default supplied at /etc/puppet/modules/osnailyfacter/modular/netconfig/configure_default_route.pp:5 on node node-4.domain.tld

[Anastasia Palkina]

Issue found on release 7.0 ISO #301 + fuel-provisioning-scripts-7.0.0-7681.1.gite013fd0.noarch

1. Create new environment (Ubuntu)
2. Choose Neutron, tunnelling segmentation
3. Choose Ceph RBD for ephemeral volumes and uncheck Cinder LVM over iSCSI for volumes
4. Add 3 controllers, 1 compute, 2 ceph
5. Deploy the environment. It has failed with error in astute.log:

2015-11-06 14:19:21 ERR [598] Task '{"priority"=>100, "type"=>"puppet", "uids"=>["4"], "parameters"=>{"puppet_modules"=>"/etc/puppet/modules", "puppet_manifest"=>"/etc/puppet/modules/osnailyfacter/modular/fuel_pkgs/fuel_pkgs.pp", "timeout"=>600, "cwd"=>"/"}}' failed on node 4

Full astute error: http://paste.openstack.org/show/478189/

Also there is error in puppet.log on node-4:

2015-11-06 14:26:01 +0000 Puppet (err): Could not find data item management_vrouter_vip in any Hiera data file and no default supplied at /etc/puppet/modules/osnailyfacter/modular/netconfig/configure_default_route.pp:5 on node node-4.domain.tld
/usr/lib/ruby/vendor_ruby/hiera_puppet.rb:14:in `lookup'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions/hiera.rb:23:in `block in <module:Functions>'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:144:in `block (2 levels) in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/none.rb:6:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:31:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/functions.rb:137:in `block in newfunction'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/function.rb:42:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:62:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/vardef.rb:14:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:62:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:15:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `each'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast/block_expression.rb:10:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/ast.rb:62:in `safeevaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/type.rb:121:in `evaluate_code'
/usr/lib/ruby/vendor_ruby/puppet/parser/resource.rb:79:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:366:in `evaluate_main'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:110:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/none.rb:6:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:31:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:110:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/parser/compiler.rb:23:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:116:in `block (2 levels) in compile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler/none.rb:6:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/util/profiler.rb:31:in `profile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:114:in `block in compile'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
/usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:113:in `compile'
/usr/lib/ruby/vendor_ruby/puppet/indirector/catalog/compiler.rb:46:in `find'
/usr/lib/ruby/vendor_ruby/puppet/indirector/indirection.rb:201:in `find'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:204:in `main'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:146:in `run_command'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block (2 levels) in run'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:470:in `plugin_hook'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block in run'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:478:in `exit_on_fail'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:137:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:91:in `execute'
/usr/bin/puppet:4:in `<main>'

Logs are here: https://drive.google.com/a/mirantis.com/file/d/0B6SjzarTGFxacTVZZTloeWpRSjg/view?usp=sharing

Controllers - node-15,14,12
Compute - node-4
Ceph - node-5,8

[slava valyavskiy (slava-val-al)]

'/etc/hiera/globals.yaml' file is not present on the failed node.

[slava valyavskiy (slava-val-al)]

I didn't find any mentions about 'globals.pp' task in puppet log for affected node/

[slava valyavskiy (slava-val-al)]

It seems that 'fuel_pkgs' has been failed on node-4.

2015-11-06T14:19:21 debug: [598] {"nodes"=>[{"status"=>"error", "error_type"=>"deploy", "uid"=>"4", "role"=>"compute"}]}
2015-11-06T14:19:21 err: [598] Task '{"priority"=>100, "type"=>"puppet", "uids"=>["4"], "parameters"=>{"puppet_modules"=>"/etc/puppet/modules", "puppet_manifest"=>"/etc/puppet/modules/osnailyfacter/modular/fuel_pkgs/fuel_pkgs.pp", "timeout"=>600, "cwd"=>"/"}}' failed on node 4

[slava valyavskiy (slava-val-al)]

2015-11-06 14:18:48 +0000 Puppet (err): Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install fuel-misc' returned 100: Reading package lists...
Building dependency tree...
Reading state information...
The following NEW packages will be installed:
  fuel-misc
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 3478 B of archives.
After this operation, 34.8 kB of additional disk space will be used.
WARNING: The following packages cannot be authenticated!
  fuel-misc
Authentication warning overridden.
Err http://mirror.fuel-infra.org/mos-repos/ubuntu/7.0/ mos7.0-proposed/main fuel-misc all 7.0.0-7245.1.git4f02781
  404 Not Found [IP: 208.78.244.194 80]
E: Failed to fetch http://mirror.fuel-infra.org/mos-repos/ubuntu/7.0/pool/main/f/fuel-library7.0/fuel-misc_7.0.0-7245.1.git4f02781_all.deb 404 Not Found [IP: 208.78.244.194 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

[slava valyavskiy (slava-val-al)]

This package mirror.fuel-infra.org/mos-repos/ubuntu/7.0/pool/main/f/fuel-library7.0/fuel-misc_7.0.0-7245.1.git4f02781_all.deb is not present in repository...looks like repository’s metadata <-> packages misconfiguration

[Kyrylo Galanov (kgalanov)]

It looks like the old package (fuel-misc_7.0.0-7245.1.git4f02781_all.deb) was purged right during the deployment:

2015-11-06 14:19:21 +0000 Puppet (err): Could not update: Execution of '/usr/bin/apt-get -q -y -o DPkg::Options::=--force-confold install fuel-misc'

New package was added recently:
fuel-misc_7.0.0-7247.1.gitf8732f1_all.deb 06-Nov-2015 11:11

--
Kyrylo

[Kyrylo Galanov (kgalanov)]

Anastasia,

Please redeploy the environment from scratch.

--
Kyrylo

[Anastasia Palkina (apalkina)]

Issue did not reproduce with new updates fuel-provisioning-scripts-7.0.0-7682.1.git25946c3.noarch
So close this bug as Invalid