Deployment failed with error /etc/puppet/modules/osnailyfacter/modular/tools/tool.pp

[Nastya Urlapova]

{
build_id: "2015-03-06_17-12-55",
ostf_sha: "8df5f2fcdae3bc9ea7d700ffd64db820baf51914",
build_number: "170",
release_versions: {
2014.2-6.1: {
VERSION: {
build_id: "2015-03-06_17-12-55",
ostf_sha: "8df5f2fcdae3bc9ea7d700ffd64db820baf51914",
build_number: "170",
api: "1.0",
nailgun_sha: "bf1ff59c9dafa23ab6a63b6c33d8c4f1a9014440",
production: "docker",
python-fuelclient_sha: "4eb787f1ad969bd23c93d192865543dbd45a8626",
astute_sha: "2d61ee42ec6dae3181d292c7769d32e40d463893",
feature_groups: [
"mirantis"
],
release: "6.1",
fuelmain_sha: "0e45b31db1677651d6ddb1c852d62ebfd8875dcd",
fuellib_sha: "7668e98767739a250cf34edfee4f195abd5c3f66"
}
}
},
auth_required: true,
api: "1.0",
nailgun_sha: "bf1ff59c9dafa23ab6a63b6c33d8c4f1a9014440",
production: "docker",
python-fuelclient_sha: "4eb787f1ad969bd23c93d192865543dbd45a8626",
astute_sha: "2d61ee42ec6dae3181d292c7769d32e40d463893",
feature_groups: [
"mirantis"
],
release: "6.1",
fuelmain_sha: "0e45b31db1677651d6ddb1c852d62ebfd8875dcd",
fuellib_sha: "7668e98767739a250cf34edfee4f195abd5c3f66"
}

Scenario:
            1. Create cluster
            2. Add 1 node with controller role
            3. Add 2 nodes with compute role
            4. Deploy the cluster
            5. Run network verification
            6. Run OSTF

Deployment failed wit error:

2015-03-06 18:32:42 ERR
[420] Error running RPC method granular_deploy: undefined method `[]' for nil:NilClass, trace:
["/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/reporter.rb:159:in `calculate_multiroles_node_progress'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/reporter.rb:96:in `node_validate'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/reporter.rb:78:in `block in get_nodes_to_report'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/reporter.rb:78:in `map'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/reporter.rb:78:in `get_nodes_to_report'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/reporter.rb:51:in `report_new_data'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/reporter.rb:44:in `report'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/context.rb:35:in `report_and_update_status'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/deployment_engine/granular_deployment.rb:231:in `rescue in post_deployment_actions'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/deployment_engine/granular_deployment.rb:218:in `post_deployment_actions'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/deployment_engine.rb:74:in `deploy'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/orchestrator.rb:118:in `deploy_cluster'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/orchestrator.rb:56:in `granular_deploy'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/dispatcher.rb:99:in `granular_deploy'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:142:in `dispatch_message'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:103:in `block in dispatch'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:64:in `call'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:64:in `block in each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:56:in `each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:56:in `each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `each_with_index'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `dispatch'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:85:in `block in perform_main_job'"]
2015-03-06 18:31:33 ERR
[420] No more tasks will be executed on the node 1
2015-03-06 18:31:33 ERR
[420] Task '{"priority"=>400, "type"=>"puppet", "uids"=>["1"], "parameters"=>{"puppet_modules"=>"/etc/puppet/modules", "puppet_manifest"=>"/etc/puppet/modules/osnailyfacter/modular/tools/tools.pp", "timeout"=>3600, "cwd"=>"/"}}' on node 1 valid, but failed

[Ryan Moe (rmoe)]

puppet_modules_source was missing from the hiera data.

From puppet.log:
Could not find data item puppet_modules_source in any Hiera data file and no default supplied at /etc/puppet/modules/osnailyfacter/modular/tools/tools.pp:19 on node node-1.local

That value was not in the data sent from Nailgun to Astute. It looks like that was changed here: https://review.openstack.org/#/c/156147/

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/162395

[OpenStack Infra (hudson-openstack)]

Change abandoned by Vladimir Kuklin (<email address hidden>) on branch: master
Review: https://review.openstack.org/162395

[Ihor Kalnytskyi (ikalnytskyi)]

The issue was introduced because of partial reverting patches of blueprint consume-external-ubuntu.

We should either revert them all or commit them all.

Currently, we have commited all needed fixes and we're waiting for tests. If they pass, we'll update iso for fuel ci (library).

[Ihor Kalnytskyi (ikalnytskyi)]

We have re-applied all needed patches, built iso and use it for fuel ci.

Master is also working.

[Anastasia Palkina (apalkina)]

Verified on ISO #200 for Ubuntu

"build_id": "2015-03-15_22-54-44", "ostf_sha": "e86c961ceacfa5a8398b6cbda7b70a5f06afb476", "build_number": "200", "release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-03-15_22-54-44", "ostf_sha": "e86c961ceacfa5a8398b6cbda7b70a5f06afb476", "build_number": "200", "api": "1.0", "nailgun_sha": "713e6684f9f54e29acfe6b8ebf641b9de2292628", "production": "docker", "python-fuelclient_sha": "cc64fff91fb0d78e5a02e7b93ceff224296f84fb", "astute_sha": "93e427ac49109fa3fd8b0e1d0bb3d14092be2e8c", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "c97fd8a789645bda48d06da224f994f8b52d82f5", "fuellib_sha": "553cb0cffa40a5f57313f962b6ec6a9bd89306ba"}}}, "auth_required": true, "api": "1.0", "nailgun_sha": "713e6684f9f54e29acfe6b8ebf641b9de2292628", "production": "docker", "python-fuelclient_sha": "cc64fff91fb0d78e5a02e7b93ceff224296f84fb", "astute_sha": "93e427ac49109fa3fd8b0e1d0bb3d14092be2e8c", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "c97fd8a789645bda48d06da224f994f8b52d82f5", "fuellib_sha": "553cb0cffa40a5f57313f962b6ec6a9bd89306ba"