Puppet error: Could not set 'present' on ensure: undefined method `[]' for nil:NilClass at ...ceilometer/radosgw_user.pp

[Maksym Strukov]

Scenario (https://mirantis.testrail.com/index.php?/tests/view/1932914)
1. Create cluster
2. Add 3 nodes with controller and ceph role
3. Add 3 nodes with compute and ceph roles
4. Add 3 nodes with mongo roles
5. Verify network
6. Deploy the cluster

Actual:

'''
2015-10-22T00:09:12.099346+00:00 info: (/Stage[main]/Main/Ceilometer_radosgw_user[ceilometer]) Starting to evaluate the resource
2015-10-22T00:09:12.101191+00:00 debug: Executing '/usr/bin/radosgw-admin user info --uid=ceilometer'
2015-10-22T00:09:12.772411+00:00 err: Could not set 'present' on ensure: undefined method `[]' for nil:NilClass at 14:/etc/puppet/modules/osnailyfacter/modular/ceilometer/radosgw_user.pp
2015-10-22T00:09:12.773064+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/provider/ceilometer_radosgw_user/user.rb:55:in `block in get_access_keys_from_config'
2015-10-22T00:09:12.773585+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/provider/ceilometer_radosgw_user/user.rb:54:in `each'
2015-10-22T00:09:12.773585+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/provider/ceilometer_radosgw_user/user.rb:54:in `get_access_keys_from_config'
2015-10-22T00:09:12.773585+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/provider/ceilometer_radosgw_user/user.rb:33:in `set_access_keys'
2015-10-22T00:09:12.773585+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/provider/ceilometer_radosgw_user/user.rb:15:in `create'
2015-10-22T00:09:12.773585+00:00 err: /etc/puppet/modules/osnailyfacter/lib/puppet/type/ceilometer_radosgw_user.rb:8:in `block (3 levels) in <top (required)>'
2015-10-22T00:09:12.773585+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/property.rb:197:in `call_valuemethod'
2015-10-22T00:09:12.773585+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/property.rb:498:in `set'
2015-10-22T00:09:12.773585+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/property.rb:581:in `sync'
2015-10-22T00:09:12.774067+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:191:in `sync'
2015-10-22T00:09:12.774988+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:128:in `sync_if_needed'
2015-10-22T00:09:12.774988+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:81:in `perform_changes'
2015-10-22T00:09:12.774988+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:20:in `evaluate'
2015-10-22T00:09:12.774988+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:174:in `apply'
2015-10-22T00:09:12.774988+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:187:in `eval_resource'
2015-10-22T00:09:12.774988+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `call'
2015-10-22T00:09:12.774988+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `block (2 levels) in evaluate'
2015-10-22T00:09:12.774988+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/util.rb:327:in `block in thinmark'
2015-10-22T00:09:12.776315+00:00 err: /usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
2015-10-22T00:09:12.776812+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/util.rb:326:in `thinmark'
2015-10-22T00:09:12.777828+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `block in evaluate'
2015-10-22T00:09:12.777828+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
2015-10-22T00:09:12.777828+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction.rb:108:in `evaluate'
2015-10-22T00:09:12.777828+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:164:in `block in apply'
2015-10-22T00:09:12.777828+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/util/log.rb:149:in `with_destination'
2015-10-22T00:09:12.777828+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:108:in `as_logging_destination'
2015-10-22T00:09:12.777828+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:163:in `apply'
2015-10-22T00:09:12.777828+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:125:in `block in apply_catalog'
2015-10-22T00:09:12.779211+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
2015-10-22T00:09:12.779791+00:00 err: /usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:124:in `apply_catalog'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/configurer.rb:192:in `run'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:268:in `apply_catalog'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:218:in `main'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:146:in `run_command'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block (2 levels) in run'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/application.rb:470:in `plugin_hook'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block in run'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/util.rb:478:in `exit_on_fail'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `run'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:137:in `run'
2015-10-22T00:09:12.780406+00:00 err: /usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:91:in `execute'
2015-10-22T00:09:12.780968+00:00 err: /usr/bin/puppet:4:in `<main>'
'''

Env:
Fuel 8.0-51 (liberty)

[Maksym Strukov (unbelll)]

Snapshot: https://drive.google.com/a/mirantis.com/file/d/0B1yfbgZlRKfxODFxTVV2djdwWnM/view?usp=sharing

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/239867

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/239867
Committed: https://git.openstack.org/cgit/openstack/fuel-library/commit/?id=553f909ad16b8d88125e9e79c982f17e20e161a9
Submitter: Jenkins
Branch: master

commit 553f909ad16b8d88125e9e79c982f17e20e161a9
Author: Michael Polenchuk <email address hidden>
Date: Wed Oct 28 11:18:16 2015 +0300

    Handle absent section with rgw credentials

    Change-Id: I91a9a55a4838c3b2941f0ba4462240067ea3df9f
    Closes-Bug: #1509549

[Dmitriy Kruglov (dkruglov)]

Verified on MOS 8.0, build 277. The issue is not reproduced.

ISO data:
VERSION:
  feature_groups:
    - mirantis
  production: "docker"
  release: "8.0"
  openstack_version: "2015.1.0-8.0"
  api: "1.0"
  build_number: "277"
  build_id: "277"
  fuel-nailgun_sha: "63478723bbb1c248f203939ac7e0a3947e25ae77"
  python-fuelclient_sha: "f96659066e522e28b389de3cc685f6f2aacca3da"
  fuel-agent_sha: "2f18b7596bc7da79d2f28c34f42620b2090d8a35"
  fuel-nailgun-agent_sha: "a33a58d378c117c0f509b0e7badc6f0910364154"
  astute_sha: "81d9319d4432ae675a412a8c81f1199c66796f5b"
  fuel-library_sha: "c8b623fcafe9410548d7c4f020cce27670d20eee"
  fuel-ostf_sha: "48883c8f1b2c53989fad74137f0bde392098b0f2"
  fuel-mirror_sha: "c66309976db831a4081e5e0064028d7c67014b2f"
  fuelmenu_sha: "027e8336d4204a12618628da996389868d16a2b4"
  shotgun_sha: "a0bd06508067935f2ae9be2523ed0d1717b995ce"
  network-checker_sha: "a3534f8885246afb15609c54f91d3b23d599a5b1"
  fuel-upgrade_sha: "1e894e26d4e1423a9b0d66abd6a79505f4175ff6"
  fuelmain_sha: "a107da70af95866681589b784176632db2c63309"

[Stefan Anjert (anjert)]

Can i apply this fix to MOS 7.0 as well?

[Michael Polenchuk (mpolenchuk)]

Sure, just take updated provider/type files from above commit (i.e. tests aren't required).

[Stefan Anjert (anjert)]

Sorry, can't get it to work. The deployment runs to 100% but fails during finalization.

MOS 7.0
Ubuntu 14.04
Juniper Contrail 3.0.1
Contrail Plugin 3.0.0
3 nodes with controller role
7 nodes with compute role
3 nodes with mongo roles
1 node with contrail
4 nodes with ceph osd
Verified network