Deployment fails due to empty /var/lib/hiera/common.yaml file

[Alexei Sheplyakov]

2015-01-23 05:41:27 +0000 Puppet (debug): hiera(): Looking for data source common
2015-01-23 05:41:27 +0000 Puppet (err): Error from DataBinding 'hiera' while looking up 'openstack::controller::custom_mysql_setup_class': Data retrieved from /var/lib/hiera/common.yaml is String not Hash on node node-2.local

The work around is to not create the /var/lib/hiera/common.yaml file (https://review.openstack.org/#/c/146897).

The issue might be caused by https://github.com/tenderlove/psych/issues/149

[Alexei Sheplyakov (asheplyakov)]

$ rm -f foo.yaml && touch foo.yaml && ruby -e "require 'yaml'; val = YAML.load_file(\"foo.yaml\")"
/usr/lib/ruby/1.9.1/psych.rb:297:in `initialize': no implicit conversion from nil to integer (TypeError)
        from /usr/lib/ruby/1.9.1/psych.rb:297:in `open'
        from /usr/lib/ruby/1.9.1/psych.rb:297:in `load_file'
        from -e:1:in `<main>'

[Aleksandr Didenko (adidenko)]

Could you please provide version of Fuel you used? We don't use default hiera configuration with empty /var/lib/hiera/common.yaml since master (6.1) build #70.

[Alexei Sheplyakov (asheplyakov)]

https://review.openstack.org/gitweb?p=stackforge/fuel-library.git;a=blob;f=deployment/puppet/cobbler/templates/snippets/ubuntu_puppet_config.erb;h=8c640f52040b2ce636204e196efdc1e7b77df6a9;hb=HEAD#l4

in-target mkdir -p /etc/puppet /var/lib/hiera && \
in-target touch /var/lib/hiera/common.yaml /etc/puppet/hiera.yaml && \

https://review.openstack.org/gitweb?p=stackforge/fuel-web.git;a=blob;f=fuel_agent/cloud-init-templates/boothook_ubuntu.jinja2;h=d89c8cff99af5ebc6b5cef29313ef3047555fc83;hb=002d1416a07290d20c3a77860881e9dc750eb68b#l82

cloud-init-per instance hiera_puppet mkdir -p /etc/puppet /var/lib/hiera
cloud-init-per instance touch_puppet touch /var/lib/hiera/common.yaml /etc/puppet/hiera.yaml

[Alexei Sheplyakov (asheplyakov)]

Alexandr,

> We don't use default hiera configuration with empty /var/lib/hiera/common.yaml

I'm afraid this statement is not quite true (see the above quoted code).

[Aleksandr Didenko (adidenko)]

It looks like hiera-1.3.0 bug.

root@node-2:~# cat /etc/issue
Ubuntu 14.04.1 LTS \n \l

root@node-2:~# hiera --version
1.3.0
root@node-2:~# hiera zzz
/usr/lib/ruby/vendor_ruby/hiera/filecache.rb:56:in `read_file': Data retrieved from /var/lib/hiera/common.yaml is String not Hash (TypeError)
        from /usr/lib/ruby/vendor_ruby/hiera/backend/yaml_backend.rb:22:in `block in lookup'
        from /usr/lib/ruby/vendor_ruby/hiera/backend.rb:73:in `block in datasources'
        from /usr/lib/ruby/vendor_ruby/hiera/backend.rb:71:in `map'
        from /usr/lib/ruby/vendor_ruby/hiera/backend.rb:71:in `datasources'
        from /usr/lib/ruby/vendor_ruby/hiera/backend/yaml_backend.rb:16:in `lookup'
        from /usr/lib/ruby/vendor_ruby/hiera/backend.rb:174:in `block in lookup'
        from /usr/lib/ruby/vendor_ruby/hiera/backend.rb:171:in `each'
        from /usr/lib/ruby/vendor_ruby/hiera/backend.rb:171:in `lookup'
        from /usr/lib/ruby/vendor_ruby/hiera.rb:60:in `lookup'
        from /usr/bin/hiera:221:in `<main>'

On other ubuntu-14.04.01 system with the same empty hiera configs it works fine:

hiera --version
1.3.4

hiera zzz
nil

So we should use 1.3.4 hiera for Ubuntu-14.04.1 since hiera will be the main source/machanism of pulling configuration data for Fuel deployment and we can't allow it to fail on empty yaml files in the hierarchy.

[OSCI Robot (oscirobot)]

DEB package ruby-hiera has been built for project packages/trusty/ruby-hiera
Package version == 1.3.1, package release == ubuntu1

Changeset: https://review.fuel-infra.org/2392
project: packages/trusty/ruby-hiera
branch: 6.1
author: Alexei Sheplyakov
committer: Alexei Sheplyakov
subject: Added ruby-hiera 1.3.1 from Puppet Labs
status: patchset-created

Files placed on repository:
ruby-hiera_1.3.1-1puppetlabs1~mos6.1+1_all.deb

NOTE: Changeset is not merged, created temporary package repository.
DEB repository URL: http://osci-obs.vm.mirantis.net:82/trusty-fuel-6.1-stable-2392/ubuntu

[OSCI Robot (oscirobot)]

DEB package ruby-hiera has been built for project packages/trusty/ruby-hiera
Package version == 1.3.1, package release == ubuntu1

Changeset: https://review.fuel-infra.org/2392
project: packages/trusty/ruby-hiera
branch: 6.1
author: Alexei Sheplyakov
committer: Alexei Sheplyakov
subject: Added ruby-hiera 1.3.1 from Puppet Labs
status: change-merged

Files placed on repository:
ruby-hiera_1.3.1-1puppetlabs1~mos6.1+1_all.deb

Changeset merged. Package placed on primary repository
DEB repository URL: http://osci-obs.vm.mirantis.net:82/trusty-fuel-6.1-stable/ubuntu

[Bogdan Dobrelya (bogdando)]

Fuel do not use hiera in 6.0, hence invalid

[Sergii Golovatiuk (sgolovatiuk)]

It was fixed by ruby-hiera 1.3.1

[Anastasia Palkina (apalkina)]

Verified on ISO #210

"build_id": "2015-03-19_22-54-44", "ostf_sha": "b9a090c71682fbea5d9351051827d7d654d07be3", "build_number": "210", "release_versions": {"2014.2-6.1": {"VERSION": {"build_id": "2015-03-19_22-54-44", "ostf_sha": "b9a090c71682fbea5d9351051827d7d654d07be3", "build_number": "210", "api": "1.0", "nailgun_sha": "1d2bd383caecc5ec3f86bf93ccca940326f23e97", "production": "docker", "python-fuelclient_sha": "b223dcaf5fdad2f714cd245958fefe03995d6207", "astute_sha": "4a117a1ca6bdcc34fe4d086959ace1a6d18eeca9", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "f3d6353c08d8eb709c7ab100b56dc2bebef4157f", "fuellib_sha": "7764225db5bc653563309912afbb4058283c808b"}}}, "auth_required": true, "api": "1.0", "nailgun_sha": "1d2bd383caecc5ec3f86bf93ccca940326f23e97", "production": "docker", "python-fuelclient_sha": "b223dcaf5fdad2f714cd245958fefe03995d6207", "astute_sha": "4a117a1ca6bdcc34fe4d086959ace1a6d18eeca9", "feature_groups": ["mirantis"], "release": "6.1", "fuelmain_sha": "f3d6353c08d8eb709c7ab100b56dc2bebef4157f", "fuellib_sha": "7764225db5bc653563309912afbb4058283c808b"