nova_admin_tenant_id_setter does not issue retries during deployment

[Vladimir Kuklin]

Extracted from diag snapshot of bug: https://bugs.launchpad.net/fuel/+bug/1450958

Actually happens due to https://bugs.launchpad.net/fuel/+bug/1451515

2015-05-01 21:53:00 +0000 Puppet (err): Received error response from Keystone server at http://192.168.0.2:35357/v2.0/tenants: Unauthorized
/etc/puppet/modules/neutron/lib/puppet/provider/nova_admin_tenant_id_setter/ini_setting.rb:39:in `handle_request'
/etc/puppet/modules/neutron/lib/puppet/provider/nova_admin_tenant_id_setter/ini_setting.rb:123:in `keystone_v2_tenants'
/etc/puppet/modules/neutron/lib/puppet/provider/nova_admin_tenant_id_setter/ini_setting.rb:140:in `find_tenant_by_name'
/etc/puppet/modules/neutron/lib/puppet/provider/nova_admin_tenant_id_setter/ini_setting.rb:164:in `get_tenant_id'
/etc/puppet/modules/neutron/lib/puppet/provider/nova_admin_tenant_id_setter/ini_setting.rb:177:in `config'
/etc/puppet/modules/neutron/lib/puppet/provider/nova_admin_tenant_id_setter/ini_setting.rb:151:in `create'
/usr/lib/ruby/vendor_ruby/puppet/property/ensure.rb:16:in `block in defaultvalues'
/usr/lib/ruby/vendor_ruby/puppet/property.rb:197:in `call_valuemethod'
/usr/lib/ruby/vendor_ruby/puppet/property.rb:498:in `set'
/usr/lib/ruby/vendor_ruby/puppet/property.rb:581:in `sync'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:191:in `sync'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:128:in `sync_if_needed'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:81:in `perform_changes'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:20:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:174:in `apply'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:187:in `eval_resource'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `call'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `block (2 levels) in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:327:in `block in thinmark'
/usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:326:in `thinmark'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `block in evaluate'
/usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:108:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:164:in `block in apply'
/usr/lib/ruby/vendor_ruby/puppet/util/log.rb:149:in `with_destination'
/usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:108:in `as_logging_destination'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:163:in `apply'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:125:in `block in apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `block in benchmark'
/usr/lib/ruby/1.9.1/benchmark.rb:295:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:124:in `apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:192:in `run'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:268:in `apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:218:in `main'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:146:in `run_command'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block (2 levels) in run'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:470:in `plugin_hook'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `block in run'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:478:in `exit_on_fail'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:137:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:91:in `execute'
/usr/bin/puppet:4:in `<main>'

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/179854

[Ivan Kolodyazhny (e0ne)]

Staging job failed: http://jenkins-product.srt.mirantis.net:8080/job/6.1.staging.ubuntu.bvt_2/269/console

[Bogdan Dobrelya (bogdando)]

I guess we need this w/a ASAP

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/179854
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=4caa9d5b1ffe9296bb300f0eb4ad64b3b82f35ed
Submitter: Jenkins
Branch: master

commit 4caa9d5b1ffe9296bb300f0eb4ad64b3b82f35ed
Author: Vladimir Kuklin <email address hidden>
Date: Mon May 4 21:00:00 2015 +0300

    Add retries to nova_admin_tenant_id_setter

    Issue retries up to ten times in case
    we face any keystone issues related
    to memcached/connectivity/etc. failures

    Copy-pasted from
    I1630f54cdfa850a6b4dcbc9b714ebc6195de1fb0

    Change-Id: I6700767bfb2465869f4e87e7c0d1c02bb2934ae2
    Related-bug: #1451515
    Closes-bug: #1451521

[Vitaly Sedelnik (vsedelnik)]

Won't Fix for 6.0-updates as there is no delivery channel for Fuel fixes in 6.0