keystone service is not always ready for floating ip creation

Bug #1351253 reported by Vladimir Kuklin
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
Critical
Sergii Golovatiuk
5.0.x
New
Undecided
Unassigned

Bug Description

{"build_id": "2014-08-01_10-37-32", "ostf_sha": "15f3be5fbafb7a8c7075b5077a5074a50e679c25", "build_number": "382", "auth_required": true, "api": "1.0", "nailgun_sha": "51f32395eebe2514e78eb7e0a85e694826be40d6", "production": "docker", "fuelmain_sha": "7990f5bfa7fea5b74ebf0402b1918109b9bc505b", "astute_sha": "f655ee86ebf0359b014f00cff63d0aaf15c65308", "feature_groups": ["mirantis"], "release": "5.1", "fuellib_sha": "5571b86a667e28d4c9770fcce4d32163dee5a710"}

Fri Aug 01 08:48:24 +0000 2014 /Stage[main]/Osnailyfacter::Cluster_ha/Nova_floating_range[10.108.1.128-10.108.1.254] (err): Could not evaluate: Oops - not sure what happened: 751: unexpected token at '<html><body><h1>504 Gateway Time-out</h1>
The server didn't respond in time.
</body></html>
'
/usr/lib/ruby/vendor_ruby/openstack/connection.rb:533:in `deal_with_faulty_error'
/usr/lib/ruby/vendor_ruby/openstack/connection.rb:510:in `raise_exception'
/usr/lib/ruby/vendor_ruby/openstack/connection.rb:207:in `req'
/usr/lib/ruby/vendor_ruby/openstack/compute/connection.rb:242:in `api_extensions'
/usr/lib/ruby/vendor_ruby/openstack/compute/connection.rb:502:in `check_extension'
/usr/lib/ruby/vendor_ruby/openstack/compute/connection.rb:475:in `get_floating_ips_bulk'
/etc/puppet/modules/nova/lib/puppet/provider/nova_floating_range/nova_manage.rb:49:in `operate_range'
/etc/puppet/modules/nova/lib/puppet/provider/nova_floating_range/nova_manage.rb:15:in `exists?'
/usr/lib/ruby/vendor_ruby/puppet/property/ensure.rb:81:in `retrieve'
/usr/lib/ruby/vendor_ruby/puppet/type.rb:1035:in `retrieve'
/usr/lib/ruby/vendor_ruby/puppet/type.rb:1063:in `retrieve_resource'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:223:in `from_resource'
/usr/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:17:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:174:in `apply'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:187:in `eval_resource'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `call'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:327:in `thinmark'
/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:326:in `thinmark'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:117:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:118:in `traverse'
/usr/lib/ruby/vendor_ruby/puppet/transaction.rb:108:in `evaluate'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:164:in `apply'
/usr/lib/ruby/vendor_ruby/puppet/util/log.rb:149:in `with_destination'
/usr/lib/ruby/vendor_ruby/puppet/transaction/report.rb:108:in `as_logging_destination'
/usr/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:163:in `apply'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:125:in `apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:161:in `benchmark'
/usr/lib/ruby/1.8/benchmark.rb:308:in `realtime'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:160:in `benchmark'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:124:in `apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/configurer.rb:192:in `run'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:268:in `apply_catalog'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:218:in `main'
/usr/lib/ruby/vendor_ruby/puppet/application/apply.rb:146:in `run_command'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `run'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:470:in `plugin_hook'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util.rb:478:in `exit_on_fail'
/usr/lib/ruby/vendor_ruby/puppet/application.rb:364:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:137:in `run'
/usr/lib/ruby/vendor_ruby/puppet/util/command_line.rb:91:in `execute'
/usr/bin/puppet:4
Fri Aug 01 08:48:24 +0000 2014 /Stage[main]/Osnailyfacter::Cluster_ha/Nova_floating_range[10.108.1.128-10.108.1.254] (info): Evaluated in 67.85 seconds

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (master)

Fix proposed to branch: master
Review: https://review.openstack.org/111229

Changed in fuel:
status: Confirmed → In Progress
Changed in fuel:
assignee: Vladimir Kuklin (vkuklin) → Sergii Golovatiuk (sgolovatiuk)
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (master)

Reviewed: https://review.openstack.org/111229
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=38adc15ae17549f37581719712f7b28e6e96cb3b
Submitter: Jenkins
Branch: master

commit 38adc15ae17549f37581719712f7b28e6e96cb3b
Author: Vladimir Kuklin <email address hidden>
Date: Fri Aug 1 14:07:51 2014 +0400

    Wait for keystone to become ready for floatingip creation

    Wait for keystone service to become ready otherwise
    haproxy reload can lead to haproxy returning
    empty response and thus nova-floating-range provider
    can not authorize with nova.

    Change-Id: I8f9623411990679560b1694569dbb185883c4733
    Closes-bug: #1351253

Changed in fuel:
status: In Progress → Fix Committed
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-library (stable/5.0)

Fix proposed to branch: stable/5.0
Review: https://review.openstack.org/114507

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-library (stable/5.0)

Reviewed: https://review.openstack.org/114507
Committed: https://git.openstack.org/cgit/stackforge/fuel-library/commit/?id=44de6e259a1191eb015ead8392f3f08e7559fea2
Submitter: Jenkins
Branch: stable/5.0

commit 44de6e259a1191eb015ead8392f3f08e7559fea2
Author: Vladimir Kuklin <email address hidden>
Date: Fri Aug 1 14:07:51 2014 +0400

    Wait for keystone to become ready for floatingip creation

    Wait for keystone service to become ready otherwise
    haproxy reload can lead to haproxy returning
    empty response and thus nova-floating-range provider
    can not authorize with nova.

    Change-Id: I8f9623411990679560b1694569dbb185883c4733
    Closes-bug: 1351253
    Closes-bug: 1348171

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.