Error occured during deployment in multihypervisors mode: "At least one of nodes have failed during provisioning"

Bug #1435954 reported by Tatyana Dubyk
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Invalid
High
Fuel Partner Integration Team

Bug Description

Error occured during deployment in multihypervisors mode:
"At least one of nodes have failed during provisioning"
Each time during deployment one of the nodes can't perform provisioning sucessfully in multihypervisors mode in 6.1 release .

steps to reproduce:
1. create env with kvm+vcenter
    Ubuntu OS
    all other settings by default
    HA mode

2. add 3 nodes with controller roles
3. click on deploy button

settings for vcenter:
----------------------------------------------------------------------
Availability zone: any
vCenter host: 172.16.0.254
vCenter username: <email address hidden>
vCenter password: Qwer!1234
Cluster: ‘Cluster1’
Service name: any
Datastore regex: .*

network settings:-
---------------------------------------------------------------------
Network type: nova-network Flat DHCP with settings:
public network: 172.16.0.0/24
private(VM) network: tag should be set to 103
management network: 172.16.1.0/24

Interfaces on slaves should be setup this way in Fuel interface:
eth0 - admin(PXE)
eth1 - public
eth2 - management
eth3 - VM(Fixed)
eth4 - storage

=================================================
api: '1.0'
astute_sha: 4a117a1ca6bdcc34fe4d086959ace1a6d18eeca9
auth_required: true
build_id: 2015-03-23_15-29-20
build_number: '218'
feature_groups:
- mirantis
fuellib_sha: a0265ae47bb2307a6967a3f1dd06fe222c561265
fuelmain_sha: a05ab877af31924585c81081f45305700961458e
nailgun_sha: 7c100f47450ea1a910e19fa09f78d586cb2bc0d3
ostf_sha: a4cf5f218c6aea98105b10c97a4aed8115c15867
production: docker
python-fuelclient_sha: 3624051242c83fdbdd1df9a0e466797c06b75043
release: '6.1'
release_versions:
  2014.2-6.1:
    VERSION:
      api: '1.0'
      astute_sha: 4a117a1ca6bdcc34fe4d086959ace1a6d18eeca9
      build_id: 2015-03-23_15-29-20
      build_number: '218'
      feature_groups:
      - mirantis
      fuellib_sha: a0265ae47bb2307a6967a3f1dd06fe222c561265
      fuelmain_sha: a05ab877af31924585c81081f45305700961458e
      nailgun_sha: 7c100f47450ea1a910e19fa09f78d586cb2bc0d3
      ostf_sha: a4cf5f218c6aea98105b10c97a4aed8115c15867
      production: docker
      python-fuelclient_sha: 3624051242c83fdbdd1df9a0e466797c06b75043
      release: '6.1'
----------------------------------------------------------------------
[708] Error running provisioning: At least one of nodes have failed during provisioning, trace:
["/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:116:in `image_provision'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:245:in `provision_piece'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:148:in `block (3 levels) in provision_and_watch_progress'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:301:in `call'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:301:in `sleep_not_greater_than'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:142:in `block (2 levels) in provision_and_watch_progress'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:141:in `loop'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:141:in `block in provision_and_watch_progress'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:140:in `catch'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:140:in `provision_and_watch_progress'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/provision.rb:54:in `provision'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/orchestrator.rb:77:in `provision'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/dispatcher.rb:50:in `provision'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/dispatcher.rb:37:in `image_provision'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:142:in `dispatch_message'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:103:in `block in dispatch'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:64:in `call'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:64:in `block in each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:56:in `each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/task_queue.rb:56:in `each'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `each_with_index'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:101:in `dispatch'",
 "/usr/lib64/ruby/gems/2.1.0/gems/astute-6.0.0/lib/astute/server/server.rb:85:in `block in perform_main_job'"]

before this:
[7f06e87dc740] (helpers) create_action_log failed: 'ThreadedDict' object has no attribute 'env'

[7f06e87dc740] (node) Node Untitled (ce:17) has provisioning or error status - status not updated by agent - it happens, when deploy starts.

Tatyana Dubyk (tdubyk)
description: updated
Changed in fuel:
milestone: none → 6.1
importance: Undecided → High
assignee: nobody → Fuel Partner Integration Team (fuel-partner)
Tatyana Dubyk (tdubyk)
description: updated
description: updated
Revision history for this message
Igor Zinovik (izinovik) wrote :

Tatyana, please provide diagnostic snapshot.

Revision history for this message
Andriy Popovych (popovych-andrey) wrote :

Exception like "[708] Error running provisioning: At least one of nodes have failed during provisioning, trace:" in astute logs not so informative actually its common to all situations if something goes wrong during puppet doing some job. We need more info from remote node logs (like puppet)

Igor Zinovik (izinovik)
tags: added: partner vcenter
Revision history for this message
Tatyana Dubyk (tdubyk) wrote :

guys, sorry but sometimes I can't provide diagnostic snapshot because due to some issue it can't be created on Support Tab.

Revision history for this message
Igor Zinovik (izinovik) wrote :

It is really hard to understand what happend without diagnostic snapshot,
for now marking as Incomplete.

Changed in fuel:
status: New → Incomplete
Igor Zinovik (izinovik)
Changed in fuel:
status: Incomplete → Invalid
status: Invalid → Incomplete
Revision history for this message
Olesia Tsvigun (otsvigun) wrote :

This issue was not reproduced at ISO#295 ubuntu
api: '1.0'
astute_sha: 5041b2fb508e6860c3cb96474ca31ec97e549e8b
auth_required: true
build_id: 2015-04-08_22-54-31
build_number: '295'
feature_groups:
- mirantis
fuellib_sha: dee81f2be4d9063808a6755271ee818314997006
fuelmain_sha: 2ca546b86e651d5638dbb1be9bae44b86c84a893
nailgun_sha: e5f101635455e94415969907deca510d2ded6b73
openstack_version: 2014.2-6.1
ostf_sha: c3b06dba5c96d225882e9f1a465f74eaa6374fbf
production: docker
python-fuelclient_sha: 5c94b59bafc8dc1cbecb088020f4ef14ce62044a
release: '6.1'
release_versions:
  2014.2-6.1:
    VERSION:
      api: '1.0'
      astute_sha: 5041b2fb508e6860c3cb96474ca31ec97e549e8b
      build_id: 2015-04-08_22-54-31
      build_number: '295'
      feature_groups:
      - mirantis
      fuellib_sha: dee81f2be4d9063808a6755271ee818314997006
      fuelmain_sha: 2ca546b86e651d5638dbb1be9bae44b86c84a893
      nailgun_sha: e5f101635455e94415969907deca510d2ded6b73
      openstack_version: 2014.2-6.1
      ostf_sha: c3b06dba5c96d225882e9f1a465f74eaa6374fbf
      production: docker
      python-fuelclient_sha: 5c94b59bafc8dc1cbecb088020f4ef14ce62044a
      release: '6.1'

Revision history for this message
Alexander Arzhanov (aarzhanov) wrote :

I'm mark this bug Invalid, because this bug not reproduced

Changed in fuel:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.