Cobbler Error - Error occured while provisioning

[Yaroslav Lobankov]

Configuration:
OS - CentOS
Mode - simple
Hypervisor - KVM
Nodes with roles - 1 controller, 1 compute, 1 cinder
Network - Nova-network (Flat) with network tagging
Storage - Cinder LVM
Services - Sahara

Response of http://10.20.0.2:8000/api/version is below.
{
    "build_id": "2014-06-30_03-01-14",
    "mirantis": "yes",
    "build_number": "77",
    "ostf_sha": "d0fe60e0eba61685008b86d101f459fc2d3bb654",
    "nailgun_sha": "dd7f32ab80c023a4afda70b521dd5391e5e464fd",
    "production": "docker",
    "api": "1.0",
    "fuelmain_sha": "d5cd1439a382b335ec9f7ac69d2ace78c9c68120",
    "astute_sha": "17a1dc816d6d56dda64c2db21b94581472cabefb",
    "release": "5.0.1",
    "fuellib_sha": "7f7a7b33a5711b6146e5a11811c68e91ca4761af"
}

Error occurs when installing OpenStack by Fuel. A bit of trace is below.

2014-07-01 08:18:32 ERR
[400] Error running provisioning: Net::ReadTimeout, trace: /usr/lib64/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'
/usr/lib64/ruby/2.1.0/net/protocol.rb:152:in `rbuf_fill'
/usr/lib64/ruby/2.1.0/net/protocol.rb:134:in `readuntil'
/usr/lib64/ruby/2.1.0/net/protocol.rb:144:in `readline'
/usr/lib64/ruby/2.1.0/net/http/response.rb:39:in `read_status_line'
/usr/lib64/ruby/2.1.0/net/http/response.rb:28:in `read_new'
/usr/lib64/ruby/2.1.0/net/http.rb:1408:in `block in transport_request'
/usr/lib64/ruby/2.1.0/net/http.rb:1405:in `catch'
/usr/lib64/ruby/2.1.0/net/http.rb:1405:in `transport_request'
/usr/lib64/ruby/2.1.0/net/http.rb:1378:in `request'
/usr/lib64/ruby/2.1.0/net/http.rb:1324:in `request_post'
/usr/lib64/ruby/2.1.0/xmlrpc/client.rb:482:in `do_rpc'
/usr/lib64/ruby/2.1.0/xmlrpc/client.rb:286:in `call2'
/usr/lib64/ruby/2.1.0/xmlrpc/client.rb:267:in `call'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/cobbler.rb:108:in `sync'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/cobbler_manager.rb:108:in `sync'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/cobbler_manager.rb:65:in `remove_nodes'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/orchestrator.rb:148:in `remove_nodes'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/orchestrator.rb:56:in `provision'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/dispatcher.rb:86:in `provision'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/server.rb:126:in `dispatch_message'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/server.rb:89:in `block in dispatch'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/task_queue.rb:64:in `call'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/task_queue.rb:64:in `block in each'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/task_queue.rb:56:in `each'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/task_queue.rb:56:in `each'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/server.rb:87:in `each_with_index'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/server.rb:87:in `dispatch'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/server.rb:72:in `block in perform_main_job'

2014-07-01 08:18:32 ERR
[400] Error occured while provisioning: #<Net::ReadTimeout: Net::ReadTimeout>

2014-06-30 19:10:58 ERR
[417] Exception during worker initialization: AMQP broker closed TCP connection before authentication succeeded: this usually means authentication failure due to misconfiguration. Settings are {:host=>"10.20.0.2", :port=>5672, :user=>"naily", :pass=>"[filtered]", :auth_mechanism=>"PLAIN", :vhost=>"/", :timeout=>nil, :logging=>true, :ssl=>false, :broker=>nil, :frame_max=>131072, :heartbeat=>0}, trace: /usr/lib64/ruby/gems/2.1.0/gems/amq-client-0.9.12/lib/amq/client/async/adapters/event_machine.rb:165:in `block in initialize'
/usr/lib64/ruby/gems/2.1.0/gems/amq-client-0.9.12/lib/amq/client/async/adapters/event_machine.rb:309:in `call'
/usr/lib64/ruby/gems/2.1.0/gems/amq-client-0.9.12/lib/amq/client/async/adapters/event_machine.rb:309:in `unbind'
/usr/lib64/ruby/gems/2.1.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:1438:in `event_callback'
/usr/lib64/ruby/gems/2.1.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run_machine'
/usr/lib64/ruby/gems/2.1.0/gems/eventmachine-1.0.3/lib/eventmachine.rb:187:in `run'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/lib/astute/server/worker.rb:41:in `run'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:343:in `worker_loop!'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:266:in `block (2 levels) in spawn_workers'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:266:in `fork'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:266:in `block in spawn_workers'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:261:in `each'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:261:in `spawn_workers'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:300:in `maintain_worker_count'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:120:in `master_loop!'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:53:in `start'
/usr/lib64/ruby/gems/2.1.0/gems/raemon-0.3.0/lib/raemon/master.rb:22:in `start'
/usr/lib64/ruby/gems/2.1.0/gems/astute-0.0.2/bin/astuted:84:in `<top (required)>'
-e:1:in `load'
-e:1:in `<main>'

Diagnostic snapshot is attached.

[Vladimir Sharshov (vsharshov)]

I believe this error can be fixed by this review: https://review.openstack.org/#/c/103513/

[Vladimir Sharshov (vsharshov)]

Sorry, this patch set for Cobbler: https://review.openstack.org/#/c/103738/

And this https://review.openstack.org/#/c/103513/ for "AMQP broker closed TCP connection before authentication succeeded".

All of them at now moment was applied to 5.0.1 by this time.

[Vladimir Sharshov (vsharshov)]

Please try to reproduce it using fresh iso (> iso number for 5.0 more than 81)