Controller has gone away and back online with error in nailgun-agent.log when puppet configures network

[Anastasia Palkina]

"build_id": "2014-04-14_01-00-26",
"mirantis": "yes",
"build_number": "101",
"nailgun_sha": "61410bcf3201cd737e68ece8ab15313acc746476",
"production": "dev",
"ostf_sha": "118c955085ea7829f3a34decd38d63554b74451c",
"fuelmain_sha": "ddc94c52c267f0276cbd8485d6e704aea05b23a5",
"astute_sha": "401bc474b1d8cebb8ba70b3b6154107e08fd725d",
"release": "5.0",
"fuellib_sha": "101f3645ead182bc47024ff7568b04554de06bba"

1. Create new environment (CentOS, HA mode)
2. Choose VLAN segmentation
3. Add 3 controllers, compute and cinder node
4. Start deployment. It was successful
5. Controller has gone away and back online during deployment.

There is error in /var/log/nailgun-agent.log:

D, [2014-04-14T11:07:27.721882 #14152] DEBUG -- : Detailed meta disks: [{:model=>"VBOX HARDDISK", :disk=>"disk/by-path/pci-0000:00:0d.0-scsi-2:0:0:0", :removable=>"0", :size=>68718428160, :extra=>["disk/by-id/scsi-SATA_VBOX_HARDDISK_VB8c1016fc-7275e104", "disk/by-id/ata-VBOX_HARDDISK_VB8c1016fc-7275e104"], :name=>"sdc"}, {:model=>"VBOX HARDDISK", :disk=>"disk/by-path/pci-0000:00:0d.0-scsi-1:0:0:0", :removable=>"0", :size=>68718428160, :extra=>["disk/by-id/scsi-SATA_VBOX_HARDDISK_VB859c5506-e44eca2b", "disk/by-id/ata-VBOX_HARDDISK_VB859c5506-e44eca2b"], :name=>"sdb"}, {:model=>"VBOX HARDDISK", :disk=>"disk/by-path/pci-0000:00:0d.0-scsi-0:0:0:0", :removable=>"0", :size=>68718428160, :extra=>["disk/by-id/scsi-SATA_VBOX_HARDDISK_VB1e6545ad-57d94a52", "disk/by-id/ata-VBOX_HARDDISK_VB1e6545ad-57d94a52"], :name=>"sda"}]
E, [2014-04-14T11:07:37.735590 #14152] ERROR -- : execution expired
/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient/timeout.rb:43:in `new'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient/session.rb:780:in `create_socket'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient/session.rb:737:in `connect'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient/timeout.rb:131:in `timeout'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient/session.rb:736:in `connect'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient/session.rb:594:in `query'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient/session.rb:161:in `query'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient.rb:1060:in `do_get_block'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient.rb:869:in `do_request'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient.rb:956:in `protect_keep_alive_disconnected'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient.rb:868:in `do_request'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient.rb:756:in `request'/usr/lib/ruby/gems/1.8/gems/httpclient-2.2.5/lib/httpclient.rb:671:in `put'/opt/nailgun/bin/agent:152:in `put'/opt/nailgun/bin/agent:554

[Dima Shulyak (dshulyak)]

Correct me if i am wrong, but why it is not acceptable to lose some node during network configuration, given that it will come back online right after?
And if its not, what should we do then?

[Andrew Woodward (xarses)]

I think we should handle the error more gracefully, its also a problem if the node was marked offline during this period. Maybe during that stage we should allow the node to be offline slightly longer.