Deploy is successful although nodes are in ERROR state

[Tatyana Kuterina]

Detailed bug description:
   2 nodes are in error state with correct error: Emulate deployment failure after netconfig! at /etc/puppet/modules/osnailyfacter/modular/netconfig/connectivity_tests.pp:2 on node node-3.test.domain.local,
but deploy is successfully finished

Found on CI:
    https://product-ci.infra.mirantis.net/job/9.x.system_test.ubuntu.bonding_ha/28/console

Test: Verify that network configuration with bonds is consistent after deployment failure
Test Group: bonding_conf_consistency

Steps to reproduce:
        1. Create an environment
        2. Add 3 nodes with controller role
        3. Add 1 node with compute role
        4. Setup bonding for all interfaces (including admin interface
           bonding)
        5. Run network verification
        6. Update 'connectivity_tests' puppet manifest to cause the deployment process fail right after 'netconfig' task is finished
        7. Start deployment and wait until it fails

Expected results:
    Deploy is failed. Nodes are in error state
Actual result:
    Deploy is successfully finished. Error appears:
    AssertionError: Task 'deploy' has incorrect status. ready != error

Description of the environment:
    9.1 snapshot #136

https://drive.google.com/a/mirantis.com/file/d/0Bz15vbpS5ZPNeFNHdWIyZ09mVFE/view?usp=sharing

[Kyrylo Galanov (kgalanov)]

The deployment actually failed:
2016-08-14 22:18:14 DEBUG [10070] Node 3 has failed to deploy. There is no more retries for puppet run.
2016-08-14 22:18:14 DEBUG [10070] {"nodes"=>[{"status"=>"error", "error_type"=>"deploy", "uid"=>"3", "role"=>"connectivity_tests"}]}
2016-08-14 22:18:14 DEBUG [10070] Task time summary: connectivity_tests with status failed on node 3 took 00:00:08

I am not sure why deployment was reported as successful. Please provide environment for investigation.