Failed compute blocks whole deployment

[Aleksandr Shaposhnikov]

Provisioning (image-based) stuck on unmounting os-root on one of the compute nodes (compute_4).
Snapshot is in attachments.

Some info from node logs:

2015-03-31 20:00:33 INFO
fuel_agent.utils.utils [-] Trying to execute command: umount /dev/mapper/os-root
2015-03-31 20:00:33 INFO
fuel_agent.utils.fs_utils [-] Trying to umount /dev/mapper/os-root
2015-03-31 20:00:33 INFO
fuel_agent.utils.utils [-] Trying to execute command: umount /dev/md0
2015-03-31 20:00:33 INFO
fuel_agent.utils.fs_utils [-] Trying to umount /dev/md0
2015-03-31 20:00:33 INFO
fuel_agent.utils.utils [-] Trying to execute command: umount /dev/mapper/vm-nova
2015-03-31 20:00:33 INFO
fuel_agent.utils.fs_utils [-] Trying to umount /dev/mapper/vm-nova
2015-03-31 20:00:33 INFO
fuel_agent.utils.utils [-] Trying to execute command: umount /tmp/target/sys
2015-03-31 20:00:33 INFO
fuel_agent.utils.fs_utils [-] Trying to umount /tmp/target/sys
2015-03-31 20:00:33 INFO
fuel_agent.utils.utils [-] Trying to execute command: umount /tmp/target/dev
2015-03-31 20:00:33 INFO
fuel_agent.utils.fs_utils [-] Trying to umount /tmp/target/dev
2015-03-31 20:00:33 INFO
fuel_agent.utils.utils [-] Trying to execute command: umount /tmp/target/proc
2015-03-31 20:00:33 INFO
fuel_agent.utils.fs_utils [-] Trying to umount /tmp/target/proc
2015-03-31 20:00:33 INFO
fuel_agent.manager [-] Umounting target file systems

'fuel -v' shows nothing so I couldn't supply you with versions and stuff. I know only that it was build 251 of 6.1.

[Aleksandr Shaposhnikov (alashai8)]

I personally think that this is a hard-drive problem but here is the problem: deployment dies but this is compute node and according to blueprint with "continue on minor failure' it should continue.

[Aleksandr Shaposhnikov (alashai8)]

Tried to remove compute_4 node and got
"Method remove_nodes. A JSON text must at least contain two octets!. Inspect Astute logs for the details".
That may be a different bug. Will attach one more snapshot.

[Aleksandr Shaposhnikov (alashai8)]

Unfortunately after last error fuel UI stops to respond and right now it's shows only animated cloud :(

[Alexander Gordeev (a-gordeev)]

> deployment dies but this is compute node and according to blueprint with "continue on minor failure' it should continue.

so, classic provisioning won't fail for that case, right? If yes, it is another bug entirely