Status of custom_graph execution is always 'ready'

[Nikita Koshikov]

MOS9

When some tasks in custom graph fail - the fuel-task is always 'ready'.

Steps to reproduce:

create custom graph with task that will fail deployment:
random.yaml
- id: fail
  type: shell
  version: 2.1.0
  role: ['/.*/']
  requires: []
  required_for: []
  parameters:
    cmd: "e=$(echo $RANDOM%2.0 | bc); exit $e"
    timeout: 60

fuel2 graph upload --env 1 --type random --file random.yaml
fuel2 graph execute -e 1 -t random
[root@nailgun ~]# fuel2 task history show 8
+-----------+---------+--------+----------------------------+----------------------------+
| task_name | node_id | status | time_start | time_end |
+-----------+---------+--------+----------------------------+----------------------------+
| fail | 7 | error | 2016-09-06T23:56:32.372567 | 2016-09-06T23:57:29.855798 |
| fail | 3 | error | 2016-09-06T23:56:31.714198 | 2016-09-06T23:57:25.138824 |
| fail | 6 | error | 2016-09-06T23:56:32.589173 | 2016-09-06T23:57:15.724997 |
| fail | 2 | error | 2016-09-06T23:56:31.887055 | 2016-09-06T23:57:14.277645 |
| fail | 8 | error | 2016-09-06T23:56:32.771233 | 2016-09-06T23:57:08.513390 |
| fail | 4 | error | 2016-09-06T23:56:32.216783 | 2016-09-06T23:57:06.992103 |
| fail | 5 | error | 2016-09-06T23:56:32.042438 | 2016-09-06T23:57:04.296577 |
| fail | 1 | error | 2016-09-06T23:56:31.079506 | 2016-09-06T23:57:03.206114 |
+-----------+---------+--------+----------------------------+----------------------------+

Actual result:

fuel task:
8 | ready | deployment | 1 | 100 | 2b09e438-ed86-4d0e-830a-dd6cfaed07d7

Expected result:

task should have non-ready status, like deployment's one.

[root@nailgun ~]# fuel env
id | status | name | release_id
---+--------------------+------+-----------
1 | partially_deployed | t1 | 2

[Vladimir Sharshov (vsharshov)]

Can you attach logs for future analysis, at least astute.log?

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/369527

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/369527
Committed: https://git.openstack.org/cgit/openstack/fuel-astute/commit/?id=67896b9a591e4bcfe17b3a6e6c425b594bf3cf61
Submitter: Jenkins
Branch: master

commit 67896b9a591e4bcfe17b3a6e6c425b594bf3cf61
Author: Vladimir Sharshov (warpc) <email address hidden>
Date: Tue Sep 13 18:16:34 2016 +0300

    Zero tolerance for errors on nodes as default behavior

    Without this change we do not mark deployment as error
    if task on node failed.

    Also use early initialize of logger for support library
    Deployment

    Change-Id: Ibcac4569756b34c3c1ac33f68ae203246d94d2a4
    Closes-Bug: #1620858

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/370027

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/370027
Committed: https://git.openstack.org/cgit/openstack/fuel-astute/commit/?id=8505096393cb8946f83f191246b4ca31e3f804d3
Submitter: Jenkins
Branch: stable/mitaka

commit 8505096393cb8946f83f191246b4ca31e3f804d3
Author: Vladimir Sharshov (warpc) <email address hidden>
Date: Tue Sep 13 18:16:34 2016 +0300

    Zero tolerance for errors on nodes as default behavior

    Without this change we do not mark deployment as error
    if task on node failed.

    Also use early initialize of logger for support library
    Deployment

    Change-Id: Ibcac4569756b34c3c1ac33f68ae203246d94d2a4
    Closes-Bug: #1620858

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: master
Review: https://review.openstack.org/373228

[OpenStack Infra (hudson-openstack)]

Fix proposed to branch: stable/mitaka
Review: https://review.openstack.org/373229

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/373229
Committed: https://git.openstack.org/cgit/openstack/fuel-astute/commit/?id=a68113ad043e05f12492f5a487be67150f41afd0
Submitter: Jenkins
Branch: stable/mitaka

commit a68113ad043e05f12492f5a487be67150f41afd0
Author: Vladimir Sharshov (warpc) <email address hidden>
Date: Tue Sep 20 15:10:49 2016 +0300

    Fix non-working zero tolerance error group

    Also:

    - catch divide by 0 in case of progress bar (node and cluster)
    - catch sutuation with report for node without current tasks

    Change-Id: If4a975abf6da4ba1848be50a23f6532f649d2982
    Closes-Bug: #1620858

[TatyanaGladysheva (tgladysheva)]

Verified on 9.1 snapshot #293.

[OpenStack Infra (hudson-openstack)]

Reviewed: https://review.openstack.org/373228
Committed: https://git.openstack.org/cgit/openstack/fuel-astute/commit/?id=9dee3b3da74126287b1423170e96cf39a78bf51c
Submitter: Jenkins
Branch: master

commit 9dee3b3da74126287b1423170e96cf39a78bf51c
Author: Vladimir Sharshov (warpc) <email address hidden>
Date: Tue Sep 20 15:10:49 2016 +0300

    Fix non-working zero tolerance error group

    Also:

    - catch divide by 0 in case of progress
    - catch sutuation with report for node without current tasks

    Change-Id: If4a975abf6da4ba1848be50a23f6532f649d2982
    Closes-Bug: #1620858

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/fuel-astute 10.0.0rc1 release candidate.

[Vladimir Sharshov (vsharshov)]

Fix merged on all releases: 9.x, 10.x and master

[OpenStack Infra (hudson-openstack)]

This issue was fixed in the openstack/fuel-astute 10.0.0 release.

[Dmitry Belyaninov (dbelyaninov)]

Verified on Newton #1566 iso.

[root@nailgun ~]# fuel2 task list
+----+--------+------------+------------+---------+--------+---------+----------+
| id | status | name | graph_type | cluster | result | dry_run | progress |
+----+--------+------------+------------+---------+--------+---------+----------+
| 1 | error | deploy | None | 1 | {} | False | 100 |
| 2 | error | deployment | random | 1 | {} | False | 100 |
+----+--------+------------+------------+---------+--------+---------+----------+