[10.0] fuel2 graph execute -t update_plugin is finished but nodes aren't updated

[Ruslan Khozinov]

newton-10.0 #1451

I've got an example_plugin_v4 from fuel-plugins and created a simple task named update_stub to check possibility of update by graph related to the feature https://mirantis.jira.com/browse/PROD-8688

so, I've 2 versions of example plugin

[root@nailgun deployment_scripts]# fuel plugins
id | name | version | package_version | releases
---+------------------+---------+-----------------+---------------------
1 | fuel_plugin_stub | 0.1 | 4.0.0 | ubuntu (newton-10.0)
2 | fuel_plugin_stub | 0.2 | 5.0.0 | ubuntu (newton-10.0)

the first version has been successfully deployed and then I've just

added the task

- id: update_stub
  type: group
  role: [update_stub]
  tasks: [hiera, globals, setup_repositories, tools, logging, netconfig, hosts, deploy_start]
  parameters:
    strategy:
      type: one_by_one

and executed the task by the command (with fuel plugin --sync)
fuel2 graph execute --env 1 --trace -t update_stub

the task has been finished successfully, but in the history was empty and nodes have been freezed at the deploying state

[root@nailgun fuel_plugin_stub-0.2]# fuel nodes
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
---+-----------+---------------------------+---------+-------------+-------------------+-------------------+---------------+--------+---------
 4 | deploying | slave-01_controller | 1 | 10.109.13.4 | 64:81:55:70:43:39 | controller | | 1 | 1
 6 | deploying | slave-02_controller | 1 | 10.109.13.5 | 64:a2:ca:62:ab:ea | controller | | 1 | 1
 1 | deploying | slave-04_compute_ceph-osd | 1 | 10.109.13.7 | 64:6f:05:dc:92:f0 | ceph-osd, compute | | 1 | 1
 5 | deploying | slave-06_compute_ceph-osd | 1 | 10.109.13.9 | 64:2f:45:b3:6f:1f | ceph-osd, compute | | 1 | 1
 2 | deploying | slave-03_controller | 1 | 10.109.13.6 | 64:94:bd:70:99:69 | controller | | 1 | 1
 3 | deploying | slave-05_compute_ceph-osd | 1 | 10.109.13.8 | 64:2c:89:4f:33:5e | ceph-osd, compute | | 1 | 1

=========================================================================
steps to reproduce:

1. deploy fuel env from fuel-10 iso
2. install v0.1 of the plugin (plugin SHOULD NOT contain a special update_plugin_task) http://paste.ubuntu.com/24240378/
3. deploy changes
4. install v0.2 of the plugin (that version SHOULD contain a special update_plugin_task) http://paste.ubuntu.com/24240379/
Note: in our case plugin versions differ only in a special update task

5. fuel2 graph execute --env 1 --trace -t default -T <update_plugin_task_name>

[Ruslan Khozinov (rkhozinov)]

[root@nailgun fuel_plugin_stub-0.2]# cat deployment_tasks.yaml
- id: fuel_plugin_stub
  type: group
  role: [fuel_plugin_stub]
  tasks:
    - hiera
    - globals
  required_for: [deploy_end]
  requires: [deploy_start]
  parameters:
    strategy:
      type: parallel

- id: fuel_plugin_stub-controller-deployment
  type: puppet
  version: 2.0.0
  groups: [primary-controller, controller]
  required_for: [connectivity_tests, deploy_end]
  requires: [netconfig, deploy_start]
  parameters:
    puppet_manifest: "deploy.pp"
    puppet_modules: "."
    timeout: 3600

- id: fuel_plugin_stub-deployment
  type: puppet
  version: 2.0.0
  groups: [fuel_plugin_stub, update_stub]
  required_for: [deploy_end]
  requires: [deploy_start]
  parameters:
    puppet_manifest: "deploy.pp"
    puppet_modules: "."
    timeout: 3600
    retries: 10

- id: fuel_plugin_stub-post-deployment-sh
  type: shell
  version: 2.0.0
  role: [fuel_plugin_stub]
  groups: [fuel_plugin_stub, update_stub]
  required_for: [post_deployment_end]
  requires: [post_deployment_start]
  parameters:
    cmd: bash deploy.sh
    retries: 3
    interval: 20
    timeout: 180

- id: update_stub
  type: group
  role: [update_stub]
  tasks: [hiera, globals, setup_repositories, tools, logging, netconfig, hosts, deploy_start]
  parameters:
    strategy:
      type: one_by_one

[Oleksiy Molchanov (omolchanov)]

Please don't forget to attach diagnostic snapshot.

[Ruslan Khozinov (rkhozinov)]

Diagnostic snapshot: https://drive.google.com/file/d/0B9nJDnRluay9VjRKSGlxZkpHeWc/view?usp=sharing

[Vladimir Kuklin (vkuklin)]

The test case seems to be invalid - there is no such role as fuel_plugin_stub, so the task is not assigned to any node. I suggest that you add it to the role '*' for example or ['/.*/']

[vnaumov (vnaumov-b)]

bug reproduced
steps
1. deploy env
2. install 0.1 plugin
3. deploy changes
4. snapshot
5. install 0.2 plugin
6. configure it
7. run fuel2 graph execute --env 1 -t update_stub (fake_task)
result:
all nodes stucks on "Installing Openstack" case

deployments_tasks.yaml = http://paste.ubuntu.com/24234647/

[Vladimir Kuklin (vkuklin)]

please provide the logs from the environment where deployment 'stucks'

[Vladimir Kuklin (vkuklin)]

I suppose, you also need to sync the tasks with fuel plugin --sync

[vnaumov (vnaumov-b)]

@vkuklin env = http://tpi77:8000/#cluster/1/nodes
please check

[vnaumov (vnaumov-b)]

@vkuklin env = http://172.16.177.77:8000/#cluster/1/nodes
please check

[Vladimir Kuklin (vkuklin)]

the task execution should be done with -T argument as -t is for graph name

[Vladimir Kuklin (vkuklin)]

nodes have status 'deploying' due to the bug https://bugs.launchpad.net/fuel/+bug/1666192

[Vladimir Kuklin (vkuklin)]

the command should be

fuel2 graph execute --env 1 --trace -T update_stub

[vnaumov (vnaumov-b)]

nodes already have status 'deploying' due to the bug https://bugs.launchpad.net/fuel/+bug/1666192
seems we need to close this bug

[Ruslan Khozinov (rkhozinov)]

We couldn't test the issue fully because of the bug https://bugs.launchpad.net/fuel/+bug/1666192

The fix has been merged, but after we've started the update task, no task has been executed.

fuel2 graph execute --env 1 --trace -t default -T update_stub
(fake_task)
task started, but there is no logs in Deployment history and task-script does'not executed

[vnaumov (vnaumov-b)]

WORKAROUND
update 03.04.17
steps
1. deploy env
2. install 0.1 plugin (without fake-update task) http://paste.ubuntu.com/24240378/
3. deploy changes
4. snapshot
5. install 0.2 plugin (with fake-update task) http://paste.ubuntu.com/24306495/
6. enable it (switch radio-button on UI)
7. fuel2 graph execute --env 1 --trace -t default -T update_stub
pay attention on update-task (work-dir must be "/" to avoid plugins-versions-sync issues)
id: update_stub
type: shell
version: 2.0.0
role: ['/.*/']
required_for: [post_deployment_end]
requires: [post_deployment_start]
parameters:
cwd: "/"
cmd: echo foo > /tmp/bar
retries: 3
interval: 20
timeout: 180
-------------------------------
RESULT: SUCCESS