Can't healing vdu in scaling group

Bug #1839440 reported by Hiroya Nakaya
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
tacker
Won't Fix
Low
Yasufumi Ogawa

Bug Description

I try to heal vdu in scaling group.
But I can't heal vdu now.
I think vdu_autoheal by ping monitoring currently doesn't support scaling group.

vdu_autoheal by ping monitoring has two issues.
1. ping monitoring is failed.
2. Tacker can't know healing target.

details
----

depolyed VNFD
```
tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0

description: sample-tosca-vnfd-scaling

metadata:
  template_name: sample-tosca-vnfd-scaling

topology_template:
  node_templates:
    VDU1:
      type: tosca.nodes.nfv.VDU.Tacker
      properties:
        image: cirros-0.4.0-x86_64-disk
        mgmt_driver: noop
        monitoring_policy:
          name: ping
          parameters:
            monitoring_delay: 45
            count: 3
            interval: 1
            timeout: 2
          actions:
            failure: vdu_autoheal
        availability_zone: nova
        flavor: m1.tiny

    CP1:
      type: tosca.nodes.nfv.CP.Tacker
      properties:
        management: true
        order: 0
        anti_spoofing_protection: false
      requirements:
        - virtualLink:
            node: VL1
        - virtualBinding:
            node: VDU1

    VL1:
      type: tosca.nodes.nfv.VL
      properties:
        network_name: net_mgmt
        vendor: Tacker

  policies:
    - SP1:
        type: tosca.policies.tacker.Scaling
        targets: [VDU1]
        properties:
          increment: 1
          cooldown: 120
          min_instances: 1
          max_instances: 3
          default_instances: 1
```

1. ping monitoring is failed.

Tacker currently can check VDU by ping.
But this monitoring fail in pattern defined scaling policy.
The reason is that ping monitoring doesn't support scaling policy's mgmt_ip.

error Log
```
2019-08-07 07:31:48.338 21971 DEBUG tacker.common.log [-] tacker.vnfm.monitor_drivers.ping.ping.VNFMonitorPing method monitor_call called with arguments () {'vnf': {'status': 'ACTIVE', 'created_at': datetime.datetime(2019, 8, 7, 7, 24, 49, 617154), 'mgmt_ip_address': '{"VDU1": ["192.168.120.8", "192.168.120.3"]}', 'description': u'sample-tosca-vnfd-scaling', 'vnfd': {'template_source': u'onboarded', 'service_types': [u'vnfd'], 'description': u'sample-tosca-vnfd-scaling', 'tenant_id': u'85683c0847024403b900d209b392507f', 'created_at': datetime.datetime(2019, 8, 7, 7, 24, 3), 'updated_at': None, 'mgmt_driver': u'noop', 'attributes': {u'vnfd': u'description: sample-tosca-vnfd-scaling\nmetadata:\n template_name: sample-tosca-vnfd-scaling\ntopology_template:\n node_templates:\n CP1:\n properties:\n anti_spoofing_protection: false\n management: true\n order: 0\n requirements:\n - virtualLink:\n node: VL1\n - virtualBinding:\n node: VDU1\n type: tosca.nodes.nfv.CP.Tacker\n VDU1:\n properties:\n availability_zone: nova\n flavor: m1.tiny\n image: cirros-0.4.0-x86_64-disk\n mgmt_driver: noop\n monitoring_policy:\n actions:\n failure: vdu_autoheal\n name: ping\n parameters:\n count: 3\n interval: 1\n monitoring_delay: 45\n timeout: 2\n type: tosca.nodes.nfv.VDU.Tacker\n VL1:\n properties:\n network_name: net_mgmt\n vendor: Tacker\n type: tosca.nodes.nfv.VL\n policies:\n - SP1:\n properties:\n cooldown: 120\n default_instances: 2\n increment: 1\n max_instances: 3\n min_instances: 1\n targets:\n - VDU1\n type: tosca.policies.tacker.Scaling\ntosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0\n'}, 'id': u'5f0e97bd-666d-421b-a851-6a5c25e32f1e', 'name': u'scaling-and-vdu-healing'}, 'vnfd_id': u'5f0e97bd-666d-421b-a851-6a5c25e32f1e', 'tenant_
id': u'85683c0847024403b900d209b392507f', 'vim_id': u'7d4ff0e3-69a7-4189-b397-a0a5dde936b4', 'updated_at': None, 'instance_id': u'40062ffa-01ac-4ad4-8c61-665d8e9fcb2c', 'error_reason': None, 'placement_attr': {'vim_name': u'admin'}, 'attributes': {'scaling_group_names': '{"SP1": "SP1_group"}', 'heat_template': "heat_template_version: 2013-05-23\ndescription: 'sample-tosca-vnfd-scaling\n\n '\nparameters: {}\nresources:\n SP1_scale_out:\n type: OS::Heat::ScalingPolicy\n properties:\n auto_scaling_group_id:\n get_resource: SP1_group\n adjustment_type: change_in_capacity\n scaling_adjustment: 1\n cooldown: 120\n SP1_group:\n type: OS::Heat::AutoScalingGroup\n properties:\n min_size: 1\n desired_capacity: 2\n cooldown: 120\n resource:\n type: SP1_res.yaml\n max_size: 3\n SP1_scale_in:\n type: OS::Heat::ScalingPolicy\n properties:\n auto_scaling_group_id:\n get_resource: SP1_group\n adjustment_type: change_in_capacity\n scaling_adjustment: -1\n cooldown: 120\noutputs: {}\n", 'SP1_res.yaml': 'heat_template_version: 2013-05-23\ndescription: Tacker Scaling template\nresources:\n VDU1:\n type: OS::Nova::Server\n properties:\n user_data_format: SOFTWARE_CONFIG\n availability_zone: nova\n image: cirros-0.4.0-x86_64-disk\n flavor: m1.tiny\n networks:\n - port:\n get_resource: CP1\n config_drive: false\n CP1:\n type: OS::Neutron::Port\n properties:\n port_security_enabled: false\n network: net_mgmt\n VL1:\n type: OS::Neutron::Net\noutputs:\n mgmt_ip-VDU1:\n value:\n get_attr:\n - CP1\n - fixed_ips\n - 0\n - ip_address\n', 'monitoring_policy': '{"vdus": {"VDU1": {"ping": {"actions": {"failure": "vdu_autoheal"}, "name": "ping", "parameters": {"count": 3, "interval": 1, "monitoring_delay": 45, "timeout": 2}, "monitoring_params": {"count": 3, "interval": 1, "monitoring_delay": 45, "timeout": 2}}}}}'}, 'id': '044e26c9-c754-4c0
d-b4f9-370a1d74e917', 'name': u'scaling-and-vdu-healing'}, 'kwargs': {u'count': 3, u'monitoring_delay': 45, u'interval': 1, 'mgmt_ip': [u'192.168.120.8', u'192.168.120.3'], u'timeout': 2}} wrapper /opt/stack/tacker/tacker/common/log.py:34
2019-08-07 07:31:48.361 21971 DEBUG tacker.agent.linux.utils [-] Running command: ['ping', '-c', '3', '-W', '2', '-i', '1', "[u'192.168.120.8', u'192.168.120.3']"] create_process /opt/stack/tacker/tacker/agent/linux/utils.py:42
2019-08-07 07:31:48.419 21971 ERROR tacker.agent.linux.utils [-]
Command: ['ping', '-c', '3', '-W', '2', '-i', '1', "[u'192.168.120.8', u'192.168.120.3']"]
Exit code: 2
Stdout: ''
Stderr: "ping: unknown host [u'192.168.120.8', u'192.168.120.3']\n"
2019-08-07 07:31:48.423 21971 WARNING tacker.vnfm.monitor_drivers.ping.ping [-] Cannot ping ip address: [u'192.168.120.8', u'192.168.120.3']: RuntimeError:
2019-08-07 07:31:48.425 21971 DEBUG tacker.vnfm.monitor [-] driver_return failure run_monitor /opt/stack/tacker/tacker/vnfm/monitor.py:188
2019-08-07 07:31:48.425 21971 DEBUG tacker.vnfm.plugin [-] policy action: vdu_autoheal action_cb /opt/stack/tacker/tacker/vnfm/plugin.py:244
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor [-] Unknown exception: Monitoring failed for VNF '044e26c9-c754-4c0d-b4f9-370a1d74e917' due to ''NoneType' object has no attribute '__getitem__'' : TypeError: 'NoneType' object has no attribute '__getitem__'
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor Traceback (most recent call last):
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor File "/opt/stack/tacker/tacker/vnfm/monitor.py", line 99, in __run__
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor self.run_monitor(hosting_vnf)
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor File "/opt/stack/tacker/tacker/vnfm/monitor.py", line 192, in run_monitor
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor hosting_vnf['action_cb'](action, vdu_name=vdu)
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor File "/opt/stack/tacker/tacker/vnfm/plugin.py", line 247, in action_cb
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor vnf_dict=hosting_vnf['vnf'], args=kwargs)
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor File "/opt/stack/tacker/tacker/common/driver_manager.py", line 70, in invoke
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor return getattr(driver, method_name)(**kwargs)
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor File "/opt/stack/tacker/tacker/vnfm/policy_actions/vdu_autoheal/vdu_autoheal.py", line 57, in execute_action
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor resource_list = _get_vdu_resources()
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor File "/opt/stack/tacker/tacker/vnfm/policy_actions/vdu_autoheal/vdu_autoheal.py", line 51, in _get_vdu_resources
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor cp_resources = vdu_resources['properties'].get('networks')
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor TypeError: 'NoneType' object has no attribute '__getitem__'
2019-08-07 07:31:48.441 21971 ERROR tacker.vnfm.monitor
```

2. Tacker can't know healing target.

Even if tacker can monitor VDU by ping, vdu_autoheal can't heal VDU.
Tacker's vdu_autoheal select target VDU by VDU name.
But, when VNFD used scale policy and VDU scale out, VNF may have multiple VDU having same name.
For that reason, Tacker can't select the target which should be healed.

Revision history for this message
Hiroya Nakaya (nakkay) wrote :

I think we need to consider how to manage scaling group's resource.

Yasufumi Ogawa (yasufum)
Changed in tacker:
assignee: nobody → Yasufumi Ogawa (yasufum)
Revision history for this message
Hiromu Asahina (h-asahina) wrote :

Hi,

As it's been a long time since this bug was reported, I'd like to confirm the current status. Are you still having trouble? or you have solved the problem by simply updating Tacker?

Yasufumi Ogawa (yasufum)
Changed in tacker:
importance: Undecided → Low
Yasufumi Ogawa (yasufum)
Changed in tacker:
status: New → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.