tosca-parser error with nfv yaml file

[david]

Dears,

I have started using tosca-parser module but I got the following error.
Kindly find bellow my .yaml file.

    tosca = ToscaTemplate(path, parsed_params, a_file)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tosca_template.py", line 101, in __init__
    self.topology_template = self._topology_template()
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tosca_template.py", line 120, in _topology_template
    None)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/topology_template.py", line 61, in __init__
    self.groups = self._groups()
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/topology_template.py", line 163, in _groups
    self.custom_defs)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/groups.py", line 29, in __init__
    custom_defs)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/entity_template.py", line 74, in __init__
    if type is not None else None
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/elements/grouptype.py", line 35, in __init__
    if self.DESCRIPTION in self.defs:
TypeError: argument of type 'NoneType' is not iterable
127.0.0.1 - - [31/Oct/2017 12:33:35] "POST /deploy_template HTTP/1.1" 500 2641

For more information, the input parameters for the following source code line are:
  tosca = ToscaTemplate(path, parsed_params, a_file)
#path= the path to the yaml file
#parsed_params={}
#a_file=True

I appriciate your support to fix this issue.
Thanks in advance.
Best regards.

##################yaml file########################
tosca_definitions_version: tosca_simple_yaml_1_0
description: This is a basic example for test
topology_template:

  inputs:
    network_name:
      type: string
      default: admin_internal_net
    odl:
      type: string
      default: 192.168.111.28:8181

  node_templates:

    VM1:
      type: tosca.nodes.Compute
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type: sfc_client
            distribution: ubuntu
            version: 14.04

    VM2:
      type: tosca.nodes.Compute
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type: sfc_client
            distribution: ubuntu
            version: 14.04

    VM3:
      type: tosca.nodes.Compute
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type: sfc_client
            distribution: ubuntu
            version: 14.04

    my_network:
      type: tosca.nodes.network.Network
      properties:
        network_name: { get_input: network_name }

    my_port1:
      type: tosca.nodes.network.Port
      requirements:
        - binding:
            node: VM1
        - link:
            node: my_network

    my_port2:
      type: tosca.nodes.network.Port
      requirements:
        - binding:
            node: VM2
        - link:
            node: my_network

    my_port3:
      type: tosca.nodes.network.Port
      requirements:
        - binding:
            node: VM3
        - link:
            node: my_network

    VNF1:
      type: tosca.nodes.nfv.VNF
      properties:
      attributes:
        type: dpi
        address: 10.100.0.105
        port: 40000
        nsh_aware: true
      requirements:
        - host: VM1

    CP11: #endpoints of VNF1 linked to VL1
      type: tosca.nodes.nfv.CP
      properties:
      attributes:
        IP_address: 10.100.0.105
        interface: ens3
        port: 30000
      requirements:
        - virtualBinding: VNF1
        - virtualLink: VL1

    VNF2:
      type: tosca.nodes.nfv.VNF
      properties:
      attributes:
        type: firewall
        address: 10.100.0.106
        port: 40000
        nsh_aware: true
      requirements:
        - host: VM2

    CP21: #endpoints of VNF2 linked to VL1
      type: tosca.nodes.nfv.CP
      properties:
      attributes:
        IP_address: 10.100.0.106
        interface: ens3
        port: 30000
      requirements:
        - virtualBinding: VNF2
        - virtualLink: VL1

    VNF3:
      type: tosca.nodes.nfv.VNF
      properties:
      attributes:
        type: napt44
        address: 10.100.0.107
        port: 40000
        nsh_aware: true
      requirements:
        - host: VM3

    CP31: #endpoints of VNF3 linked to VL2
      type: tosca.nodes.nfv.CP
      properties:
      attributes:
        IP_address: 10.100.0.107
        port: 30000
        interface: ens3
      requirements:
        - virtualBinding: VNF3
        - virtualLink: VL1

    VL1:
      type: tosca.nodes.nfv.VL
      properties:
        vendor: HP
      attributes:
        type: ip
        transport_type: vxlan-gpe

    Forwarding_path1:
      type: tosca.nodes.nfv.FP
      description: the path (CP11->CP21->CP31)
      properties:
        policy:
      requirements:
        - forwarder: CP11
        - forwarder: CP21
        - forwarder: CP31

  #################################################
  # VNF Forwarding Graph nodes and the associated
  # Network Forwarding Paths
  #################################################

  groups:
    VNF_FG1:
      type: tosca.groups.nfv.VNFFG
      description: VNF forwarding graph
      properties:
        vendor:
        version:
        connection_point: [ CP11, CP21, CP31 ]
        dependent_virtual_link: [ VL1 ]
        constituent_vnfs: [ VNF1, VNF2, VNF3 ]
      members: [ Forwarding_path1 ]

  outputs:
    vnf1_ip:
      description: The private IP address of the VNF container1.
      value: { get_attribute: [VM1, private_address] }
    vnf2_ip:
      description: The private IP address of the VNF container2.
      value: { get_attribute: [VM2, private_address] }
    vnf3_ip:
      description: The private IP address of the VNF container3.
      value: { get_attribute: [VM3, private_address] }

[Sahdev Zala (spzala)]

Hi David, are you using any specific release of tosca-parser? or master version? Also subscribing Bob to this bug. Thanks!

[david (davidgab)]

Thank you for the reply.
These commands may be useful to describe my execution environment.
I highlight that I executed them few days before to setup my openstack instance.

sudo pip install pyyaml
sudo pip install python-novaclient
sudo pip install --upgrade setuptools
pip install packaging
sudo pip install tosca-parser
cd PATH_TO_TOSCA-PARSER/Tosca-parser && sudo python setup.py install
cd PATH_TO_HOT-TRANSLATOR/hot-translator && sudo python setup.py install
sudo pip install ceilometerclient

Kind regards.

[Bob Haddleton (bob-haddleton)]

Hi David:

First, you should not need to do both the pip install tosca-parser and the python setup.py install of tosca-parser. Theoretically it should work but since you are instaling with pip into the system python (with sudo) and running setup.py as your user there could be conflicts at runtime.

You might want to try using a virtualenv and just install using pip.

Second, you are trying to parse a template that has NFV nodes in it but does not use the NFV template version, so the parser cannot find the definitions of the NFV objects.

You need to change the template version to: "tosca_simple_profile_for_nfv_1_0_0" so the parser can find the definitions for the NFV nodes.

Hope this helps

Bob

[david (davidgab)]

Thanks for your support.
I uninstall tosca-parser module then I run pip command (sudo pip install tosca-parser) to re-install it.
I also update the header of my .yaml file as suggested by Bob.
Now I get a new error as described below:
    tosca = ToscaTemplate(path, parsed_params, a_file)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tosca_template.py", line 101, in __init__
    self.topology_template = self._topology_template()
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tosca_template.py", line 120, in _topology_template
    None)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/topology_template.py", line 60, in __init__
    self.graph = ToscaGraph(self.nodetemplates)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tpl_relationship_graph.py", line 19, in __init__
    self._create()
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tpl_relationship_graph.py", line 40, in _create
    relation = node.relationships
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/nodetemplate.py", line 61, in relationships
    explicit = self._get_explicit_relationship(r, value)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/nodetemplate.py", line 144, in _get_explicit_relationship
    for rtype in self.type_definition.relationship.keys():
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/elements/nodetype.py", line 89, in relationship
    relatednode = NodeType(node_type, self.custom_def)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/elements/nodetype.py", line 30, in __init__
    super(NodeType, self).__init__(ntype, self.NODE_PREFIX, custom_def)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/elements/statefulentitytype.py", line 38, in __init__
    if entitytype.startswith(self.TOSCA + ":"):
AttributeError: 'NoneType' object has no attribute 'startswith'

Kindly find bellow my .yaml file in case of further information.

tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0
description: This is a basic example for test
topology_template:

  inputs:
    network_name:
      type: string
      default: admin_internal_net
    odl:
      type: string
      default: 157.159.233.40:8181

  node_templates:

    VM1:
      type: tosca.nodes.Compute
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type: sfc_client
            distribution: ubuntu
            version: 14.04

    VM2:
      type: tosca.nodes.Compute
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type: sfc_client
            distribution: ubuntu
            version: 14.04

    VM3:
   ...

[Miguel Caballer (micafer1)]

Hi David,

It seems that the error comes from this node:

    Forwarding_path1:
      type: tosca.nodes.nfv.FP
      description: the path (CP11->CP21->CP31)
      properties:
        policy:
      requirements:
        - forwarder: CP11
        - forwarder: CP21
        - forwarder: CP31

The forwarder requirement needs that the node specified (in this case CP11) has the capability: "tosca.capabilities.nfv.Forwarder".

CP111 is a "tosca.nodes.nfv.CP" nodetype does not has it and it is the origin of the error.

[david (davidgab)]

Thanks Miguel for your help.
I updated my .yaml file as suggested and also changed 'VM1' by 'VDU1'.
Besides I changed 'tosca.nodes.Compute' by 'tosca.nodes.nfv.VDU'.
But I got the following error. I am wondering if the use of 'tosca.nodes.nfv.VL' is right or I have to use 'tosca.nodes.nfv.VL.ELAN'?
Kindly find bellow my new .yaml file.

  tosca = ToscaTemplate(path, parsed_params, a_file)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tosca_template.py", line 101, in __init__
    self.topology_template = self._topology_template()
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tosca_template.py", line 120, in _topology_template
    None)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/topology_template.py", line 60, in __init__
    self.graph = ToscaGraph(self.nodetemplates)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tpl_relationship_graph.py", line 19, in __init__
    self._create()
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/tpl_relationship_graph.py", line 40, in _create
    relation = node.relationships
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/nodetemplate.py", line 61, in relationships
    explicit = self._get_explicit_relationship(r, value)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/nodetemplate.py", line 144, in _get_explicit_relationship
    for rtype in self.type_definition.relationship.keys():
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/elements/nodetype.py", line 89, in relationship
    relatednode = NodeType(node_type, self.custom_def)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/elements/nodetype.py", line 30, in __init__
    super(NodeType, self).__init__(ntype, self.NODE_PREFIX, custom_def)
  File "/usr/local/lib/python2.7/dist-packages/toscaparser/elements/statefulentitytype.py", line 38, in __init__
    if entitytype.startswith(self.TOSCA + ":"):
AttributeError: 'NoneType' object has no attribute 'startswith'

#####################.yaml file#############################
tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0
description: This is a basic example for test
topology_template:

  inputs:
    network_name:
      type: string
      default: admin_internal_net
    odl:
      type: string
      default: 157.159.233.40:8181

  node_templates:

    VDU1:
      type: tosca.nodes.nfv.VDU
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type: sfc_client
            distribution: ubuntu
            version: 14.04

    VDU2:
      type: tosca.nodes.nfv.VDU
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type...

[Miguel Caballer (micafer1)]

Hi David,

You have defined the CP11 node with type: tosca.capabilities.nfv.Forwarder

It is not correct you must define a node type "tosca.node.*" that exposes the "tosca.capabilities.nfv.Forwarder" capability.

Best regards.

[david (davidgab)]

thanks you Miguel for your help.
But I am confusing and I don't know exactly how to do this ?
Could you tell me if this is the suitable standard I have to consider: http://docs.oasis-open.org/tosca/tosca-nfv/v1.0/tosca-nfv-v1.0.html ?
Is it possible to share a .yaml example I can follow to correctly write mine ?
This is the new .yaml file.

tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0
description: This is a basic example for test
topology_template:

  inputs:
    network_name:
      type: string
      default: admin_internal_net
    odl:
      type: string
      default: 157.159.233.40:8181

  node_templates:

    VDU1:
      type: tosca.nodes.nfv.VDU
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type: sfc_client
            distribution: ubuntu
            version: 14.04

    VDU2:
      type: tosca.nodes.nfv.VDU
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type: sfc_client
            distribution: ubuntu
            version: 14.04

    VDU3:
      type: tosca.nodes.nfv.VDU
      capabilities:
        # Host container properties
        host:
         properties:
           num_cpus: 2
           disk_size: 10 GB
           mem_size: 512 MB
        # Guest Operating System properties
        os:
          properties:
            # host Operating System image properties
            architecture: x86_64
            type: sfc_client
            distribution: ubuntu
            version: 14.04

    my_network:
      type: tosca.nodes.network.Network
      properties:
        network_name: { get_input: network_name }

    my_port1:
      type: tosca.nodes.network.Port
      requirements:
        - binding:
            node: VDU1
        - link:
            node: my_network

    my_port2:
      type: tosca.nodes.network.Port
      requirements:
        - binding:
            node: VDU2
        - link:
            node: my_network

    my_port3:
      type: tosca.nodes.network.Port
      requirements:
        - binding:
            node: VDU3
        - link:
            node: my_network

    VNF1:
      type: tosca.nodes.nfv.VNF
      properties:
      attributes:
        type: dpi
        address: 10.100.0.105
        port: 40000
        nsh_aware: true
      requirements:
        - host: VDU1

    CP11: #endpoints of VNF1 linked to VL1
      type: tosca.nodes.nfv.VduCpd
      properties:
      attributes:
        IP_address: 10.100.0.105
        interface: ens3
        port: 30000
      requirements:
        - virtualBinding: VNF1
        - virtualLink: VL1
      capabilities: tosca.capabilities.nfv.Forwa...

[Miguel Caballer (micafer1)]

Hi David,

I'm not an expert on the NFV extensions of TOSCA.
I only see the error that cause the TOSCA exception that does not follow the TOSCA specification.

[Sahdev Zala (spzala)]

Thanks Bob and agree with Miguel. So I would recommend taking a look at few examples as you asked here - https://github.com/openstack/tosca-parser/tree/master/toscaparser/extensions/nfv/tests/data and https://github.com/openstack/tacker/tree/master/samples/tosca-templates May be good idea that you use couple of those templates first as such and see if that works and then modify your using them as reference. Thanks!

[david (davidgab)]

Thank you very much for these useful examples.
Would like to share the standard that I have to consider.
This one http://docs.oasis-open.org/tosca/tosca-nfv/v1.0/tosca-nfv-v1.0.html should be up to date or not ?

Best regards