[2.20-Build 64] schema-transformer crash due to its does not handle 'Security-group ID" correctly

Bug #1472988 reported by Daisuke Nakajima
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R2.20
Fix Committed
High
Sachin Bansal
Trunk
Fix Committed
High
Sachin Bansal

Bug Description

Schema transformer failed to run after rebooting node due to its does not handle 'Security-group ID" correctly.

Traceback (most recent call last):
  File "/usr/bin/contrail-schema", line 9, in <module>
    load_entry_point('schema-transformer==0.1dev', 'console_scripts', 'contrail-schema')()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 3932, in server_main
    main()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 3926, in main
    args)
  File "/usr/lib/python2.7/dist-packages/cfgm_common/zkclient.py", line 291, in master_election
    self._election.run(self._zk_election_callback, func, *args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/kazoo/recipe/election.py", line 48, in run
    func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/cfgm_common/zkclient.py", line 283, in _zk_election_callback
    func(*args, **kwargs)
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 3905, in run_schema_transformer
    transformer = SchemaTransformer(args)
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 2701, in __init__
    self.reinit()
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 2778, in reinit
    SecurityGroupST.locate(sg.get_fq_name_str(), sg, sg_acl_dict)
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 128, in locate
    cls._dict[name] = cls(name, *args)
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 1302, in __init__
    self.set_configured_security_group_id(config_id)
  File "/usr/lib/python2.7/dist-packages/schema_transformer/to_bgp.py", line 1315, in set_configured_security_group_id
    if self.name == self._sg_id_allocator.read(int(sg_id)):
  File "/usr/lib/python2.7/dist-packages/cfgm_common/zkclient.py", line 156, in read
    id_str = "%(#)010d" % {'#': idx}
TypeError: %d format: a number is required, not unicode

Security group JSON;
{
    "security-group": {
        "access_control_lists": [
            {
                "href": "http://172.27.113.85:8082/access-control-list/57302f9e-3c55-4ad1-a337-e51a7b86ca50",
                "to": [
                    "default-domain",
                    "demo",
                    "test-SG",
                    "ingress-access-control-list"
                ],
                "uuid": "57302f9e-3c55-4ad1-a337-e51a7b86ca50"
            },
            {
                "href": "http://172.27.113.85:8082/access-control-list/417148e9-d612-4a94-9c00-16edab58d2bb",
                "to": [
                    "default-domain",
                    "demo",
                    "test-SG",
                    "egress-access-control-list"
                ],
                "uuid": "417148e9-d612-4a94-9c00-16edab58d2bb"
            }
        ],
        "configured_security_group_id": 100,
        "display_name": "test-SG",
        "fq_name": [
            "default-domain",
            "demo",
            "test-SG"
        ],
        "href": "http://172.27.113.85:8082/security-group/44e3f960-8a70-4c6c-a104-88cf9bba69f3",
        "id_perms": {
            "created": "2015-07-02T03:00:45.004296",
            "creator": null,
            "description": null,
            "enable": true,
            "last_modified": "2015-07-02T04:58:34.486914",
            "permissions": {
                "group": "admin",
                "group_access": 7,
                "other_access": 7,
                "owner": "admin",
                "owner_access": 7
            },
            "user_visible": true,
            "uuid": {
                "uuid_lslong": 11602548965265533427,
                "uuid_mslong": 4964085407327210604
            }
        },
        "name": "test-SG",
        "parent_href": "http://172.27.113.85:8082/project/8e19297b-5c3c-4b48-85f2-76229e27b1a1",
        "parent_type": "project",
        "parent_uuid": "8e19297b-5c3c-4b48-85f2-76229e27b1a1",
        "security_group_entries": {
            "policy_rule": [
                {
                    "direction": ">",
                    "dst_addresses": [
                        {
                            "network_policy": null,
                            "security_group": "local",
                            "subnet": null,
                            "virtual_network": null
                        }
                    ],
                    "dst_ports": [
                        {
                            "end_port": 65535,
                            "start_port": 0
                        }
                    ],
                    "ethertype": "IPv4",
                    "protocol": "tcp",
                    "rule_uuid": "eb8a1984-f31b-4c39-8107-b6bc1b02c198",
                    "src_addresses": [
                        {
                            "network_policy": null,
                            "security_group": null,
                            "subnet": {
                                "ip_prefix": "0.0.0.0",
                                "ip_prefix_len": 0
                            },
                            "virtual_network": null
                        }
                    ],
                    "src_ports": [
                        {
                            "end_port": 65535,
                            "start_port": 0
                        }
                    ]
                },
                {
                    "direction": ">",
                    "dst_addresses": [
                        {
                            "network_policy": null,
                            "security_group": "local",
                            "subnet": null,
                            "virtual_network": null
                        }
                    ],
                    "dst_ports": [
                        {
                            "end_port": 65535,
                            "start_port": 0
                        }
                    ],
                    "ethertype": "IPv4",
                    "protocol": "tcp",
                    "rule_uuid": "7f2fc51f-d393-4ead-b5a5-d75fa6360ed5",
                    "src_addresses": [
                        {
                            "network_policy": null,
                            "security_group": "default-domain:demo:default",
                            "subnet": null,
                            "virtual_network": null
                        }
                    ],
                    "src_ports": [
                        {
                            "end_port": 65535,
                            "start_port": 0
                        }
                    ]
                },
                {
                    "direction": ">",
                    "dst_addresses": [
                        {
                            "network_policy": null,
                            "security_group": null,
                            "subnet": {
                                "ip_prefix": "0.0.0.0",
                                "ip_prefix_len": 0
                            },
                            "virtual_network": null
                        }
                    ],
                    "dst_ports": [
                        {
                            "end_port": 65535,
                            "start_port": 0
                        }
                    ],
                    "ethertype": "IPv4",
                    "protocol": "any",
                    "rule_uuid": "c3d3dc1d-087e-4508-a893-a52073bc823a",
                    "src_addresses": [
                        {
                            "network_policy": null,
                            "security_group": "local",
                            "subnet": null,
                            "virtual_network": null
                        }
                    ],
                    "src_ports": [
                        {
                            "end_port": 65535,
                            "start_port": 0
                        }
                    ]
                },
                {
                    "direction": ">",
                    "dst_addresses": [
                        {
                            "network_policy": null,
                            "security_group": null,
                            "subnet": {
                                "ip_prefix": "::",
                                "ip_prefix_len": 0
                            },
                            "virtual_network": null
                        }
                    ],
                    "dst_ports": [
                        {
                            "end_port": 65535,
                            "start_port": 0
                        }
                    ],
                    "ethertype": "IPv6",
                    "protocol": "any",
                    "rule_uuid": "2a814208-ec4d-4dc0-8a7a-c3b7325d17e1",
                    "src_addresses": [
                        {
                            "network_policy": null,
                            "security_group": "local",
                            "subnet": null,
                            "virtual_network": null
                        }
                    ],
                    "src_ports": [
                        {
                            "end_port": 65535,
                            "start_port": 0
                        }
                    ]
                }
            ]
        },
        "security_group_id": "100",
        "uuid": "44e3f960-8a70-4c6c-a104-88cf9bba69f3"
    }
}

Tags: ui
information type: Proprietary → Public
Revision history for this message
Prakash Bailkeri (prakashmb) wrote :

Issue can be recreated if security group id is explicitly assigned and schema is restarted.

I patched the setup to convert the security-group id to int() before calling zk api.

    def set_configured_security_group_id(self, config_id):
        if self.config_sgid == config_id:
            return
        self.config_sgid = config_id
        sg_id = self.obj.get_security_group_id()
        if config_id:
            if sg_id is not None:
                if int(sg_id) > SGID_MIN_ALLOC:
                    self._sg_id_allocator.delete(int(sg_id) - SGID_MIN_ALLOC) <<<
                else:
                    if self.name == self._sg_id_allocator.read(int(sg_id)): <<<
                        self._sg_id_allocator.delete(int(sg_id)) <<<
            self.obj.set_security_group_id(str(config_id))
        else:

Changed in juniperopenstack:
assignee: nobody → Sachin Bansal (sbansal)
Revision history for this message
Sachin Bansal (sbansal) wrote :

Users should not be editing security-group-id. This field is for internal use only and can only be set by schema transformer. If you want to change the sg-id, please set configured-security-group-id field.

Changed in juniperopenstack:
status: New → Invalid
Revision history for this message
Daisuke Nakajima (dnakajima) wrote :
Download full text (5.2 KiB)

WebUI set "security_group_id" instead of "congigured-security-group-id" if security group is manually configured.
And, if user cannot modify "security_group_id", please return Error code such as 501 or 503.

root@console:~/Documents/python_test# ./Contrail_json.py -d security-group/eacd4e45-5d41-48bf-b6f1-f1d94f2c7f34
{
    "security-group": {
        "access_control_lists": [
            {
                "href": "http://172.27.113.85:8082/access-control-list/2b42e15c-fdc9-4c1f-8fda-e9124479d360",
                "to": [
                    "default-domain",
                    "demo",
                    "PR1472988",
                    "ingress-access-control-list"
                ],
                "uuid": "2b42e15c-fdc9-4c1f-8fda-e9124479d360"
            },
            {
                "href": "http://172.27.113.85:8082/access-control-list/36e7e82c-ed76-4fc6-8664-ed50590eca18",
                "to": [
                    "default-domain",
                    "demo",
                    "PR1472988",
                    "egress-access-control-list"
                ],
                "uuid": "36e7e82c-ed76-4fc6-8664-ed50590eca18"
            }
        ],
        "configured_security_group_id": 1000,
        "display_name": "PR1472988",
        "fq_name": [
            "default-domain",
            "demo",
            "PR1472988"
        ],
        "href": "http://172.27.113.85:8082/security-group/eacd4e45-5d41-48bf-b6f1-f1d94f2c7f34",
        "id_perms": {
            "created": "2015-07-10T00:09:10.164829",
            "creator": null,
            "description": null,
            "enable": true,
            "last_modified": "2015-07-10T00:09:10.265780",
            "permissions": {
                "group": "admin",
                "group_access": 7,
                "other_access": 7,
                "owner": "admin",
                "owner_access": 7
            },
            "user_visible": true,
            "uuid": {
                "uuid_lslong": 13182583499928665908,
                "uuid_mslong": 16919265434925091007
            }
        },
        "name": "PR1472988",
        "parent_href": "http://172.27.113.85:8082/project/8e19297b-5c3c-4b48-85f2-76229e27b1a1",
        "parent_type": "project",
        "parent_uuid": "8e19297b-5c3c-4b48-85f2-76229e27b1a1",
        "security_group_entries": {
            "policy_rule": [
                {
                    "direction": ">",
                    "dst_addresses": [
                        {
                            "network_policy": null,
                            "security_group": null,
                            "subnet": {
                                "ip_prefix": "0.0.0.0",
                                "ip_prefix_len": 0
                            },
                            "virtual_network": null
                        }
                    ],
                    "dst_ports": [
                        {
                            "end_port": 65535,
                            "start_port": 0
                        }
                    ],
                    "ethertype": "IPv4",
                    "protocol": ...

Read more...

Changed in juniperopenstack:
status: Invalid → New
status: New → In Progress
Revision history for this message
Daisuke Nakajima (dnakajima) wrote :

Above comment, WebUI seems to set both "configured_security_group_id" and "security_group_id".

Revision history for this message
Sachin Bansal (sbansal) wrote :

webui should not set security_group_id. It should only set configured_security_group_id.

Changed in juniperopenstack:
assignee: Sachin Bansal (sbansal) → Rahul (rahuls)
tags: added: ui
Changed in juniperopenstack:
status: In Progress → New
Changed in juniperopenstack:
importance: Undecided → High
Revision history for this message
asbalaji (asbalaji) wrote :

I tried to do create a SG with 'auto' and edited same SG with user configured value. In both the cases I could see web ui setting configured_security_group_id, not security_group_id. Is there any setup where I can reproduce this issue. I used R2.2 - 64 build as mentioned in the bug title.

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.20

Review in progress for https://review.opencontrail.org/13218
Submitter: Sachin Bansal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/13218
Committed: http://github.org/Juniper/contrail-controller/commit/9168173d55bbb23d3da52e54e3f8f5d43cf2dd1c
Submitter: Zuul
Branch: R2.20

commit 9168173d55bbb23d3da52e54e3f8f5d43cf2dd1c
Author: Sachin Bansal <email address hidden>
Date: Thu Aug 20 10:50:28 2015 -0700

Convert SG id to integer before calling index allocator api

Change-Id: Ie8ff18a58a7e713991c3b6e6a633c65bfb74ccae
Closes-Bug: 1472988

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] master

Review in progress for https://review.opencontrail.org/13254
Submitter: Sachin Bansal (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/13254
Committed: http://github.org/Juniper/contrail-controller/commit/0eb3799a6935c77d15da03252578165e830b57d8
Submitter: Zuul
Branch: master

commit 0eb3799a6935c77d15da03252578165e830b57d8
Author: Sachin Bansal <email address hidden>
Date: Thu Aug 20 10:50:28 2015 -0700

Convert SG id to integer before calling index allocator api

Change-Id: Ie8ff18a58a7e713991c3b6e6a633c65bfb74ccae
Closes-Bug: 1472988
(cherry picked from commit 15e703f8520c7c0aed0c17196bd04562c69a44bb)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.22-dev

Review in progress for https://review.opencontrail.org/13927
Submitter: Vinay Vithal Mahuli (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : [Review update] R2.20.x

Review in progress for https://review.opencontrail.org/14207
Submitter: Hampapur Ajay (<email address hidden>)

Revision history for this message
OpenContrail Admin (ci-admin-f) wrote : A change has been merged

Reviewed: https://review.opencontrail.org/14207
Committed: http://github.org/Juniper/contrail-controller/commit/6557a4daebdf3ab927bbdc499fe4ce5985d5c121
Submitter: Zuul
Branch: R2.20.x

commit 6557a4daebdf3ab927bbdc499fe4ce5985d5c121
Author: Sachin Bansal <email address hidden>
Date: Thu Aug 20 10:50:28 2015 -0700

Convert SG id to integer before calling index allocator api

Change-Id: Ie8ff18a58a7e713991c3b6e6a633c65bfb74ccae
Closes-Bug: 1472988
(cherry picked from commit 9168173d55bbb23d3da52e54e3f8f5d43cf2dd1c)

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.