Juniper Openstack

R5.0-214: policy creation with 1500 rules fails with error "Request to large"

Bug #1794296 reported by alok kumar
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Juniper Openstack
Status tracked in Trunk
R5.0
New
High
alok kumar
Juniper Openstack r5.0.3
Trunk
New
High
alok kumar
Juniper Openstack r5.1.0

Bug Description

Network policy creation with 1500 rules fails with below error in contrail-api.log:

09/25/2018 12:52:26 PM [contrail-api] [ERROR]: __default__ [SYS_ERR]: VncApiError: <pre>Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 2253, in handler_trap_exception
    response = handler(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/cfgm_common/vnc_api_stats.py", line 17, in wrapper
    response = func(api_server_obj, resource_type, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/vnc_cfg_api_server/vnc_cfg_api_server.py", line 901, in http_resource_create
    obj_dict = get_request().json[resource_type]
  File "/usr/lib/python2.7/site-packages/bottle.py", line 165, in __get__
    if key not in storage: storage[key] = self.getter(obj)
  File "/usr/lib/python2.7/site-packages/bottle.py", line 1119, in json
    b = self._get_body_string()
  File "/usr/lib/python2.7/site-packages/bottle.py", line 1182, in _get_body_string
    raise HTTPError(413, 'Request to large')
HTTPError
</pre>

exeception seen while debugging in pdb session as:

(Pdb) policy_objs_list = policy_test_helper._create_n_policy_n_rules(self, number_of_policy, valid_rules, 1432,policy_name='1432')
2018-09-25 12:52:26,050 - ERROR - Exception
    <!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
    <html>
        <head>
            <title>Error: 413 Request Entity Too Large</title>
            <style type="text/css">
              html {background-color: #eee; font-family: sans;}
              body {background-color: #fff; border: 1px solid #ddd;
                    padding: 15px; margin: 15px;}
              pre {background-color: #eee; border: 1px solid #ddd; padding: 5px;}
            </style>
        </head>
        <body>
            <h1>Error: 413 Request Entity Too Large</h1>
            <p>Sorry, the requested URL <tt>&#039;http://10.1.0.16:8082/network-policys&#039;</tt>
               caused an error:</p>
            <pre>Request to large</pre>
        </body>
    </html>
 occured while creating 1 policy with 1434 rules
*** AssertionError: Exception occured while creating 1 policy with 1434 rules

while debugging it was seen that, policy with 1433 rules went through fine but fails with 1434 onwards.

This is causing test case test_one_policy_rules_scaling_with_ping to fail.

test details:
- test tries to create policy with 1502 rules and then validates simple ping test.
- 2 rules are added to allow icmp and udp traffic and rest of the rules are deny rules with diff. src/dst ports.

sample deny rule(total 1500):
                {
                    'direction': '<>', 'simple_action': 'deny',
                    'protocol': 'udp', 'src_ports': (x, x),
                    'dst_ports': (y, y),
                    'source_network': 'any',
                    'dest_network': 'any',
                }

allow rules:
            {
                'direction': '<>', 'simple_action': 'pass',
                'protocol': 'icmp', 'src_ports': 'any',
                'dst_ports': 'any',
                'source_network': 'any',
                'dest_network': 'any',
            },
            {
                'direction': '<>', 'simple_action': 'pass',
                'protocol': 'udp', 'src_ports': 'any',
                'dst_ports': 'any',
                'source_network': 'any',
                'dest_network': 'any',
            }

contrail logs from one of the controller will be copied at /cs-shared/bugs/<bugId>

Revision history for this message
Shivayogi Ugaji (shivayogi123) wrote :

Recently a bottle request check was introduced.
Can you try enabling the following line from etc/contrail/contrail-api.conf
# max_request_size = 1024000

Revision history for this message
alok kumar (kalok) wrote :

what is the max value supported for max_request_size.
I tried with max_request_size = 1024000 in default section, but still hitting the same issue.

Revision history for this message
Yijie Xiao (yixiao2018) wrote :

For the ‘Request too large’ problem, it is because the request length is indeed too large and Yogi’s idea is right.

/usr/lib/python2.7/site-packages/bottle.py:

(Pdb) l
1178 ''' read body until content-length or MEMFILE_MAX into a string. Raise
1179 HTTPError(413) on requests that are to large. '''
1180 clen = self.content_length
1181 if clen > self.MEMFILE_MAX:
1182 import pdb; pdb.set_trace()
1183 -> raise HTTPError(413, 'Request to large')
1184 if clen < 0: clen = self.MEMFILE_MAX + 1
1185 data = self.body.read(clen)
1186 if len(data) > self.MEMFILE_MAX: # Fail fast
1187 raise HTTPError(413, 'Request to large')
1188 return data
(Pdb) p self.MEMFILE_MAX
1024000
(Pdb) p clen
1063685
(Pdb)

As we can see, the size of request to create 1500 rules is larger than 1024000. In this case, you can set the ‘max_request_size’ to a larger number in contrail-api.conf file.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.