contrail-analytics-api log file flooded with ConnectionError traceback

[Sundaresan Rajangam]

contrail-analytics-api.log file is being flooded with the following Exception traceback

05/06/2016 12:13:01 AM [contrail-analytics-api]: Exception ConnectionError in uve stream proc. Arguments:
("Error while reading from socket: ('timed out',)",) : traceback Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/opserver/partition_handler.py", line 354, in _run
    for message in pb.listen():
  File "/usr/lib/python2.7/dist-packages/redis/client.py", line 1715, in listen
    r = self.parse_response()
  File "/usr/lib/python2.7/dist-packages/redis/client.py", line 1659, in parse_response
    response = self.connection.read_response()
  File "/usr/lib/python2.7/dist-packages/redis/connection.py", line 315, in read_response
    response = self._parser.read_response()
  File "/usr/lib/python2.7/dist-packages/redis/connection.py", line 105, in read_response
    response = self.read()
  File "/usr/lib/python2.7/dist-packages/redis/connection.py", line 94, in read
    (e.args,))
ConnectionError: Error while reading from socket: ('timed out',)

We need to catch the ConnectionError and not log the traceback in this case

[Robert Rosiak (robert-rosiak)]

anyone knows what is the root cause of this error ? All 3 HA analytics nodes are flooded with this error.

contrail-analytics 3.0.0.0-2725 2725
contrail-openstack-analytics 3.0.0.0-2725 2725

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20125
Submitter: Sundaresan Rajangam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20126
Submitter: Sundaresan Rajangam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/20125
Submitter: Sundaresan Rajangam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20126
Committed: http://github.org/Juniper/contrail-controller/commit/1ef74537fbf652c38088c8706c0cd8bbd60efb40
Submitter: Zuul
Branch: R3.0

commit 1ef74537fbf652c38088c8706c0cd8bbd60efb40
Author: Sundaresan Rajangam <email address hidden>
Date: Wed May 11 14:50:58 2016 -0700

Don't log traceback for redis ConnectionError exception

In partition_handler, the redis connection timesout if there is no
activity for 90 seconds. We have catch all exception that logs the
traceback. Need to catch the ConnectionError exception and not log the
traceback for this case.

Change-Id: If2f391d5ddf3a04aed82b889b1d780de60c4da6b
Closes-Bug: #1578779
(cherry picked from commit 2539e2e1ed9bab1cffce2fa47ebc066d3693c53d)

[OpenContrail Admin (ci-admin-f)]

Reviewed: https://review.opencontrail.org/20125
Committed: http://github.org/Juniper/contrail-controller/commit/5e0db117316e1cbd5d1a2df4c418d952259bd089
Submitter: Zuul
Branch: master

commit 5e0db117316e1cbd5d1a2df4c418d952259bd089
Author: Sundaresan Rajangam <email address hidden>
Date: Wed May 11 14:50:58 2016 -0700

Don't log traceback for redis ConnectionError exception

In partition_handler, the redis connection timesout if there is no
activity for 90 seconds. We have catch all exception that logs the
traceback. Need to catch the ConnectionError exception and not log the
traceback for this case.

Change-Id: If2f391d5ddf3a04aed82b889b1d780de60c4da6b
Closes-Bug: #1578779

[Jun Wei Wang (wjw7869)]

I still met with this issue, after mergeing this patch.

[Sundaresan Rajangam (srajanga)]

@Jun Wei Wang, Can you please upload the log file?

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/32767
Submitter: Sundaresan Rajangam (<email address hidden>)

[OpenContrail Admin (ci-admin-f)]

Review in progress for https://review.opencontrail.org/32768
Submitter: Sundaresan Rajangam (<email address hidden>)

[Sundaresan Rajangam (srajanga)]

Ignore #12 and #14