Intermittent ValueErrors thrown from pynetsnmp causes ipdevpoll jobs to fail

[Morten Brekkevold]

Some customers experience intermittent issues with 1minstats (most often) ipdevpoll jobs failing with a "ValueError".

It seems this is either a bug in pynetsnmp or a bug in the actual response from the SNMP agent itself. NAV should handle these errors, either by translating them into "empty" results, or retrying a couple of times before returning an empty result or a timeout error.

The traceback looks as follows:

2014-12-14 06:46:04,909 [ERROR jobs.jobhandler] [1minstats example-sw.example.org] Plugin nav.ipdevpoll.plugins.statsystem.StatSystem(u'example-sw.example.org') reported an unhandled failure
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 397, in errback
    self._startRunCallbacks(fail)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 464, in _startRunCallbacks
    self._runCallbacks()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 551, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1101, in gotResult
    _inlineCallbacks(r, g, deferred)
--- <exception caught here> ---
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1043, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/nav/ipdevpoll/plugins/statsystem.py", line 69, in handle
    cpu = yield self._collect_cpu()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1043, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/nav/ipdevpoll/plugins/statsystem.py", line 118, in _collect_cpu
    load = yield self._get_cpu_loadavg(mib)
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1043, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/nav/ipdevpoll/plugins/statsystem.py", line 129, in _get_cpu_loadavg
    load = yield mib.get_cpu_loadavg()
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 1043, in _inlineCallbacks
    result = result.throwExceptionIntoGenerator(g)
  File "/usr/lib/python2.7/dist-packages/twisted/python/failure.py", line 382, in throwExceptionIntoGenerator
    return g.throw(self.type, self.value, self.tb)
  File "/usr/lib/python2.7/dist-packages/nav/mibs/cisco_process_mib.py", line 34, in get_cpu_loadavg
    TOTAL_1_MIN_REV,
  File "/usr/lib/python2.7/dist-packages/twisted/internet/defer.py", line 551, in _runCallbacks
    current.result = callback(current.result, *args, **kw)
  File "/usr/lib/pymodules/python2.7/pynetsnmp/tableretriever.py", line 59, in saveResults
    for oid, value in values:
exceptions.ValueError: too many values to unpack

[Morten Brekkevold (mbrekkevold)]

fix here: https://nav.uninett.no/hg/stable/rev/19ffa7226e9e