Servicemon crashes when service checker init fails

[Morten Brekkevold]

On several of our clients' installations, servicemon will invariably crash just after startup. This must have been introduced witht he bugfixes in NAV 3.5.1. Traceback is as follows:

[2009-03-12 08:31:50] abstractChecker.py:__init__:105 [Info] New checker instance for www.example.org:http
Traceback (most recent call last):
  File "/usr/lib/nav/servicemon.py", line 274, in ?
    start(fork)
  File "/usr/lib/nav/servicemon.py", line 229, in start
    myController.main()
  File "/usr/lib/nav/servicemon.py", line 128, in main
    self.getCheckers()
  File "/usr/lib/nav/servicemon.py", line 101, in getCheckers
    newcheckers = self.db.getCheckers(self.dirty)
  File "/usr/lib/python2.4/site-packages/nav/statemon/db.py", line 245, in getCheckers
    newChecker = checker(service, status=up)
TypeError: __init__() got an unexpected keyword argument 'status'

[Morten Brekkevold (mbrekkevold)]

It appears that this is caused by two separate problems.

The DummyChecker was fixed in NAV 3.5.1, so that it now can actually be imported by servicemon. The DummyChecker __init__ method, however, is not up to spec and does not accept keyword arguments, so it raises an exception.

Unfortunately, servicemon does not have any robustness against checker failure, it seems, so it crashes completely when the exception is raised.

So, in conclusion: Servicemon should catch exceptions from checker instantiation (in nav/statemon/db.py), log a big fat informative error when it happens, and then continue as normal. The DummyChecker should accept the correct set of arguments (adding a **kwargs argument is probably enough).

Updating the report title to be more accurate.

[Morten Brekkevold (mbrekkevold)]

Oh well, the PostgreSQLChecker was fixed in the same manner as the DummyChecker, and apparently has the same problem that causes it to raise an exception when the __init__ method is called.

I'm reserving this bug report for robustness in the daemon code itself, and reporting the plugins bugs as separate issues. See bug 341617 and bug 341619.

[Morten Brekkevold (mbrekkevold)]

Fix committed and pushed to series/3.5.x: http://metanav.uninett.no/hg/series/3.5.x/rev/372f9eeb6efe