Under NAV 4.3.0, we have observed on some installations a crash in the ipdevpoll statuscheck job, as ipdevpoll prepares to dispatch a chassis event. This means the job stops, and the event is not (fully) posted.
Traceback looks like this:
2015-08-19 00:04:21,910 [ERROR jobs.jobhandler] [statuscheck example-sw.example.org] Caught exception during cleanup. Last manager = EntityManager(<class 'nav.ipdevpoll.shadows.entity.NetboxEntity'>, 'ContainerR
epository'(...))
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/nav/ipdevpoll/jobs.py", line 390, in cleanup_containers_after_save
manager.cleanup()
File "/usr/lib/python2.7/dist-packages/nav/ipdevpoll/shadows/entity.py", line 102, in cleanup
self._verify_stack_degradation(to_set_missing)
File "/usr/lib/python2.7/dist-packages/nav/ipdevpoll/shadows/entity.py", line 150, in _verify_stack_degradation
chassis_event.start(chass.device, chass.netbox, chass.id).save()
File "/usr/lib/python2.7/dist-packages/nav/models/event.py", line 304, in save
self.varmap = self.varmap
File "/usr/lib/python2.7/dist-packages/nav/models/event.py", line 94, in __set__
self._update_variables(obj, vardict)
File "/usr/lib/python2.7/dist-packages/nav/models/event.py", line 138, in _update_variables
variable.save()
File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 463, in save
self.save_base(using=using, force_insert=force_insert, force_update=force_update)
File "/usr/lib/python2.7/dist-packages/django/db/models/base.py", line 551, in save_base
result = manager._insert([self], fields=fields, return_id=update_pk, using=using, raw=raw)
File "/usr/lib/python2.7/dist-packages/django/db/models/manager.py", line 203, in _insert
return insert_query(self.model, objs, fields, **kwargs)
File "/usr/lib/python2.7/dist-packages/django/db/models/query.py", line 1593, in insert_query
return query.get_compiler(using=using).execute_sql(return_id)
File "/usr/lib/python2.7/dist-packages/django/db/models/sql/compiler.py", line 912, in execute_sql
cursor.execute(sql, params)
File "/usr/lib/python2.7/dist-packages/django/db/backends/postgresql_psycopg2/base.py", line 52, in execute
return self.cursor.execute(query, args)
IntegrityError: insert or update on table "eventqvar" violates foreign key constraint "eventqvar_eventqid_fkey"
DETAIL: Key (eventqid)=(1512169) is not present in table "eventq".
The problem seems to be that the event data is not stored inside a transaction, but using autocommit. The event record is being culled from the eventq table by eventengine before ipdevpoll has a chance to completely store all the associated eventqvar records.
The event generating code should be surrounded by a transactional decorator.