Node event log contains thousands of invalid password entries

[Jason Hobbs]

I have an IPMI system with an incorrect password. The node event log has about 4000 entries of:

WARNING 10 minutes ago Failed to query node's BMC — Node could not be queried node-fd29da40-2fb7-11e4-a1a9-00163e11c884 (nasty-library.local) ipmi failed with return code 2: Invalid password
WARNING 15 minutes ago Failed to query node's BMC — Node could not be queried node-fd29da40-2fb7-11e4-a1a9-00163e11c884 (nasty-library.local) ipmi failed with return code 2: Invalid password
WARNING 20 minutes ago Failed to query node's BMC — Node could not be queried node-fd29da40-2fb7-11e4-a1a9-00163e11c884 (nasty-library.local) ipmi failed with return code 2: Invalid password

This was with MAAS 1.7.0~beta2+bzr2993-0ubuntu1~ppa1

[Christian Reis (kiko)]

How long did it take for those 4000 entries to be generated? Would the fact that this will appear in the node page have avoided this going so long undetected?

[Jason Hobbs (jason-hobbs)]

This WAS the log on the node page - the event log. The entries were added every 5 minutes, so about 13 hours.

[Julian Edwards (julian-edwards)]

I'm raising this to critical because it's completely obnoxious to users.

We need either:
 * something like syslog does where it detects duplicate messages and summarises them in a single entry every now and then,
 * totally ignore the message if it's a dupe of the most recent one

[Christian Reis (kiko)]

We ran out of time and 1.7 needs to RC. This is just one log entry every 5 minutes, so not really a DOS situation.

And finally, this only makes sense if we have a log entry that indicates that the BMC information has been fixed and verified. We can do this as part of the BMC validation work we have on the roadmap.

[Andres Rodriguez (andreserl)]

This is usability issue that needs to be fixed as part of 1.7.X.

[Steveartifex (stephew)]

Is there a solution to that can resolve the 'Failed to query node's BMC'?