process-hwdb-submissions.py ignores the '-q' flag

[Steve McInerney]

cronscripts/process-hwdb-submissions.py -m 50000 -q
run on a nightly cron job, is generating ~ 1.5Mb of WARNING's output.

eg:
2009-05-25 17:30:42 WARNING Parsing submission 3fe1b364b90bb9e31cc6e4bebb8a77b4: Submission contains unprocessed <context> data.
2009-05-25 17:30:42 WARNING Parsing submission 3fe1b364b90bb9e31cc6e4bebb8a77b4: Submission does not provide property system.kernel.version for /org/freedesktop/Hal/devices/computer.
2009-05-25 17:30:44 WARNING Parsing submission c976e9266ba19603a47d85b565dc19a2: Submission contains unprocessed <context> data.
2009-05-25 17:30:44 WARNING Parsing submission c976e9266ba19603a47d85b565dc19a2: Submission does not provide property system.kernel.version for /org/freedesktop/Hal/devices/computer.
2009-05-25 17:30:47 ERROR Parsing submission 05c9bb2b7eff668857f809464b018bd2: not well-formed (invalid token): line 4793, column 1
2009-05-25 17:30:48 WARNING Parsing submission 57134621e1ba557055d1d6b6fa8ebd38: Submission contains unprocessed <context> data.

Would expect that with a -q to only see ERROR's.

Additionally, the ERROR messages that are generated - similar to that above - appear to be WARNING status.
ie. There is no sysadmin/losa action required to be performed with those ERRORs. ergo, they're warnings only, not errors.

Alternately, if this information _is_ required, we can drop into a logfile and rotate daily, syncing to devpad. Perhaps a 1 week retention only?
Alternately 2, we can send all stdout to /dev/null ?

[Diogo Matsubara (matsubara)]

The last one I could find was from June 30th. Is this still a problem?

[Steve McInerney (spm)]

Yes, still a problem.
We've put a work around in place to cut down on the cron spam we get by outputting all data into logfiles that are daily rotated and thrown away after 5 iterations. And I mean all output - the current output from this task is ALL on stderr which is a big no-no.

So:
1. the WARNINGS shouldn't be outputted with a -q flag
2. the ERRORS are warnings, not actual errors - as mentioned initially, an error is something that requires sysadmin intervention, else it's just informational and hence is cron spam.
3. the output is to stderr - should be stdout; except for *real* errors. ie script fatal error etc

else it all gets treated as cron spam and will be ignored. insert links to 'boy who cries wolf' etc :-)

[Stuart Bishop (stub)]

I've successfully argued on similar issues that if no action is to be taken, it should be INFO or lower. This leaves WARNING, ERROR and FATAL for actionable notifications.

Note that we have DEBUG2 -> DEBUG9 available now if DEBUG and INFO is becoming too cluttered for diagnostic purposes.

[Robert Collins (lifeless)]

@abel - ping - is this in progress ?

[Colin Watson (cjwatson)]

We've removed the hardware database, so this is no longer applicable.