squid should default to more human friendly timestamps

Bug #1780341 reported by James Troup on 2018-07-05
This bug affects 1 person
Affects Status Importance Assigned to Milestone
squid (Ubuntu)
Robie Basak

Bug Description

squid defaults to unix timestamps (i.e. seconds since 1970-01-01) in its log files. This is very unusual and very unfriendly to human admins who have to read log files. If upstream can't be convinced to change this then, in the interests of Ubuntu having good opinionated defaults, I think we should make the default timestamp format in Ubuntu be saner.

James Troup (elmo) wrote :

On a fresh 18.04 container after installing squid:

| root@temp-delete-me:~# head -n1 /var/log/squid/access.log
| 1530832866.793 1 TAG_NONE/400 3918 GET / - HIER_NONE/- text/html

Amos Jeffries (yadi) wrote :

Squid logs are a customized timestamp format based on UNIX times but also including millisecond (and optionally nanosecond) resolution not available in strictly UNIX timestamps.

This format and resolution is necessary for accuracy of log information - which is intended more for machine interpretation via log analysis tools, not human.

Human timestamps are far to inaccurate and malleable for this type of data recording. They also vary depending on where in the world the reader is located, independent of where the proxy is located. Human calendar systems are a legal fiction, not a technical or scientific measure - the government of any country can redefine them at any time (and some do so even today). So the time of the recording matters as much as the value recorded in human systems.
 Mapping between the fixed and deterministic Squid log values and a readers local human system is the purpose of log analysis tools and far more easily done from UNIX-like timestamps than other types.

For admin who wish to read the logs manually and provide inaccurate data to their analysis tools other formats are configurable.

Robie Basak (racb) wrote :

I appreciate the pain of not being able to immediately read squid's logs. I've suffered this myself.

Amos, it occurs to me that most of your argument applies equally to every service running on an Ubuntu server, not just to squid. I think a reasonable counterargument is that Ubuntu users should expect the same principle to be applied to all packages. Therefore I think it makes sense for distribution packaging to change upstream defaults so that distribution users see consistency across all packages.

This may temporarily break log analysers, but I think that's something that should be fixed in the log analysers to be able to cope with this new form of output - it doesn't make sense to lock in the log format forever if we think an improvement can be made, just for that. If we do conclude that we want to make the change, then the right time is as early as possible to give plenty of time to catch issues before the next Ubuntu LTS release.

Amos, in mitigation, how would you feel about adding a human readable timestamp like other system log files but retaining the current numeric timestamp? Is this something that would be acceptable to do upstream? Then you'll get the accuracy/unambiguity of numeric timestamps retained but we'll also get the non-human-readability issue fixed.

> Human calendar systems are a legal fiction, not a technical or scientific measure - the government of any country can redefine them at any time (and some do so even today).

So is a Unix timestamp. Admittedly less so, but there are complications wrt. leap seconds, Google's "smearing" and so forth. Time is complicated :)

Amos Jeffries (yadi) wrote :

Squid already several contains such other formats. All they take is configuration. The default one is intended for fast machine interpretation, since that is the majority use-case for all Squid installations.

Robie Basak (racb) on 2018-11-20
Changed in squid (Ubuntu):
status: New → In Progress
assignee: nobody → Robie Basak (racb)
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers