DateTime default timezone wrong in some locations

[Stuart Bishop]

DateTime currently relies on just time.tzname to determine the default timezone. Unfortunatly, these strings are not unique. This is the case in Australia, and possibly other areas in the world, on many Unix based systems:

  >>> time.tzname
  ('EST', 'EST')
  >>> time.timezone
  -36000
  >>> time.altzone
  -39600

A possible solution would be to insert a check. After retrieving the timezone strings, check time.timezone to see if the offset in seconds matches what is expected. If not, convert the offset to a GMT+XX string and use that as the local timezone. This solution would also future proof the DateTime library to some extent, as well as making Australian's happier. Current workarounds do not handle daylight savings issues.

[Lennart Regebro (regebro-gmail)]

Status: Pending => Rejected

There are unique international standards for what to call your timezones. If Zope doesn't support them, then that is a bug with Zope. If your unix system doesn't support, then that is also a bug, but not with Zope. :)

[Stuart Bishop (stub)]

> There are unique international standards for what to call your timezones.
> If Zope doesn't support them, then that is a bug with Zope. If your unix
> system doesn't support, then that is also a bug, but not with Zope. :)

If there actually is an ISO or IEEE standard on this, Countries and other regions do not follow it. The operating systems implement the realities in the countries. This behaviour is common to Solaris 2.8, Redhat 7.2 and OS X 10.2 and probably
every other Unix that uses zoneinfo. I think this issue needs to remain open.

[Carl Brewer (bleve)]

Did anything ever get done about this? I've a zope 2.9.3 server that seems to exhibit a broken timezone problem, and my timezone is Australia/Melbourne - but it shows as EST

[Stuart Bishop (stub)]

This issue has never been dealt with AFIK, and the original reason for rejecting the bug is incorrect - Zope2 was assuming that timezone abbreviations are unique which is incorrect, and there are no standards that attempt to make it so. Timezones are political decisions and are not determined by anyone in the IT industry.

That said, EST is the correct abbreviation for both Eastern Standard Time and Eastern Summer time in Australia according to the database used by almost all Unix systems.

[Alexander Limi (limi)]

This is a Zope bug, not an OS bug. All our Australian friends have this problem all the time. Could we have this re-opened, please?

[Martijn Pieters (mjpieters)]

I say this is a valid bug. DateTime should not use time.tzname to find timezone offsets. time.timezone and time.altzone are much better indicators. We are not going to fix unix zoneinfo databases, which currently return ('EST', 'EST') for eastern Australia, thus making time.tzname effectively useless. This is a problem that should be fixed in Zope.

[James Purser (purserj)]

Just confirming Martijn's views on the matter.

I have just spent the past two hours trying to get this fixed.

After discussion with MJ|10 and betabug on #zope, MJ|10 suggested the following solution as a work around:

Replace the try/except surround the following snippet:

        # Get the name of the current time zone depending
        # on DST.
        _localzone = _cache._zmap[tzname[isDST].lower()]

with if False:/else:

That will ensure that tzname is never looked at, and that the timezone information is pulled correctly. I can confirm that it is working for Australian Daylight Savings time on two installations.

[James Purser (purserj)]

Simple patch to enact the above

[Andreas Jung (ajung)]

This patch is not acceptable. The "if False" clause makes no sense (it's dead code) and there are no tests for the patch.

[Martijn Pieters (mjpieters)]

Andreas, it was not meant as a definitive patch, but as a workaround. Of course the 'if False:' is dead code, it disables the code that uses time.tzname, which is what is causing the problem for certain timezones.

Yes, a proper patch needs written, and the tests need updating. I doubt James Purser will write it though, he was just one of the victims of the bug.

[Lennart Regebro (regebro-gmail)]

None of the suggested fixes work very well, as the best case is to figure out what your current time zone offset is. What you want to know is not only the current offset, but also when DST goes on and off. The best way of doing that, on unices at least, is to wrap /etc/localrime, which keeps this information. On Windows I don't know.

dateutil has an implementation of this.
More information on the various time zone issues I have discovered during a two day time head on attack on time zone issues is here:
http://regebro.wordpress.com/2007/12/18/python-and-time-zones-fighting-the-beast/

The best path forward here would be to merge pytz and dateutil.tz. I have no idea of that is feasible.

[Lennart Regebro (regebro-gmail)]

In fact, as a comment on the blog Stuart shows how to do the same with pytz, so no merging needed, just a bit of wrapping code and testing on Windows.

[James Purser (purserj)]

Just out of curiosity, what is happening with this bug?

[Lennart Regebro (regebro-gmail)]

Trying out a fix for this, that avoids the problem by not looking at the Three Letter Abbreviation of the timezone, but instead creates a pytz timezone from the TZ variable, defaulting to /etc/timezone .

http://svn.zope.org/Zope/branches/regebro-timezone-142148/

Major difference: Both DateTime._localzone0 and DateTime._localzone1 will typically now be "US/Eastern". This seems to work well, though.
Opinions are welcome.

[Andreas Jung (ajung)]

If there are no objections, please merge the fix.

[Lennart Regebro (regebro-gmail)]

OK, I'm just going to make it work on Windows as well...

[Lennart Regebro (regebro-gmail)]

There was a miscommunication between me and Stuart Bishop. The branch I have made didn't do what I thought it did, and it won't work on Mac OS X (and getting it to work under Windows seems tricky). A fix is on hold, again.

[Lennart Regebro (regebro-gmail)]

Please, all that has this problem, go here and follow the instructions.
http://regebro.wordpress.com/2008/05/09/please-run-a-test-for-me-it-takes-only-minutes/

I want to see if, and how I can reliably figure out the timezone.

[Lennart Regebro (regebro-gmail)]

My conclusion of those tests is that I can't reliably figure out what time zone the computer is using. This pretty much means that the bug is unfixable as long as we use pytz. I suggest we replace the the current timezone implementation for DateTime with one using dateutil.tz, which can fix the problem thanks to it's tzlocal and tzwin timezone implementations.

More here: http://regebro.wordpress.com/2008/05/13/thanks-for-the-testing-help-conclusions/

[Chris Hamono (chris-hamono)]

CST Bug

This problem has been a long running Issue. With the help of Dixond on the plone irc chat we have resolved the issue that was effecting us in Adelaide Australia.

None of the other fixes noted elsewhere worked

The issue stems form both CST and EST being hard coded into the datetime module.

I do not know why that was done, perhaps the attached patch may break some installs but for us it meant we now get the correct date being reported when we use plone

with the patch installed our installation now reports the correct time. I havent checked other timezones and we are currently in DST. I haven't yet checked normal time.

We did not require the addition of the TZ environment variable as noted elsewhere. It did not work anyway.

DC

[Dylan Jay (t-launchpad-dylanjay-com)]

Chris your fix is no better than the previous ones. It will break american EST. This is a really hard problem to solve for all the reasons above.

The best work around for now is the one I mention in https://bugs.launchpad.net/zope2/+bug/501805.
We use this for all our australian clients. We're an Australian Plone integrator.

That is override the TZ env var on an instance by instance basis. (the code below is up to date with the latest NSW timezone rules btw).

[instance]
recipe = plone.recipe.zope2instance
...
environment-vars =
    TZ AEST-10AEDT-11,M10.1.0/02:00:00,M4.1.0/03:00:00

The only caveat is that there seems to be a bug in zope that to make this work you have to run

bin/instance console

instead of runzope.py.

but I'm told "instance console" is now the recommended of running plone in production anyway so it's not that much of an issue.

It would sitll be nice to have a zope (and python) that "just works" for AEST :)

[Chris Hamono (chris-hamono)]

Sorry but I am not particularly knowledgeable about zope/plone or even python. I cannot find bin/instance? Perhaps it is created in the buildout process.

We don't use buildouts, not my call that's just the way it is.
so I cant use...
[instance]
recipe = plone.recipe.zope2instance
...
environment-vars =
    TZ AEST-10AEDT-11,M10.1.0/02:00:00,M4.1.0/03:00:00

I tried setting the TZ in zope.conf but it did not take. I assume that is because of the environment not being passed to the child environment as mentioned in other posts about this bug.

When I tried to use TZ I used zopectl fg it didnt work. I think zopectl is the actual script you are talking about. I can't see any documentation mentioning other ways to start Zope. and as stated I used zopectl and got no joy when setting the TZ variable.

The patch I submitted makes it work for Australian and presumably other users. The fact that the timezone was hardcoded into DateTime only shows the bug is more insidious than a simple 'typo'.

Why was it hardcoded into DateTime? perhaps because older servers had the timezone configured to CST instead of US/Central!

Isn't it better that they configure their servers to the correct timezone instead of using an old non portable syntax?

I stick by the fix, cst:US/Central in DateTime appears to be a kludge it is non conformant with other entries in there.

Needless to say it works for us and it may also be the impetus for the maintainers to actually fix the issue, which has remained open for 8 years. It certainly points towards where the real culprit lies.

Your fix may work well for you, but for us it did not work, probably because we don't do it the "correct way".

I only bothered posting the fix here for those miserable few like us that can't use other fixes.

DC

[Lennart Regebro (regebro-gmail)]

Chris: bin/instance is the script you use to start the zope instance with. It may have different names. runzope is common,

There *is no way* to reliably determine what timezone you are in. DateTime can only do a best guess. And that means that if you are in CST, it will guess that this means Central Standard Time, as this is what it usually means. If it guesses incorrectly, the solution is to use a specific TZ variable as Dylans solution above.

[Colin Watson (cjwatson)]

The datetime project on Launchpad has been archived at the request of the Zope developers (see https://answers.launchpad.net/launchpad/+question/683589 and https://answers.launchpad.net/launchpad/+question/685285). If this bug is still relevant, please refile it at https://github.com/zopefoundation/datetime.