[6.0,trunk] pytz zoneinfo missing from windows packaging - timezone conversion not working

[philu]

OpenERP server and web client 6.0.1 running on Ubuntu server (32-bit) 10.10
Postgres 8.4
Windows GTK client and web browser (Google Chrome) on WIndows 7 (64-bit)

(Server and web client were set up as per http://powerphil.wordpress.com/2011/01/28/how-to-install-and-configure-an-openerp-6-0-1-server-and-web-server-on-an-ubuntu-10-10-server/).

Problem is that the times shown in the GTK client are shown at GMT (unless the server's TZ environment variable is set), i.e. they ignore the user's Timezone preference.
However, the web client works correctly.

More details:
Ubuntu server's time is correctly set to GMT. For example, if the time in London is 1am, the server's time also shows 1am (by doing a "date" on the command line, and the TZ environment variable is not set to anything).

Both openerp-server and openerp-web processes are running without any TZ setting.

Then log into OpenERP using a web browser; set the User's preference to Australia/Sydney; create a new Phone Call, Outbound; the time correctly shows 12 noon (GMT + 11 hours, Sydney currently being on summer time).

Do the same in the GTK client; the time shows 1am - wrong; it should be 12 noon.

---------

More information:

So try to work around the problem: in the .bashrc file of the openerp user that runs the openerp-server process, set the TZ, viz:
TZ='AEST-10AEDT-11,M10.5.0,M3.5.0'
export TZ

Restart the service; log into the GTK client; create a new phone call; the time now shows correctly (for the wrong reasons, admittedly).
Go to the web browser and do the same thing; the time now shows 19 hours ahead of what it should be (on the next day!). Wrong.

So it looks like the web client is taking timezone time into account, in addition to whatever the openerp-server is doing, and it should be the other way around: the openerp-server should be calculating the correct time, and the web server should just be presenting it.

[philu (philu)]

This is NOT a duplicate of bug #701407 because:

- there IS a full list of timezones in the User's preferences.
- I also installed the latest version of pytz-2010o; this does not affect the problem.

The server DOES observe the TZ environment variable and DOES calculate the time, but the problem is that the web-client ALSO calculates time, but using the time from the server, so that if you are 10 hours ahead of GMT, your web client is 20 hours ahead!

[Azazahmed Saiyed (OpenERP) (saz-openerp)]

Hello,

Would you please tell me the way, how you got the list of timezones in windows installer of stable 6.0.

As I am not able to get the list in gtk-client for windows. I have checked the same thing in Linux and it works fine for both clients.

Thanks.

[philu (philu)]

I am using Ubuntu 10.10 server (x86) with Python 2.6 and Postgres 8.4. If you want the full configuration steps, you can find them here:
http://powerphil.wordpress.com/2011/01/28/how-to-install-and-configure-an-openerp-6-0-1-server-and-web-server-on-an-ubuntu-10-10-server/

Here are the version numbers of the packages I have installed:

Package Version
----------- ----------
python 2.6
python-psycopg2 2.2.1
python-reportlab 2.4-3
python-egenix-mxdatetime
python-pychart 1.39-7
python-pydot 1.0.2-1
python-lxml 2.2.6.1
python-vobject 0.8.1c-3
python-setuptools 0.6.14-3
PyYaml 3.09-py2.6
python-mako 0.3.4-2
python-yaml 3.09-4
python-dev 2.6.6
python-tz 2010o

I then ran the OpenERP GTK Client and configured OpenERP modules:

1. Created a new database; the first time I created a database, I created it WITH Demonstration Data; I don't know if this has any significance. Later, I also created a database WITHOUT Demonstration Data. Both databases had a list of available Timezones available in the User's Preferences.

2. Selected "Extended" interface.

3. Configured Company Information.

4. Installed these Applications:
- CRM
- Warehouse Management
- Invoicing
- Purchase Management
- Point of Sale
- Sales Management
- Manufacturing
- Accounting and Finance
- Human Resources
- Advanced Reporting

5. Configured Reporting Tools:
- OpenOffice Report Designer
- QueryBuilder

6. Configured Human Resources:
- Holidays / Leave Management
- Employee's Contracts
- Payroll Accounting
- Expenses
- Timesheets
- Attendances
- Payroll

7. Configured Chart of Accounts
- Generic Chart of Accounts

8. Configured CRM:
- Claims
- Opportunity to Quotation

9. Configured Purchases:
- Purchase Requisition
- Purchase Analytic Plans

10. Configured Sales Management:
- Delivery Costs
- Sales Order Layout Improvement
- Sales Order Dates
- Invoicing Journals
- Margins in Sales Orders

11. Configured Manufacturing:
- Manufacturing Operations
- Repairs

12. Configured Report Designer

13. Configured Accounting Application:
- Voucher Management
- Suppliers Payment Management
- Anglo-Saxon Accounting
- Followups Management
- Multiple Analytic Plans

14. Configured Sales Order Logistics:
- Direct Delivery
- Invoice Based on Deliveries
- Picking List and Delivery Order

[Olivier Dony (Odoo) (odo-openerp)]

@philu:
Thanks for the detailed explanation! You're right, something wrong is happening with timezone conversions on Windows. However from my tests (win XP), it seems to be the same issue for GTK and Web client both, as soon as they are running on Windows (for the web client, keep in mind that what matters is where the Web Client Service is running, not the browser!).

The current logic is the following, which is close to what you suggested:
1. the server does not do any time conversion or offsetting, it treat all times and dates as if they are in its own timezone (i.e with no timezone info set), and stores them as such in the database
2. the clients (web and gtk both) are in charge of doing the conversion on-the-fly, displaying the local date/time info to the user (according to the timezone chosen in the user preferences), and sending/reading the values to/from the server in the server timezone
3. in order to do the conversion, clients need to know 2 timezones: the one of the user (set in user preferences) and the one of the server (returned by the server when you ask it)
4. if the client timezone is not set, all date/time info is displayed with no conversion, i.e. in the same TZ as the server
5. if the server cannot detect its own timezone to broadcast it (see tools.misc.get_server_timezone()), it defaults to "UTC"

I have tested with a server on Ubuntu which is in CET timezone and a user which is in the "Australia/Brisbane" timezone. When the clients are running on Ubuntu it works fine and the conversions happens as described above. But when the clients are running on Windows no conversion occurs and it displays the datetime info in the server timezone, just as if no timezone was set.
The reason, in my case, is really that the timezone info from pytz was not properly installed. By enabling the Web client error log file, I noticed the following typical traceback in the log (I expect GTK client to have the same ones):
[22/Feb/2011:12:33:03] Error in timezone formatting:
Traceback (most recent call last):
  File "openobject\i18n\format.pyo", line 153, in format_datetime
  File "openobject\i18n\format.pyo", line 90, in tz_convert
  File "pytz\__init__.pyo", line 157, in timezone
UnknownTimeZoneError: 'Australia/Brisbane'

After correcting it and making sure that the "zoneinfo" folder was really present in the pytz folder of my python libraries, the problem was fixed and *both* GTK and Web behaved as they do on Ubuntu.

Therefore the root issue for your bug can only be one thing: the conversion is failing, either because pytz's zoneinfo data is really not properly reachable by the pytz module, or for another similar reason.
To pinpoint the issue you can enable the error log file in your Web Client Service running on windows (i.e by uncommenting the "log.error_file"' line in the doc/openerp-web.cfg, choosing a valid path, and restarting the service) to see the specific exception trace.

In the mean time, I will at least confirm this issue for the reason that the pytz zoneinfo data is missing *again* in all windows packages

[Olivier Dony (Odoo) (odo-openerp)]

Confirming it affects both web and GTK clients.

[philu (philu)]

I stopped the openerp-web service, checked the log files; they are:

# logging
log.access_file = "/var/log/openerp-web/access.log"
log.error_file = "/var/log/openerp-web/error.log"

Restarted openerp-web; logged in on the web browser, went to Sales / Phones / Outbound / New.
The time is incorrect: it shows: "05/03/2011 09:26:07" - wrong; it should show "04/03/2011 17:26:07".
Went into the GTK client on the Windows 7 PC; it showed "04/03/2011 17:35:13" - correct.

Logged into the server and ran the date and echo $TZ commands:

/home/philu> date
Fri Mar 4 17:36:28 AEDT 2011
/home/philu> echo $TZ
AEST-10AEDT-11,M10.5.0,M3.5.0

Time is shown correctly.

Double-checked the user preferences: Australia/Sydney - correct.
The error log file didn't show any errors at all; all it showed was:

[03/Mar/2011:22:20:51] INFO Loading module 'openerp'
[03/Mar/2011:22:20:52] INFO Loading module 'view_calendar'
[03/Mar/2011:22:20:53] INFO Loading module 'view_graph'
[03/Mar/2011:22:20:53] INFO Loading module 'view_diagram'

-----------

Dug back through the openerp-server log file; found this - the interesting bit is the fourth-last line:

[2011-03-04 17:26:07,676][?] DEBUG_RPC:service:'object'
[2011-03-04 17:26:07,676][?] DEBUG_RPC:method:'execute'
[2011-03-04 17:26:07,677][?] DEBUG_RPC:params:['openerp601',
[2011-03-04 17:26:07,677][?] DEBUG_RPC:params: 1,
[2011-03-04 17:26:07,678][?] DEBUG_RPC:params: '*',
[2011-03-04 17:26:07,678][?] DEBUG_RPC:params: 'crm.phonecall',
[2011-03-04 17:26:07,678][?] DEBUG_RPC:params: 'default_get',
[2011-03-04 17:26:07,678][?] DEBUG_RPC:params: ['user_id',
[2011-03-04 17:26:07,678][?] DEBUG_RPC:params: 'description',
[2011-03-04 17:26:07,678][?] DEBUG_RPC:params: 'categ_id',
[2011-03-04 17:26:07,678][?] DEBUG_RPC:params: 'partner_address_id',
[2011-03-04 17:26:07,678][?] DEBUG_RPC:params: 'section_id',
[2011-03-04 17:26:07,679][?] DEBUG_RPC:params: 'priority',
[2011-03-04 17:26:07,679][?] DEBUG_RPC:params: 'duration',
[2011-03-04 17:26:07,679][?] DEBUG_RPC:params: 'state',
[2011-03-04 17:26:07,679][?] DEBUG_RPC:params: 'partner_mobile',
[2011-03-04 17:26:07,679][?] DEBUG_RPC:params: 'date',
[2011-03-04 17:26:07,679][?] DEBUG_RPC:params: 'opportunity_id',
[2011-03-04 17:26:07,679][?] DEBUG_RPC:params: 'partner_id',
[2011-03-04 17:26:07,679][?] DEBUG_RPC:params: 'partner_phone',
[2011-03-04 17:26:07,680][?] DEBUG_RPC:params: 'name'],
[2011-03-04 17:26:07,680][?] DEBUG_RPC:params: {'_terp_view_name': u'Outbound',
[2011-03-04 17:26:07,680][?] DEBUG_RPC:params: 'active_id': 245,
[2011-03-04 17:26:07,680][?] DEBUG_RPC:params: 'active_ids': [245],
[2011-03-04 17:26:07,680][?] DEBUG_RPC:params: 'active_model': 'ir.ui.menu',
[2011-03-04 17:26:07,680][?] DEBUG_RPC:params: 'bin_size': True,
[2011-03-04 17:26:07,680][?] DEBUG_RPC:params: 'client': 'web',
[2011-03-04 17:26:07,680][?] DEBUG_RPC:params: 'default_categ_id': 13,
[2011-03-04 17:26:07,680][?] DEBUG_RPC:params: 'default_state': 'open',
[2011-03-04 17:26:07,681][?] DEBUG_RPC:params: 'lang': u'en_GB',
[2011-03-04 17:26:07,681][?] DEBUG_RPC:params: 'search_default_current': 1,
[2011-03-04 17:26:07,681][?] DEBUG_RPC:params: 'search_default_section_id': False,
[...

[GEM (nimp3)]

Hi,
Warning, same missing folder zoneinfo with server in server\library.zip, but I don't know if zoneinfo is necessary for the server. But more than missing folder zoneinfo, the file reference.py is missing also.
If necessary, to add reference.py, add pytz in the list of required package in server\setup.py line 196 :
- "uuid",
+ "uuid", "pytz"
If necessary zoneinfo in server, add the code after in server\setup.py as in client

to add complete pytz library (with folder zoneinfo), like in client\setup.nsi in branch 5.0, add at the end of the script client\setup.py :

if has_py2exe:
    # Sometime between pytz-2008a and pytz-2008i common_timezones started to
    # include only names of zones with a corresponding data file in zoneinfo.
    # pytz installs the zoneinfo directory tree in the same directory
    # as the pytz/__init__.py file. These data files are loaded using
    # pkg_resources.resource_stream. py2exe does not copy this to library.zip so
    # resource_stream can't find the files and common_timezones is empty when
    # read in the py2exe executable.
    # This manually copies zoneinfo into the zip. See also
    # http://code.google.com/p/googletransitdatafeed/issues/detail?id=121
    import pytz
    import zipfile
    import tempfile
    import shutil
    # Make sure the layout of pytz hasn't changed
    assert (pytz.__file__.endswith('__init__.pyc') or
          pytz.__file__.endswith('__init__.py')), pytz.__file__

    temp_dir = None
    pytz_dir = os.path.dirname(pytz.__file__)
    zoneinfo_dir = os.path.join(pytz_dir, 'zoneinfo')
    if not os.path.exists(zoneinfo_dir):
        egg = os.path.dirname(pytz_dir)

        if zipfile.is_zipfile(egg):
            temp_dir = tempfile.mkdtemp()
            zoneinfo_dir = os.path.join(temp_dir, 'pytz', 'zoneinfo')
            os.makedirs(zoneinfo_dir)

            archive = zipfile.ZipFile(egg)
            for filename in archive.namelist():
                if filename.startswith('pytz/zoneinfo/'):
                    file_path = os.path.join(temp_dir, filename)
                    destination = file_path.replace('/', os.sep)
                    if not file_path.endswith('/'):
                        try:
                            os.makedirs(os.path.dirname(destination))
                        except os.error:
                            pass
                        fp = file(destination, 'w')
                        fp.write(archive.read(filename))
                        fp.close()
            archive.close()

    # '..\\Lib\\pytz\\__init__.py' -> '..\\Lib'
    disk_basedir = os.path.dirname(os.path.dirname(zoneinfo_dir))
    zipfile_path = os.path.join(options['py2exe']['dist_dir'], 'library.zip')
    z = zipfile.ZipFile(zipfile_path, 'a')
    for absdir, directories, filenames in os.walk(zoneinfo_dir):
        zip_dir = absdir[len(disk_basedir):]
        for f in filenames:
            z.write(os.path.join(absdir, f), os.path.join(zip_dir, f))
    z.close()

    if temp_dir is not None:
        shutil.rmtree(temp_dir)

Bye

[Stephane Wirtel (OpenERP) (stephane-openerp)]

There are some news for this bug ?

[Vo Minh Thu (thu)]

For the record:
- The zoneinfo files are correctly copied in 6.0.2 and trunk.
- The file reference.py in pytz is just a reference implementation, and is not used. So, no problem if it is not available under Windows.

[Vo Minh Thu (thu)]

We couldn't reproduce the problem here using Ubuntu.

We have checked here the situation by trying different cases:

- server is running with the system settings, web client too
- server is running with TZ='AEST-10AEDT-11,M10.5.0,M3.5.0', web client left as-is
- server and web client are running with TZ='AEST-10AEDT-11,M10.5.0,M3.5.0'

In all cases, both the GTK client and the Browser show the correct values.

Keep in mind that using something like TZ='AEST-10AEDT-11,M10.5.0,M3.5.0' is just for testing purpose: the server stores date and time in its current timezone.

The previous comment (#9) was about the server. Probably the GTK client installer still doesn't correctly include the zoneinfo data.

[Vo Minh Thu (thu)]

The zoneinfo data are now correctly copied in the Windows library.zip file in both trunk (at revision 1864) and 6.0 (at revision 1831) branches.

The time is correctly is displayed in the GTK client running under Windows too; I will close this report.

[philu (philu)]

Problem has not been fixed and exists exactly as originally reported; the times in the web browser and the GTK client still do not match.
Changed the status back from "Fix Released" to "In progress".

I tested all this with:
OpenERP server and web client 6.0.2 running on Ubuntu server (32-bit) 10.10
Postgres 8.4
Windows GTK client 6.0.2 and web browser (Google Chrome) on WIndows 7 (64-bit)

[philu (philu)]

Here is exactly how to test the problem:

1. Install OpenERP 6.0.2 server and web-client on Ubuntu 10.10.

2. Install OpenERP 6.0.2 GTK client on Windows 7 64-bit.

3. Log in to OpenERP using the GTK client. Change the User Preference Timezone to "Australia/Sydney".

4. Log in to OpenERP using a web browser on the Windows PC.
Create a new outbound phone call. It shows the correct time.
E.g. if the Ubuntu's server's time is April 21 03:20:00 GMT, the phone call time is 04/21/2011 13:20:00 which is correct.

4. Go to Sales / Phone Calls / Outbound / New. Note the time and date. It is wrong.
E.g. if the Ubuntu's server's time is April 21 03:20:00 GMT, then the GTK client says the time of the call is 04/20/2011 20:20:00 which is wrong. It is out by 17 hours. It should be 04/21/2011 13:20:00, the same as the shown in the web browser.

[GEM (nimp3)]

Hi,
another problem I have :
when I build the windows installer server with python 2.6, folder pytz\zoneinfo is not added in server\dist\library.zip\pytz.
works with python 2.5 (tested with same sources).
bye

[philu (philu)]

Did you run the GTK client on Ubuntu or on Windows?

Regards,

Phil

On Tue, Apr 5, 2011 at 11:30 PM, Vo Minh Thu (OpenERP) <
<email address hidden>> wrote:

> We couldn't reproduce the problem here using Ubuntu.
>
> We have checked here the situation by trying different cases:
>
> - server is running with the system settings, web client too
> - server is running with TZ='AEST-10AEDT-11,M10.5.0,M3.5.0', web client
> left as-is
> - server and web client are running with TZ='AEST-10AEDT-11,M10.5.0,M3.5.0'
>
> In all cases, both the GTK client and the Browser show the correct
> values.
>
> Keep in mind that using something like TZ='AEST-
> 10AEDT-11,M10.5.0,M3.5.0' is just for testing purpose: the server stores
> date and time in its current timezone.
>
> The previous comment (#9) was about the server. Probably the GTK client
> installer still doesn't correctly include the zoneinfo data.
>
> --
> You received this bug notification because you are a direct subscriber
> of the bug.
> https://bugs.launchpad.net/bugs/711084
>
> Title:
> [6.0,trunk] pytz zoneinfo missing from windows packaging - timezone
> conversion not working
>
> Status in OpenERP GTK Client:
> Confirmed
> Status in OpenERP Web Client:
> Confirmed
>
> Bug description:
> OpenERP server and web client 6.0.1 running on Ubuntu server (32-bit)
> 10.10
> Postgres 8.4
> Windows GTK client and web browser (Google Chrome) on WIndows 7 (64-bit)
>
> (Server and web client were set up as per
> http://powerphil.wordpress.com/2011/01/28/how-to-install-and-
> configure-an-openerp-6-0-1-server-and-web-server-on-an-
> ubuntu-10-10-server/).
>
> Problem is that the times shown in the GTK client are shown at GMT (unless
> the server's TZ environment variable is set), i.e. they ignore the user's
> Timezone preference.
> However, the web client works correctly.
>
> More details:
> Ubuntu server's time is correctly set to GMT. For example, if the time in
> London is 1am, the server's time also shows 1am (by doing a "date" on the
> command line, and the TZ environment variable is not set to anything).
>
> Both openerp-server and openerp-web processes are running without any
> TZ setting.
>
> Then log into OpenERP using a web browser; set the User's preference
> to Australia/Sydney; create a new Phone Call, Outbound; the time
> correctly shows 12 noon (GMT + 11 hours, Sydney currently being on
> summer time).
>
> Do the same in the GTK client; the time shows 1am - wrong; it should
> be 12 noon.
>
> ---------
>
> More information:
>
> So try to work around the problem: in the .bashrc file of the openerp user
> that runs the openerp-server process, set the TZ, viz:
> TZ='AEST-10AEDT-11,M10.5.0,M3.5.0'
> export TZ
>
> Restart the service; log into the GTK client; create a new phone call; the
> time now shows correctly (for the wrong reasons, admittedly).
> Go to the web browser and do the same thing; the time now shows 19 hours
> ahead of what it should be (on the next day!). Wrong.
>
> So it looks like the web client is taking timezone time into account,
> in addition to whatever the openerp-server is doing, and it should be
> the other way around: the openerp-server sho...

[Claire (cjin)]

Hi, we set Australia as our timezone. However, the time recorded for all sales orders, purchase orders etc are still GMT time......