[7.0] 1.8.4 Multiworker include resource bug

[Sandy Carter (http://www.savoirfairelinux.com)]

Description
--------------

When running a 7.0 server with workers>1, each worker will crash when they spawn when they try to import resource.

What is happening is that when odoo/addons is included in the sys.path, when python tries to import resource in (openerp/service/workers.py), openerp's custom import code will correctly try to import from the system before trying to import from its addons (for legacy code). Since the odoo/addons is in the path, the openerp module odoo/addons/resource will be imported instead of the system library resource

The reason this is a recipe bug, is that when running the buildout recipe, the start_openerp script will include odoo/addons to the sys.path. This inclusion causes the wrong library to be imported by multiworkers. When they spawn, they will repeatedly output the following error:

2014-07-15 22:18:57,027 12681 INFO ? openerp: OpenERP version 7.0
2014-07-15 22:18:57,027 12681 INFO ? openerp: addons paths: /opt/openerp7/git/odoo/openerp/addons,/opt/openerp7/git/odoo/addons,/opt/openerp7/bzr/partner-contact-management,/opt/openerp7/bzr/travel,/opt/openerp7/bzr/hr,/opt/openerp7/git/server-env-tools,/opt/openerp7/git/program,/opt/openerp7/git/connector,/opt/openerp7/bzr/knowledge-addons,/opt/openerp7/git/openerp/addons
2014-07-15 22:18:57,027 12681 INFO ? openerp: database hostname: localhost
2014-07-15 22:18:57,027 12681 INFO ? openerp: database port: 5432
2014-07-15 22:18:57,027 12681 INFO ? openerp: database user: openerp7
2014-07-15 22:18:57,028 12681 WARNING ? openerp.modules.module:
Ambiguous import: the OpenERP module `resource` is shadowed by another
module (available at /opt/openerp7/git/odoo/addons/resource).
To import it, use `import openerp.addons.<module>.`.
2014-07-15 22:18:57,037 12686 INFO ? openerp.service.workers: Worker WorkerHTTP (12686) alive
2014-07-15 22:18:57,039 12687 INFO ? openerp.service.workers: Worker WorkerHTTP (12687) alive
2014-07-15 22:18:57,039 12686 ERROR ? openerp.service.workers: Worker (12686) Exception occured, exiting...
Traceback (most recent call last):
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 304, in run
    self.process_limit()
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 265, in process_limit
    soft, hard = resource.getrlimit(resource.RLIMIT_AS)
AttributeError: 'module' object has no attribute 'getrlimit'
2014-07-15 22:18:57,040 12687 ERROR ? openerp.service.workers: Worker (12687) Exception occured, exiting...
Traceback (most recent call last):
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 304, in run
    self.process_limit()
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 265, in process_limit
    soft, hard = resource.getrlimit(resource.RLIMIT_AS)
AttributeError: 'module' object has no attribute 'getrlimit'
2014-07-15 22:18:57,040 12688 INFO ? openerp.service.workers: Worker WorkerHTTP (12688) alive
2014-07-15 22:18:57,042 12689 INFO ? openerp.service.workers: Worker WorkerHTTP (12689) alive
2014-07-15 22:18:57,043 12688 ERROR ? openerp.service.workers: Worker (12688) Exception occured, exiting...
Traceback (most recent call last):
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 304, in run
    self.process_limit()
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 265, in process_limit
    soft, hard = resource.getrlimit(resource.RLIMIT_AS)
AttributeError: 'module' object has no attribute 'getrlimit'
2014-07-15 22:18:57,043 12690 INFO ? openerp.service.workers: Worker WorkerHTTP (12690) alive
2014-07-15 22:18:57,045 12689 ERROR ? openerp.service.workers: Worker (12689) Exception occured, exiting...
Traceback (most recent call last):
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 304, in run
    self.process_limit()
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 265, in process_limit
    soft, hard = resource.getrlimit(resource.RLIMIT_AS)
AttributeError: 'module' object has no attribute 'getrlimit'
2014-07-15 22:18:57,046 12690 ERROR ? openerp.service.workers: Worker (12690) Exception occured, exiting...
Traceback (most recent call last):
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 304, in run
    self.process_limit()
  File "/opt/openerp7/git/odoo/openerp/service/workers.py", line 265, in process_limit
    soft, hard = resource.getrlimit(resource.RLIMIT_AS)
AttributeError: 'module' object has no attribute 'getrlimit'
2014-07-15 22:18:57,047 12691 INFO ? openerp.service.workers: Worker WorkerHTTP (12691) alive
2014-07-15 22:18:57,050 12691 ERROR ? openerp.service.workers: Worker (12691) Exception occured, exiting...

How to reproduce
--------------------

Using 1.8.4 install using the git source for odoo
run start_openerp --workers=2

Expected output
-------------------

The start_openerp script doesn't append odoo/addons to the system path
Workers spawn without a fuss, importing the system library named resource and calling the existing getrlimit function

Actual output
----------------

The start_openerp script appends odoo/addons to the system path
Workers spawn, import the odoo module named resource (odoo/addons/resource) and when calling the existing getrlimit function, crash saying there is no such attribute

[Georges Racinet (gracinet)]

It's weird that nobody has stumbled on this one before.

I've never been completely sure about these sys.path entries questions, as you can guess by reading comments in the relevant part of the code. Also we have a source of incertainty in that I'm not sure what you suggest isn't the contrary of another fix of the same kind (also related to non-namespacing in OpenERP past history).

So In short, I'd like to be cautious about this, because it'll require a lot of testing on our side

Anyway, thanks for the analysis

Now some questions :
- do you need a fix for this in the stable branch of the recipe ?
- did this use to work on an earlier version of the recipe ? If yes, please provide me with the version number, it'll help us browse the history and maybe detect if there was any reason for that sys.path entry in OpenERP 7

[Georges Racinet (gracinet)]

Ah, I noticed you're using the odoo github repo, that may be a consequence of the new layout, and that could explain why it hasn't been noticed before.

[Georges Racinet (gracinet)]

I did not reproduce with the old layout (e.g, bzr branches)

I just got a similar problem as yours with the github layout on version 7, triggered by shadowing of the 'calendar' standard module by the 'calendar' addon, which had been solved earlier for version 8.

Quick, conservative fix is on its way for the stable branch.

[Georges Racinet (gracinet)]

It really looks like a mistake that's been sitting around since 6.1, waiting to fire back at us the day there'd be an addons directory right next to the openerp package !