[trunk] Addons path incorrect algorithm to load files.

[Nhomar - Vauxoo]

Hello.

This error took from us some time to solve, but i think it is important.

When you define the first addons base in a folder called addons and in the same directory that your other addons branches i mean:

/working/directory/trunk/addons
/working/directory/trunk/addons-other
/working/directory/trunk/web
/working/directory/trunk/server

And you start server with addons path:

 /working/directory/trunk/addons,/working/directory/trunk/addons-other,/working/directory/trunk/web/addons

When the second addons is taken in account the algorithm already in server brakes telling you can not load __opener__.py server that dont existe, the traceback looks like this:

2012-11-08 15:08:29,685 27703 ERROR tgaby openerp: Failed to initialize database `tgaby`.
Traceback (most recent call last):
  File "openerp-server", line 94, in preload_registry
    db, registry = openerp.pooler.get_db_and_pool(dbname, update_module=openerp.tools.config['init'] or openerp.tools.config['update'], pooljobs=False)
  File "/home/nhomar/instancias/7.0/server/openerp/pooler.py", line 33, in get_db_and_pool
    registry = RegistryManager.get(db_name, force_demo, status, update_module, pooljobs)
  File "/home/nhomar/instancias/7.0/server/openerp/modules/registry.py", line 156, in get
    update_module, pooljobs)
  File "/home/nhomar/instancias/7.0/server/openerp/modules/registry.py", line 178, in new
    openerp.modules.load_modules(registry.db, force_demo, status, update_module)
  File "/home/nhomar/instancias/7.0/server/openerp/modules/loading.py", line 328, in load_modules
    processed = load_marked_modules(cr, graph, states_to_load, force, status, report, loaded_modules)
  File "/home/nhomar/instancias/7.0/server/openerp/modules/loading.py", line 241, in load_marked_modules
    graph.add_modules(cr, module_list, force)
  File "/home/nhomar/instancias/7.0/server/openerp/modules/graph.py", line 102, in add_modules
    info = openerp.modules.module.load_information_from_description_file(module)
  File "/home/nhomar/instancias/7.0/server/openerp/modules/module.py", line 352, in load_information_from_description_file
    f = tools.file_open(terp_file)
  File "/home/nhomar/instancias/7.0/server/openerp/tools/misc.py", line 157, in file_open
    return _fileopen(name, mode=mode, basedir=base, pathinfo=pathinfo, basename=basename)
  File "/home/nhomar/instancias/7.0/server/openerp/tools/misc.py", line 229, in _fileopen
    raise IOError('File not found: %s' % basename)
IOError: File not found: /home/nhomar/instancias/7.0/addons-vauxoo/user_story/__openerp__.py

then the error have to parts:

1.- The file /home/nhomar/instancias/7.0/addons-vauxoo/user_story/__openerp__.py exists, then is so confusing an error telling it is not, in this point start my timing consuming, the reason is the raise bad concatenated with incorrect name:

here is where the first diff comes out:

=== modified file 'openerp/tools/misc.py'
--- openerp/tools/misc.py 2012-10-24 16:15:27 +0000
+++ openerp/tools/misc.py 2012-11-08 15:12:24 +0000
@@ -218,7 +218,7 @@
     # Not found
     if name.endswith('.rml'):
         raise IOError('Report %r doesn\'t exist or deleted' % basename)
- raise IOError('File not found: %s' % basename)
+ raise IOError('File not found: %s' % name)

 #----------------------------------------------------------

We must use "name" (modified name that is realle used to find __openerp__.py file) and not "basename" (original name first used to compare if it is base module or not).
2.- The second part is "Why the file is bad named?" if all folders worked __exactly__ in this naming convention on 6.1?:

The reason is the commit 4420 done for Stephan:

The diff of this revno 4420 :
http://bazaar.launchpad.net/~openerp/openobject-server/trunk/revision/4420

For some reason, he replace basename = name with a new return, imho it was not necesary, but bte, just changing veriable it works very well.

3rd. Part: Bad Algorithm:.

The algorithm to find the path that is in misc.py file IMHO is incorrect, because compare "String of path" and not "Path itself" to decide if it is or not base-addons.

You can see:

line ~139 openerp/tools/misc.py:

        for root in adps + [rtp]:
            if name.startswith(root):
                base = root.rstrip(os.sep)
                name = name[len(base) + 1:]
                break

What is wrong, is that we are using 'startswith' instead a path tool to compare and strip folders name, the in case you use a second folder with a second list of addons it is striping incorrectly, and giving an incorrect message (corrected in point [2])

Printing root, adps and

[u'/home/nhomar/instancias/7.0/addons', u'/home/nhomar/instancias/7.0/addons-vauxoo', u'/home/nhomar/instancias/7.0/web/addons', u'/home/nhomar/instancias/7.0/test_pycon', u'/home/nhomar/instancias/7.0/openerp-venezuela-localization', '/home/nhomar/instancias/7.0/server/openerp/addons']
/home/nhomar/instancias/7.0/addons
/home/nhomar/instancias/7.0/server/openerp

In my case as you can see:

When you make name.startswith(root):

/home/nhomar/instancias/7.0/addons
/home/nhomar/instancias/7.0/addons-vauxoo

This case will pass and:

name = name[len(base) + 1:]

will become:

vauxoo/user_story/__openerp__.py

what is incorrect, because this folder doesn't exists.

Solution:
I will make an MP, solving the problem with 'os' library, soon in touch

[Nhomar - Vauxoo (nhomar)]

Hello,

I fix the issue, in this way, we avoid use string features without os.path verifications.

Regards/.

[Nhomar - Vauxoo (nhomar)]

Testit.

http://runbot.openerp.com/vauxoo.html

[Nhomar - Vauxoo (nhomar)]

How test?

download your official branches and use this enviroment folder:

lp:openobject-server call it server
lp:openobject-addons call it addons
lp:openerp-web call it web

Extra branch:

Example:

lp:addons-vauxoo callit addons vauxoo

Important:
All in the same folder:

Declare your addons-path with absolute paths to this projects.

Up your server and create a new db.

Your traceback will look as the explained here.

[Olivier Dony (Odoo) (odo-openerp)]

Looks like a valid bug, thanks for the extended analysis and the merge proposal! We'll review it and merge an appropriate fix.

[jugglefish (pniederlag)]

Using current trunk as of 2012-12-21 this bug seems to be still there. For me it doesn't depend on the path being defined as relative or absolute. As long as the path for additional addons resides in the same folder as the defaults I run into troubles.

As workaround one has to move those addons to a really separate dir:
mv /opt/WORKDIR/addons-custom /opt/SOMEOTHERDIR
and setting addons-path=addons,web/addons,/opt/SOMEOTHERDIR

[Christophe CHAUVET (christophe-chauvet)]

Hi

I have the same problem, all path starts with addons have been replace in "name" variable

/home/xxxxx/project/addons-customer/ become /home/xxxxx/project/addons/customer/
/home/xxxxx/project/addons-jasper/ become /home/xxxxx/project/addons/jasper/

all - (dash) have been replace by a / (slash)

if the folder is name jasper-addons et works because folder name endswith "addons"

Regards,

[Vo Minh Thu (thu)]

I could not reproduce your problem. I have tried many possibilities, here is the last one. Does the problem still exists for you in trunk ?

/home/thu/t
├── addons
│   ├── account
│   │   ├── __init__.py
│   │   ├── installer.py
│   │   └── ...
│   │
│   ├── account_accountant
│   │   └── ...
│   └── ...
│
├── addons-other
│   └── openacademy
│      ├── __init__.py
│      ├── __openerp__.py
│   └── ...
│
├── server
│   ├── openerp
│   │   ├── addons
│   │   │   ├── base
│   │   │   │   ├── __init__.py
│   │   │   │   ├── __openerp__.py
│   │ │ │ └── ...
│   │  │   └── ...
│   │   ├── __init__.py
│   │  └── ...
│   │
│   ├── openerp-server
│   └── ...
│
└── web
    ├── addons
    │   ├── web
    │   │   ├── __init__.py
    │   │   ├── __openerp__.py
    │ │ └── ...
    │ └── ...
    └── ...

[thu@nostromo t]$ pwd
/home/thu/t
[thu@nostromo t]$ ls
addons addons-other server web
[thu@nostromo t]$ python2 server/openerp-server --addons-path /home/thu/t/addons,/home/thu/t/addons-other,/home/thu/t/web/addons -d xx -i openacademy --stop-after-init --test-enable
2013-06-06 10:12:44,322 4470 INFO ? openerp: OpenERP version 8.0alpha1
2013-06-06 10:12:44,322 4470 INFO ? openerp: addons paths: /home/thu/t/addons,/home/thu/t/addons-other,/home/thu/t/web/addons
...