webdav very slow

[Ferdinand]

demo data base
try to open products ~50 items / directories

it takes 30 seconds (a dual processor) to open a list of 50 products and puts 50% load on each processor

[m.karrer (m-karrer)]

Same here even is there is only one item in the list it takes like 7 seconds to open the folder and puts 100% load on the python process for this time?!?

I think this is to be considered really really slow - its simply not usable in an production environment where a lot of people connect to "open erp as a folder" via webdav!

(Clean Database, only one user! and a 3.5 GHZ intel prozessor)

[Ferdinand (office-chricar)]

i tried also with cadaver - most response times around 10 seconds -
I see very heavy database access
I have a structure with m:n - typically a multiple of 10 (1:10) on each level of the levels - the deepest level has 290 records in total - so this is not a very important number

the first level is a recursive tree ( stock.location )

I confirm - not usable in production (and daunting at demos)

[OpenERP Romania (x-contact-diginesis-com)]

Interesting :)

[Andrei Levin (andrei-levin)]

I produce a patch that will make webdav at least 5 times faster. It will also reduce database access.

Apply a patch outside from document_webdav directory with:
patch -p0 < patch_webdav.diff

[Andrei Levin (andrei-levin)]

BTW, a more elegant way will be to use decorator like this:

def uri_cache(f):
    cache = {} # map from key to value
    def wrapper(self, cr, uid, pool, uri):
        key = repr(uri)
        if not key in cache:
            cache[key] = f(self, cr, uid, pool, uri)
        return cache[key]
    return wrapper

And apply it to uri2object function:

    @uri_cache
    def uri2object(self, cr, uid, pool, uri):
...

[Thomas Rehn (thomas-rehn)]

I encountered the same issue last week. After a bit of analysis, I am afraid there is no easy solution to this obvious performance problem. One problem is that the result of uri2object depends on the cursor cr, so the cache would have to depend on the cursor, too. However, the cursor changes with every(?) call of uri2object, which makes caching impossible. A wrongly cached cursor leads to
 WARNING openerp.addons.document_webdav.webdav_server: GET exception: Unable to use a closed cursor
problems for me.
I don't know if obtaining a new cursor every time is intended, but it also is not too easy to change. There is a central method to obtain a cursor (dav_fs.py:openerp_dav_handler.get_cr(.)), but every calling method closes the cursor. So caching of get_cr alone does not help either. One idea could be to add methods to dav_fs.py:openerp_dav_handler that get called at the beginning and at the end of every request, opening and closing a cursor, respectively. But there probably are nicer ways to achieve a similar effect.

[Thomas Rehn (thomas-rehn)]

I implemented the changes that I sketched in my previous comment and implemented them in https://code.launchpad.net/~initos.com/openobject-addons/7.0-715444-webdav-performance-improvement/+merge/200288

[Andrei Levin (andrei-levin)]

I am afraid, that if you compare your solution with mine, you will see no performance difference, as opening/closing cursor is very fast operation. Though I agree with you that continually opening/closing cursor looks like something wrong.

[Thomas Rehn (thomas-rehn)]

In my DAV application, a deep virtual document tree and many leaves, I did not experience a big performance gain when the cursor was part of the cache-key because the cursor is closed and opened many times which interferes with the cache.

[Andrei Levin (andrei-levin)]

In fact my decorator is caching uri only, because of the changing cursors