[6.0][6.1][stock] get_product_available needs optimization

[Sébastien BEAU - http://www.akretion.com]

Hi all
Everything is in the title ;)
When I open the menu "location" from "warehouse=>configuration=>warehouse management=>location"
OpenERP need 45 second (on a core i5) to open it, python process run at 100% :(

The problem is really easy to understand. Let's explain.
Scenario a database with:
- 4000 product
- two language installed and all product fields translated

Two bug :
First bug : useless stock level computation is done when we open the menu location
Second bug : useless read on all translated field on the object 'product.product' when we compute the stock level

First bug:
The view "view_location_tree2" in the file "stock/stock_view.xml" add the field "stock_real" and "stock_virtual"

        <record id="view_location_tree2" model="ir.ui.view">
            <field name="name">stock.location.tree</field>
            <field name="model">stock.location</field>
            <field name="type">tree</field>
            <field name="priority" eval="2"/>
            <field name="arch" type="xml">
                <tree string="Stock Location" colors="blue:usage=='view';darkred:usage=='internal'">
                    <field name="complete_name"/>
                    <field name="usage"/>
                    <field name="stock_real" invisible="'product_id' not in context"/>
                    <field name="stock_virtual" invisible="'product_id' not in context"/>
                </tree>
            </field>
        </record>

In this case we don't need it and so we make it invisible. But making the field invisible will not remove it from the view and we still need to compute them.
In this case computing the stock level of the location is useless and long (45s for 4000 product)

Second bug:
Using browse method on the object "product.product" in the function 'get_product_available' (file : stock/product.py) should be not use. Why?
Because we prefetch all fields with browse method. And when your database have 4000 products translated, the translation of the sale description of all of your product will be read in order to compute the stock level (yes you read it)

For this second bug, I just apply an ugly path to test and fix it

+ uom_ids=[]
         product2uom = {}
- for product in self.browse(cr, uid, ids, context=context):
- product2uom[product.id] = product.uom_id.id
- uoms_o[product.uom_id.id] = product.uom_id
+ for product in self.read(cr, uid, ids, ['uom_id'], context=context):
+ product2uom[product['id']] = product['uom_id'][0]
+ if not product['uom_id'][0] in uom_ids : uom_ids.append(product['uom_id'][0])
+ print 'uom_ids', uom_ids

+ for uom in self.pool.get('product.uom').browse(cr, uid, uom_ids, context=context):
+ uoms_o[uom.id] = uom
+

Just replacing the browse method by a read method and then I was able to open my menu in less than 2 seconds.
It's mean that computing stock level can be done 22x faster when you have translated contain!! (great news, no?)

Exactly the problem is not the browse method because it's cleaner to use browse in the code ;). The huge problem is that browse method do not support field restriction, I mean we can not just apply a browse on specific field as we can do with the read method.

To finish If I remove the two field stock_real and stock_virtual from the tree view opening the menu is instantaneous.

Hope my explication is clear. We should really fix this issue.
Reading the translation of the sale description for computing the stock is not clean at all ;).

Have a nice day everyone

Also I send an email to support team with LP link, my customer have an OPW

[Yann Papouin (yann-papouin)]

I'm using 6.0.3 in a french context and I'm currently filling my company database with supplier's products, currently I have 143000 products in it.

Some actions cannot be done like "procurement request" or "automatic orderpoint" because it results in a 100% python CPU process until my server crash, RAM (8Go) and SWAP(8Go) are filled both filled.

It seems related to this bug because the thread is stuck in get_product_available

[Yann Papouin (yann-papouin)]

Thank you Sébastien, your explanation helped me to wrote a fix for my issue, I reported it as Bug #915541

[Yann Papouin (yann-papouin)]

I'm attaching the patch that MUST be applied to 6.1 if you are in a real production environment

[Yann Papouin (yann-papouin)]

Oops, I mean 6.0 as It needs to be adapted for 6.1

[Yann Papouin (yann-papouin)]

Correct Patch for 6.0 / 6.1

[Amit Dodiya (OpenERP) (ado-openerp)]

For 6.0 the fix is landed with following revision:
Revision-id: <email address hidden>
Revision-no: 5122

[Amit Dodiya (OpenERP) (ado-openerp)]

Hello,

For 6.1 the issue is fixed with following branch :
Branch: lp:~openerp-dev/openobject-addons/6.1-opw-50947-ado
Revision-id: <email address hidden>
Revision-no: 6987

Regards,
Amit Dodiya