[performance] confirming a purchase order takes ages

[Holger Brunn (Therp)]

It takes hours to confirm a purchase order with order lines which refer to some hundreds of procurements, thereby creating some hundreds of stock moves.

The bottleneck seems to be the fact that there are some function fields on stock.move (created by purchase, stock, delivery, stock_account - nothing fancy) that are evaluated each on its own. The attached patch fixes this by suppressing function field evaluation for the create call and evaluating them en bloc afterwards. This way, the confirmation needs only a minute compared to some hours.

In my opinion, there should be a bulk_create() function with the same signature as create() that does exactly this. I couldn't find anything like that in the apiculture branch, does it make sense to prepare a merge proposal for that?

[Holger Brunn (Therp) (hbrunn)]

PS: I think doing the same in procurements would also massively speed up the procurement process

[Josse Colpaert (OpenERP) (jco-openerp)]

Dear Holger,

On itself, this is a good idea. I checked with Raphaël and he says it would also be portable to the new api, but with recompute instead of the whole store_get/set_values. We wonder however if this is working fine as it is in your merge. (_inherits is not needed, and the ids passed to _store_set might give errors). It is also usable for the procurements, but I do not think the effect will be as big as it is here.

[Holger Brunn (Therp) (hbrunn)]

Thanks for your quick reply. In my first tests, it worked fine. Currently, there's a job running that should be done by tomorrow, if I'll encounter any problems there, I'll post them here.

Further: I simply did what create() does in http://bazaar.launchpad.net/~openerp/openobject-server/trunk/view/head:/openerp/osv/orm.py#L4260, could you point me to the differences that could break it?

[Holger Brunn (Therp) (hbrunn)]

Meanwhile, this is tested with several hundred real live purchase orders, and I didn't encounter any problems. It's rougly 80% faster in my case.

Then I was digging into mrp, the result is the branch linked today. As you predicted, the effect is a lot smaller (~10%)

Futher I was working on tweaks for stock.move (https://code.launchpad.net/~therp-nl/openobject-addons/trunk-wms-stock-performance). The changes I made have measurable, but no impressive effects. I'm stuck here currently at http://bazaar.launchpad.net/~therp-nl/openobject-addons/trunk-wms-stock-performance/view/head:/purchase/stock.py#L48 (calling workflows has to be per ID). Any idea how to optimize this kind of code?