[Trunk] Procurement request leads to server crash

[Yann Papouin]

I'm using 6.0.3 in a french context and I'm currently filling my company database with supplier's products, currently I have 114000 products in it.

Some actions cannot be done like "procurement request" or "automatic orderpoint" because it results in a 100% python CPU process until my server crash, RAM (8Go) and SWAP(8Go) are filled both filled. The thread is stuck in get_product_available.

=== FIX ===

The fix attached allows to check automatic orderpoint on 100.000 products in only 10s, no functionality is lost (technically based on Bug #899470 )
I know that the 6.1 is next to be release but I think that a fix like this is mandatory.

[Raphaël Valyi - http://www.akretion.com (rvalyi)]

Hello,

So if I understand right, you are avoiding a gigantic product browse over all products and do a fast read and then browse only what really needs to. Sounds great, it's amazing you are winning so much time!

Now, I would like to mention that the 6.1 doesn't have such an optimization, so if the win is so important, I strongly suggest to make the patch on 6.1 too. Thanks for the idea.

[Ravish(OpenERP) (rmu-openerp)]

Hello,

I am agree with Yann Papouin . It is good to use read method instead of browse method.
For "Time Consuming" point of you ,I am considering this issue as good improvement

Thanks!

[Mayur Maheshwari(OpenERP) (mma-openerp)]

Hello,

I have apply your patch .Thanks for your contribution

It has been fixed in lp:~openerp-dev/openobject-addons/trunk-bug-915541-mma
Revision ID: <email address hidden>
Revision num: 6353

It will be available in trunk soon.

[Kyle Waid (midwest)]

=== modified file 'procurement/schedulers.py'
--- procurement/schedulers.py 2011-01-14 00:11:01 +0000
+++ procurement/schedulers.py 2012-01-18 19:01:57 +0000
@@ -53,13 +53,18 @@

             procurement_obj = self.pool.get('procurement.order')
             if not ids:
- ids = procurement_obj.search(cr, uid, [], order="date_planned")
+ ids = procurement_obj.search(cr, uid,
+ [('state', '=', 'exception')], order="date_planned")
             for id in ids:
- wf_service.trg_validate(uid, 'procurement.order', id, 'button_restart', cr)
+ wf_service.trg_validate(uid, 'procurement.order',
+ id, 'button_restart', cr)
             if use_new_cursor:
                 cr.commit()
- company = self.pool.get('res.users').browse(cr, uid, uid, context=context).company_id
- maxdate = (datetime.today() + relativedelta(days=company.schedule_range)).strftime(tools.DEFAULT_SERVER_DATE_FORMAT)
+ company = self.pool.get('res.users').browse(cr, uid,
+ uid, context=context).company_id
+ maxdate = (datetime.today() + \
+ relativedelta(days=company.schedule_range)).\
+ strftime(tools.DEFAULT_SERVER_DATE_FORMAT)
             start_date = time.strftime('%Y-%m-%d, %Hh %Mm %Ss')
             offset = 0
             report = []
@@ -67,19 +72,25 @@
             report_except = 0
             report_later = 0
             while True:
- cr.execute("select id from procurement_order where state='confirmed' and procure_method='make_to_order' order by priority,date_planned limit 500 offset %s", (offset,))
- ids = map(lambda x: x[0], cr.fetchall())
- for proc in procurement_obj.browse(cr, uid, ids, context=context):
+ ids = procurement_obj.search(cr, uid, [
+ ('state', '=', 'confirmed'),
+ ('procure_method', '=', 'make_to_order')
+ ], offset=offset, limit=500,
+ order='priority, date_planned', context=context
+ )
+ for proc in procurement_obj.browse(cr, uid, ids,
+ context=context):
                     if maxdate >= proc.date_planned:
- wf_service.trg_validate(uid, 'procurement.order', proc.id, 'button_check', cr)
+ wf_service.trg_validate(uid,
+ 'procurement.order', proc.id, 'button_check', cr)
                     else:
- offset += 1
                         report_later += 1
- for proc in procurement_obj.browse(cr, uid, ids, context=context):
+ offset += 1
+
                     if proc.state == 'exception':
- report.append('PROC %d: on order - %3.2f %-5s - %s' % \
- (proc.id, proc.product_qty, proc.product_uom.name,
- proc.product_id.name))
+ report.append(_('PROC %d: on order - %3.2f %-5s - %s') % \
+ (proc.id, proc.product_qty, proc.product...

[Yann Papouin (yann-papouin)]

Hi Kyle, I think that your patch is attached to the wrong bug report, would you mean Bug 918288 ? If so can you link your patch as attachment.

[Raphael Collet (OpenERP) (rco-openerp)]

landed in trunk
revno: 6399 [merge]
revision-id: <email address hidden>

[Yann Papouin (yann-papouin)]

Please review bug #899470 as this fix is also needed in 6.1:

for product in self.browse(cr, uid, ids, context=context): MUST BE replaced by a READ method in get_product_available of product.py