[7.0] large number of mail.message prevents displaying a record

[Alexandre Fayolle - camptocamp]

Hello,

On a test instance running on my customer, the scheduler runs through a cron about 6x each day.

Each of these runs generates 3 mail.messages on procurements which are in exception:

=> select id,date, author_id, subject, body from mail_message where model='procurement.order' and res_id = 97 order by date;

   4577 | 2013-07-19 10:40:10 | 6 | | <p>Aucun fournisseur défini pour cet article !</p>
   4569 | 2013-07-19 10:40:10 | 6 | | <div>     • <b>Status</b>: Exception → Confirmed</div>
   4576 | 2013-07-19 10:40:10 | 6 | | <div>     • <b>Status</b>: Confirmed → Exception</div>
   4715 | 2013-07-19 11:40:42 | 6 | | <div>     • <b>Status</b>: Exception → Confirmed</div>
   4727 | 2013-07-19 11:40:43 | 6 | | <p>Aucun fournisseur défini pour cet article !</p>
   4726 | 2013-07-19 11:40:43 | 6 | | <div>     • <b>Status</b>: Confirmed → Exception</div>
   4775 | 2013-07-19 12:40:08 | 6 | | <div>     • <b>Status</b>: Exception → Confirmed</div>
   4786 | 2013-07-19 12:40:09 | 6 | | <div>     • <b>Status</b>: Confirmed → Exception</div>
   4787 | 2013-07-19 12:40:09 | 6 | | <p>Aucun fournisseur défini pour cet article !</p>
   [...]

On some of these records, the number of generated mail.messages over time can be high:

=> select count(id) from mail_message where model='procurement.order' and res_id = 97;
 count
-------
  5287
(1 ligne)

It is not possible to display that record in Firefox or Chrome, because the page takes forever to display with the browser becoming totally unresponsive, until the tab is killed.

Several problems here:

* automated generation of messages on a periodic tasks, which runs the risk of generating an uncontrolled amount of messages.
* no pagination of the display of messages. The feature seems to exists (cf. the Messaging -> Inbox screen) but it is not used when displaying a record with attached messages
* no way of mass marking messages as read (typically admin is used to run the scheduler via cron and admin is not connected every day on a customer's instance to mark messages as read. When I connect using that account to do support, I am blocked by the amount of unread messages on the procurement order)

[Nhomar - Vauxoo (nhomar)]

I confirm this issue, we are cleaning manually the messages, but we face this problem 2 or 3 times.

Even we customize the view of message to achieve it but IMHO it a bug, not leak of feature.

[Alexandre Fayolle - camptocamp (alexandre-fayolle-c2c)]

Quick and dirty cleanup query:

DELETE FROM mail_message WHERE model='procurement.order' AND date < CURRENT_DATE - 1 ;

[Martin Trigaux (OpenERP) (mat-openerp)]

Hello,

You are right it's not possible to work with 5000 messages loading on your record. It would be good to do a partial loading of the messages, adding a "read more button".

A simpler (short time vision) is to reduce the number of messages we send. Do we really need to log these exceptions ? It's already on the "Latest error" field. I would say to send this message only the first time. What do you think ?

We could do this for current version and add the pagination function for trunk.

[Alexandre Fayolle - camptocamp (alexandre-fayolle-c2c)]

Hello Martin,

Thanks for commenting on this issue.

I also had problems with way less than 5k messages :-) A couple 100s is enough to give firefox a sweat.

I agree that logging the messages only the first time is an acceptable solution for my customer. I have checked on the production database, and procurement.order is the only model which is affected by the issue in their database (and it is quite likely that other people are going to be affected by this issue, since setting up a cron for this is common). So in the short term, having something preventing messages from accumulating on procurement.orders is a quick win (and IMO that change should be kept in trunk in order to prevent uncontrolled DB growth)

Waiting for 8.0 to get messages pagination is ok with me.

[Martin Trigaux (OpenERP) (mat-openerp)]

Well actually do we even need to send messages on the procurement ? The procurement has a field message that contains the information about the error. The history of this is not really useful, the important point is why my procurement is blocked now...

In my 7.0 branch (that will be kept when forward porting), I have removed the message_post for this method.
I have created a task for R&D to add this feature for 8.0. Hopefully this will be added soon.

[Alexandre Fayolle - camptocamp (alexandre-fayolle-c2c)]

The fact that these messages accumulate tends to prove that we don't really need them on procurements. If the OPW patch I asked for consists in totally suppressing these error messages, and it does not get negative reviews on the editors side, I'll apply it happily on my customer's instance.

[Martin Trigaux (OpenERP) (mat-openerp)]

Ok so I have merged the branch lp:~openerp-dev/openobject-addons/7.0-purchase-reduce-mail-mat into 7.0

revno: 9502 [merge]
revision-id: <email address hidden>

This will prevent having more errors for this module and we are working on the chatter to avoid similar issues.

[Martin Trigaux (OpenERP) (mat-openerp)]

Hello,

We have merged another fix for this issue into 7.0

revno: 9527 [merge]
revision-id: <email address hidden>

This allows to reduce the messages generated by the track_visibility attribute to appear only once.

As mentioned in the code, this is rather hackish but was needed to make a change into stable version (and should probably be improved for next release).