mailgate filename unique error - DATA LOSS

[Ferdinand]

under certain conditions mails are not imported corretly and raise error
for testing purpose I am running fetchmail with option keep so I didn't loose data myself, but this is not the regular setup as it is necessary to examine and compare the mail account with what is imported - hence CRM helpdesk is not suitable for production !!
see also https://bugs.launchpad.net/bugs/746265 which will make it into trunk hopefully soon.

2011-04-04 19:01:10,368 WARNING Failed to process incoming email. Source of the failed mail is available at debug level.
Traceback (most recent call last):
  File "/home/terp/OpenERP/trunk/openobject-server/6.0/bin/addons/mail_gateway/scripts/openerp_mailgate/openerp_mailgate.py", line 57, in parse
    res_id = self.rpc('email.server.tools', 'process_email', self.model, xmlrpclib.Binary(message), custom_values)
  File "/home/terp/OpenERP/trunk/openobject-server/6.0/bin/addons/mail_gateway/scripts/openerp_mailgate/openerp_mailgate.py", line 37, in __call__
    return self.rpc.execute(self.dbname, self.user_id, self.passwd, *request, **kwargs)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1199, in __call__
    return self.__send(self.__name, args)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1489, in __request
    verbose=self.__verbose
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1253, in request
    return self._parse_response(h.getfile(), sock)
  File "/usr/lib64/python2.6/xmlrpclib.py", line 1392, in _parse_response
    return u.close()
  File "/usr/lib64/python2.6/xmlrpclib.py", line 838, in close
    raise Fault(**self._stack[0])
Fault: <Fault warning -- ValidateError

File name must be unique!: 'Traceback (most recent call last):\n File "/home/terp/OpenERP/trunk/openobject-server/6.0/bin/netsvc.py", line 489, in dispatch\n result = ExportService.getService(service_name).dispatch(method, auth, params)\n File "/home/terp/OpenERP/trunk/openobject-server/6.0/bin/service/web_services.py", line 599, in dispatch\n res = fn(db, uid, *params)\n File "/home/terp/OpenERP/trunk/openobject-addons/6.0/audittrail/audittrail.py", line 511, in execute\n res = my_fct(db, uid, model, method, *args)\n File "/home/terp/OpenERP/trunk/openobject-addons/6.0/audittrail/audittrail.py", line 495, in my_fct\n return fct_src(db, uid, model, method, *args)\n File "/home/terp/OpenERP/trunk/openobject-server/6.0/bin/osv/osv.py", line 128, in wrapper\n self.abortResponse(1, inst.name, inst.exc_type, inst.value)\n File "/home/terp/OpenERP/trunk/openobject-server/6.0/bin/netsvc.py", line 70, in abortResponse\n raise Exception("%s -- %s\\n\\n%s"%(origin, description, details))\nException: warning -- ValidateError\n\nFile name must be unique!\n'>
Traceback (most recent call last):
  File "/usr/lib64/python2.6/logging/__init__.py", line 768, in emit
    msg = self.format(record)
  File "/usr/lib64/python2.6/logging/__init__.py", line 648, in format
    return fmt.format(record)
  File "/usr/lib64/python2.6/logging/__init__.py", line 436, in format
    record.message = record.getMessage()
  File "/usr/lib64/python2.6/logging/__init__.py", line 306, in getMessage
    msg = msg % self.args
TypeError: not all arguments converted during string formatting
reading message chricar-crm@10.147.160.3:139 of 140 (1547 header octets) (73290 body octets) not flushed

[Ferdinand (office-chricar)]

it seems this has revived in 6.0.2
https://bugs.launchpad.net/openobject-addons/+bug/730632

[Ferdinand (office-chricar)]

it seems to happen if the requestor attaches vcards or similar attachments with same name (and size) to all his mails. this is not uncommon.

IMHO if attachment name AND checksum are identical it is neither necessary to save the attachment again and again nor to raise an error.

[Ferdinand (office-chricar)]

IMHO if attachment name AND checksum are identical it is neither necessary to save the attachment again and again nor to raise an error
ELSE
the attachment must be ( renamed and) saved

[Ferdinand (office-chricar)]

the problem is another one

after modifying
addons/document/ document.py
            #if len(res):
            # return False
to allow duplicate entries

during "create" of the first crm_helpdesk entry 2 (!) identical attachments (instead of one) are created.
So it always fails if the partner sends an attachment.

IMHO this may be new in 6.0.2 but similar to
https://bugs.launchpad.net/bugs/731624 where same peration is executed twice

Question, why does OpenERP checks hard for duplicate entries
IMHO it would be sufficient to quietly scip attachments with the same checksum and store the others (as already suggested)

[Eric Caudal - www.elico-corp.com (elicoidal)]

I tried the patch from xrg without success.

Ferdinand is right there is a duplicate attachment insertion somewhere. I have no time to investigate this so I changed a little bit the code as:
                try:
                    attachments.append(att_obj.create(cr, uid, {'type':'binary','res_model':case._name,'res_id':case.id,'name': att[0],
                    'datas': base64.encodestring(att[1])}))
                except:
                    pass
It is not perfect (what if there is actually 2 times the same attachment in the mail?) but it solved my problem.