Magentoerpconnect attribute data corruption

[Kyle Waid]

This is one of the most serious bugs I have seen in the connector. Everyone who uses trunk is impacted. The bug. Core magento attributes are corrupted if altered or deleted. Corruption is irreversible. Let me explain and tell how to reproduce.

Start with fresh OpenERP 6.1 database. Use extra-trunk extra addons, 6.1 addons, openerp-server 6.1, magentoerpconnect trunk.
Sync the ERP with your favorite demo magento site, 1.4, 1.5, 1.6, whatever.

Import the steps, products, attirbutes, blah blah.

Click on a product, Open Magento Fields. You can see "visibility" (Required attibute) by default is set "Catalog, Search". If I open the resource, I can see the configuration, or I can search and see other available items for selection. Great, it works perfectly.

NOW. Go to magentoerpconnect, attributes. DELETE all attributes.

Go and re-sync your site data. Import attributes over again. You see your attributes that were previously deleted, created once again and everything looks fine!

Go back to the same product as before. Open Magento fields. You see that "Catalog, Search" is still selected. If you click the search to see other selections, behold, none exist. My point, once the data has changed, its irreversible and the entire database is corrupted. Re-syncing site data does nothing to correct the problem. it is lost forever. This leads to another bug recently reported, that will actually export this corrupted product to the website successfully but it will not be visible on the admin because the visibility mapping is corrupted.

Even with the current options selected, you cannot export to the website, because the Catalog, Search selected is pointing to a non-existent resource, and even worse, you cannot select any other resource, its broken. Many fields are corrupted, not just visibility.

To cap, Any user using trunk, if they alter their attributes, delete just one, or try to migrate to some other site,they will experience this issue.

[Kyle Waid (midwest)]

This bug exists in server 6.1, and the akretion 6.1 > 6.0 server backport

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

Hi,
For each attribute there is a field (ir.model.fields) in product.template.
For attributes with options (m2o, m2m), a domain is declared on the field.

So let me explain with the visibility as exemple.
You have to edit the field x_magerp_visibility of product.template, you'll see that it is a m2o pointing to magerp.product.attributes.options (something like that) and it has a domain like [('attribute_id', '=', 10). Here, 10 is the id of the visibility attribute when it has been created the first time. You have to replace it by the new id of the visibility attribute and it should be restored.
Then repeat this for each relational attribute.

On my side, I do not understand why you would need to delete attributes on the OpenERP side?
If that's an user error so maybe we just have to forbid that?

Guewen

[Kyle Waid (midwest)]

Hi,

I understand what you are saying. I found some domains in product.py but none that were statically assigned which you described. The issue, is that once these attributes are unlinked, even importing the attributes over again does not fix the problem. Once it is unlinked it is in this 'corrupted' state. The attribute which you are talking about is actually on product_product not product_template. In older versions of the connector it uses x_magerp_visibility but the newest I believe is a serialized value under magerp_variant, even without product_variant installed. I get confused because this is changing quite a bit.

I see in the database that there is a constraint against table magerp.product.attributes.options Just as you have said in the M2O. The major issue is that, as I stated, once these are unlinked it cant be fixed. I tried fixing this in the database but it does not make the fields searchable. What I can do, is go to the database and manually assign the attribute. In Open Magento Fields it will show the value I selected, but the export will not recognize the value.

 It is a serious issue for me for so many customers. This corruption can easily happen. The way I described is to show you how to easily reproduce the issue, but it is not the only way the corruption occurs. I am unsure of all of the ways but I assure you it happens and it makes export completely unusable. What happens in this state, it will actually export the product successfully to the website, but the field visibility is required by the magento ORM so the product will not show on the admin panel, but will exist in the database. The only way to fix it is to go into the magento database and set this field under catalog_product_entity_int. The issue with updating products, is that it runs from ERP successfully with no error but will not change anything about the product on the website. This problem effectively ruins product management.

When a resource gets unlinked or deleted, I should have the ability to re-import attributes and have it repaired. This is not the case, once the damage is done, its done. Please try the steps that I have outlined, its very easy to see the issue. Once you see the problem, you will understand my frustration.

[Kyle Waid (midwest)]

Just to clarify my first statement, I have checked what you said. In table magerp_product_attribute_options, the id is the link to the value of the attribute. column attribute_id is the link to the attribute that the value applied. column value is the actual value stored in the magento database. If these values become unlinked by deleting them or otherwise, if I go into the database and repair this just as you have described it does NOT make the values visible on the product form. They are still not searchable. There is something else causing this issue.

Please test as I have described and tell me your results

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

Hi,

Ok, what I said will correct the field, but not the stored ids of the options.

So we have to forbid the deletion of an attribute when it has options and at least one option is used on a product.

[Kyle Waid (midwest)]

There are other ways the corruption occurs aside from deleting the attribute. Also, even if I correct the value of the field, export does not work correctly. Please evaluate each statement, there are multiple impacts of the problem

[Sébastien BEAU - http://www.akretion.com (sebastien.beau)]

Hello Kyle.
I already face to the same problem, removing attributs is quite dangerous and the connector doesn't support correctly this feature (but this will come in the future).

The solution is to
- delete the attributs
- delete also the Magento Field of the product.template
- delete the contain of the magento json
- delete the mapping
- importing the attributs
- importing the product

I already did it various time for my customer, and it's work perfectly.

As we plan to refactor the support of attributs for 6.2 (we want to extract this feature in a generic modules that inherits ir.model.fields). I will try to support better the fact to remove attribut. But for now the best solution is to "not deleting attributs' or 'using my process'

[Kyle Waid (midwest)]

Hi, Thanks for a work around. I dont quite understand the problem. When I empty some tables like attributes, mappings, and re-import it didnt fix it it was still broken. Im not sure I tried everything you outline here. I would like to understand why this is a problem. There is no way to patch this in 6.1?