Charmworld indexes far too many fields due to Elasticsearch dynamic mapping

Bug #1206659 reported by Aaron Bentley on 2013-07-30
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
charmworld
High
Unassigned

Bug Description

Dynamic mapping is Charmworld's default behaviour, but it hurts us because sometimes it gets the type wrong, and the ultimate mapping is gigantic (more than 400k pretty-printed).

The dynamic mappings give is no advantage because we don't search on those fields. We search on specific fields. It should be disabled.

Related branches

Curtis Hovey (sinzui) on 2013-09-16
tags: added: elasticsearch
Curtis Hovey (sinzui) wrote :

Work on this has stalled. The comments in charmworld.search indicate that the mapping can switch to static. I believe the blocking issue is bug 1194907.

On 16.09.2013 17:50, Curtis Hovey wrote:
> Work on this has stalled. The comments in charmworld.search indicate
> that the mapping can switch to static. I believe the blocking issue is
> bug 1194907.
>

I think r393 should have fixed the bug.

Curtis Hovey (sinzui) wrote :

I think bundles are dynamic. I see

        with translate_error():
            try:
                for (name, properties, dynamic) in (
                        (CHARM, charm_properties, False),
                        (BUNDLE, bundle_properties, True)):
                    self._client.put_mapping(self.index_name, name, {
                        name: {
                            'type': 'object',
                            'dynamic': dynamic,
                            'properties': properties,
                        }
                    })

I suspect that we can change bundled to dynamic=False, or remove the opportunity to accidentally pass dynamic=True:

        with translate_error():
            try:
                for (name, properties) in (
                        (CHARM, charm_properties,),
                        (BUNDLE, bundle_properties)):
                    self._client.put_mapping(self.index_name, name, {
                        name: {
                            'type': 'object',
                            'dynamic': False,
                            'properties': properties,
                        }
                    })

Abel Deuring (adeuring) wrote :

Right, bundles still have a dynamic mapping. Sorry I simply forgot about them when I wrote my last comment...

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers