sync-index stops at the first failure when a charm is indexed.

Bug #1214328 reported by Abel Deuring on 2013-08-20
4
This bug affects 1 person
Affects Status Importance Assigned to Milestone
charmworld
High
Unassigned

Bug Description

Below is a modified variant of TestCharmSource.test_sync_index_script() that shows the error. (It contains changes from lp:~adeuring/charmworld/1206659-simpler-es-mapping which are not yet merged. Most of the changes from this branch are not relevant for this problem, except of the fact that earlier variants of the test in this branch failed spuriously.)

    def test_sync_index_script(self):
        factory.makeCharm(self.db, owner='a')[0]
        factory.makeCharm(self.db, owner='c')[0]
        # Create a charm which is index-incompatible with the previous one.
        factory.makeCharm(self.db, owner='b',
                          description={'text': 'whatever'})[0]
        handler = self.get_handler('sync-index')
        with patch('charmworld.models.configure_logging'):
            with patch('charmworld.models.get_ini', lambda: INI):
                with force_index_client(self.use_index_client()):
                    sync_index()
            messages = [r.getMessage() for r in handler.buffer]
            self.assertIn('Unable to index charm.', messages)
        self.assertEqual(2, len(self.index_client.api_search()))

The setup: Three charms, those owned by "a" and "c" should be indexed; indexing b's charm will fail. But only c's charm is indexed.

The cause:

class CharmSource:
    def sync_index(self):
        for charm in _find_charms(self.collection):
            yield charm['_id']
            self.index_client.index_charm(charm)

def sync_index():
    """"Entry point for sync-index script."""
    configure_logging()
    logger = logging.getLogger('sync-index')
    charm_source = CharmSource.from_settings(get_ini())
    charm_ids = charm_source.sync_index()
    while True:
        try:
            charm_id = next(charm_ids)
        except StopIteration:
            break
        except Exception:
            logger.exception('Unable to index charm.')
        else:
            logger.info('Syncing %s to Elasticsearch' % charm_id)

_find_charms() returns all charms in descending order of (owner, series, charm_name), so charm "c" is indexed first; when index_client.index_charm(charm) fails for charm "b", CharmSource.sync_index() does not catch the error. It is instead passed to the function sync_index(), where the error is logged; the next attempt to call next(charm_ids) raises StopIteration.

Curtis Hovey (sinzui) on 2013-08-20
Changed in charmworld:
importance: Undecided → High
status: New → Triaged
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers