Profile pictures cannot be found by catalog path

[Jason Lantz]

While working on the metadata extraction script for usage reporting, I ran into an error with using find_model on profile pictures using the path from the address_to_docid index of the catalog. The code below run in debug will highlight the problem. If you want to see the individual traceback, just remove the try/except.

>>> from repoze.bfg.traversal import find_model
>>> for path, docid in root.catalog.document_map.address_to_docid.items():
... try:
... obj = find_model(root, path)
... except:
... print 'Failed to find model at path: %s' % path
...
Failed to find model at path: /profiles/Anya%20Schiffrin/photo.jpg
Failed to find model at path: /profiles/Daniel%20Domagala/photo.jpg
Failed to find model at path: /profiles/David%20MacDonald/photo.jpg
...

[Chris Rossi (chris-archimedeanco)]

Rescheduled to this week.

[Paul Everitt (paul-agendaless)]

Let's hand this one over to Shane.

[Shane Hathaway (shane-hathawaymix)]

I believe these paths were caused by the fix_userids script when it was not updating the catalog's document_map. If so, we just need to pull out the old paths in debug mode.

[Shane Hathaway (shane-hathawaymix)]

The script below, which I tested on kdi-dev, will fix the issue. It turns out this problem still exists in the fix_userids script, so the long-term fix (if we need it) is to fix the fix_userids script.

def fix_photo_map():
    from repoze.bfg.traversal import model_path
    docid_to_address = root.catalog.document_map.docid_to_address
    address_to_docid = root.catalog.document_map.address_to_docid
    for profile in root['profiles'].values():
        for obj in profile.values():
            docid = getattr(obj, 'docid', None)
            if docid is not None:
                new_path = model_path(obj)
                old_path = docid_to_address.get(docid)
                if old_path != new_path:
                    docid_to_address[docid] = new_path
                    address_to_docid[new_path] = docid
                    print 'fixed %s' % new_path

def clean_document_map():
    docid_to_address = root.catalog.document_map.docid_to_address
    address_to_docid = root.catalog.document_map.address_to_docid
    for path, docid in list(address_to_docid.items()):
        correct_path = docid_to_address.get(docid)
        if path != correct_path:
            del address_to_docid[path]
            print 'removed %s' % path

fix_photo_map()
clean_document_map()
import transaction; transaction.commit()

[Shane Hathaway (shane-hathawaymix)]

... and then we need to run 'bin/reindex_catalog -p /profiles'.

[Shane Hathaway (shane-hathawaymix)]

Data fix applied on staging and in production on 2009 July 16.