(OXFAM GB Prod) Admin: Ascii codec decode error in upload_users_csv

Bug #614398 reported by Jim B. Glenn
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
KARL3
Fix Released
Low
Tres Seaver

Bug Description

Reported via error monitor:

ENTRY
Fri Aug 6 10:55:57 2010
Exception when processing https://karl.oxfam.org.uk/upload_users_csv.html
Traceback (most recent call last):
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/errorlog.py", line 15, in middleware
    return app(environ, start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.zodbconn-0.11-py2.5.egg/repoze/zodbconn/connector.py", line 21, in __call__
    result = self.next_app(environ, start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.retry-0.9.4-py2.5.egg/repoze/retry/__init__.py", line 88, in __call__
    app_iter = self.application(environ, replace_start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.tm2-1.0a5-py2.5.egg/repoze/tm/__init__.py", line 23, in __call__
    result = self.application(environ, save_status_and_headers)
  File "/opt/karl/oxfam/3.4.6-1/eggs/Paste-1.7.2-py2.5.egg/paste/registry.py", line 350, in __call__
    app_iter = self.application(environ, start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.who-1.0.15-py2.5.egg/repoze/who/middleware.py", line 107, in __call__
    app_iter = app(environ, wrapper.wrap_start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.browserid-0.3-py2.5.egg/repoze/browserid/middleware.py", line 127, in __call__
    return self.app(environ, start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/router.py", line 130, in __call__
    response = view_callable(context, request)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/configuration.py", line 1753, in _secured_view
    return view(context, request)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/configuration.py", line 1663, in _bfg_class_view
    response = inst()
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/views/admin.py", line 580, in __call__
    profiles[username] = profile
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.folder-0.4-py2.5.egg/repoze/folder/__init__.py", line 141, in __setitem__
    return self.add(name, other)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.folder-0.4-py2.5.egg/repoze/folder/__init__.py", line 176, in add
    objectEventNotify(ObjectAddedEvent(other, self, name))
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/event.py", line 33, in objectEventNotify
    adapters = zope.component.subscribers((event.object, event), None)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/_api.py", line 138, in subscribers
    return sitemanager.subscribers(objects, interface)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/registry.py", line 315, in subscribers
    return self.adapters.subscribers(objects, provided)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.interface-3.5.1-py2.5-linux-x86_64.egg/zope/interface/adapter.py", line 535, in subscribers
    subscription(*objects)
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/models/subscribers.py", line 62, in index_content
    catalog.index_doc(docid, node)
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/models/catalog.py", line 40, in index_doc
    super(CachingCatalog, self).index_doc(*arg, **kw)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.catalog-0.7.0-py2.5.egg/repoze/catalog/catalog.py", line 31, in index_doc
    index.index_doc(docid, obj)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.catalog-0.7.0-py2.5.egg/repoze/catalog/indexes/common.py", line 20, in index_doc
    value = self.discriminator(object, _marker)
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/models/site.py", line 157, in get_textrepr
    adapter = queryAdapter(object, ITextIndexData)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/_api.py", line 104, in queryAdapter
    return adapter_hook(interface, object, name, default)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/hookable.py", line 33, in __call__
    return self.__implementation(*args, **kw)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/_api.py", line 158, in adapter_hook
    return sitemanager.queryAdapter(object, interface, name, default)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/registry.py", line 222, in queryAdapter
    return self.adapters.queryAdapter(object, interface, name, default)
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/models/profile.py", line 180, in profile_textindexdata
    text.append(unicode(v))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 20: ordinal not in range(128)

Revision history for this message
Paul Everitt (paul-agendaless) wrote :

Rossi has gotten a bunch of these monitor-reported little bugs, time to triage a little and give some to Tres.

Changed in karl3:
assignee: nobody → Tres Seaver (tseaver)
importance: Undecided → Low
milestone: none → m45
Revision history for this message
Jim B. Glenn (jimbglenn) wrote :
Download full text (43.9 KiB)

More tracebacks from error monitor for this issue:

ENTRY
Fri Aug 6 11:18:28 2010
Exception when processing https://karl.oxfam.org.uk/upload_users_csv.html
Traceback (most recent call last):
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/errorlog.py", line 15, in middleware
    return app(environ, start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.zodbconn-0.11-py2.5.egg/repoze/zodbconn/connector.py", line 21, in __call__
    result = self.next_app(environ, start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.retry-0.9.4-py2.5.egg/repoze/retry/__init__.py", line 88, in __call__
    app_iter = self.application(environ, replace_start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.tm2-1.0a5-py2.5.egg/repoze/tm/__init__.py", line 23, in __call__
    result = self.application(environ, save_status_and_headers)
  File "/opt/karl/oxfam/3.4.6-1/eggs/Paste-1.7.2-py2.5.egg/paste/registry.py", line 350, in __call__
    app_iter = self.application(environ, start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.who-1.0.15-py2.5.egg/repoze/who/middleware.py", line 107, in __call__
    app_iter = app(environ, wrapper.wrap_start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.browserid-0.3-py2.5.egg/repoze/browserid/middleware.py", line 127, in __call__
    return self.app(environ, start_response)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/router.py", line 130, in __call__
    response = view_callable(context, request)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/configuration.py", line 1753, in _secured_view
    return view(context, request)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/configuration.py", line 1663, in _bfg_class_view
    response = inst()
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/views/admin.py", line 580, in __call__
    profiles[username] = profile
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.folder-0.4-py2.5.egg/repoze/folder/__init__.py", line 141, in __setitem__
    return self.add(name, other)
  File "/opt/karl/oxfam/3.4.6-1/eggs/repoze.folder-0.4-py2.5.egg/repoze/folder/__init__.py", line 176, in add
    objectEventNotify(ObjectAddedEvent(other, self, name))
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/event.py", line 33, in objectEventNotify
    adapters = zope.component.subscribers((event.object, event), None)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/_api.py", line 138, in subscribers
    return sitemanager.subscribers(objects, interface)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.component-3.6.0-py2.5.egg/zope/component/registry.py", line 315, in subscribers
    return self.adapters.subscribers(objects, provided)
  File "/opt/karl/oxfam/3.4.6-1/eggs/zope.interface-3.5.1-py2.5-linux-x86_64.egg/zope/interface/adapter.py", line 535, in subscribe
rs
    subscription(*objects)
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/models/subscribers.py", line 62, in index_content
    catalog.index_doc(docid, node)
  File "/opt/karl/oxfam/3.4.6-1/src/karl/karl/models/catalog.py", line 40, in index_doc
    super(CachingCatalog, self).index_doc(*arg,...

Revision history for this message
Jim B. Glenn (jimbglenn) wrote :

More errors

Tres Seaver (tseaver)
Changed in karl3:
status: New → In Progress
Revision history for this message
Tres Seaver (tseaver) wrote :

Fix and tests committed to trunk in r6000.

Changed in karl3:
status: In Progress → Fix Committed
Revision history for this message
Tres Seaver (tseaver) wrote : Re: [Bug 614398] Re: (OXFAM GB Prod) Admin: Ascii codec decode error in upload_users_csv

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

 status inprogress

I need to double check on this one -- the fix is really papering over
the problem that the CSV loader is somehow assigning encoded strings as
attributes, when it should be assigning unicode.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkxkBzAACgkQ+gerLs4ltQ4IhQCgwdh5GyXZgd/4Qq+SZ8bzaDUQ
w+AAn1R7o8Nl5+pHIHSmLgXVsNjXiH+G
=HcGq
-----END PGP SIGNATURE-----

Changed in karl3:
status: Fix Committed → In Progress
Revision history for this message
Tres Seaver (tseaver) wrote :

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

 status fixcommitted

I have added tests and decode scrubbing to the admin view for uploading
users as CSV data, which should prevent the root cause of the bug from
reappearing.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkxkCvcACgkQ+gerLs4ltQ5kYQCgsxM9z2cztLro4IgKj+Efu40k
JhEAoJUVTmo/VPmO2E3Ajeo+KLL3UEjh
=tPSL
-----END PGP SIGNATURE-----

Tres Seaver (tseaver)
Changed in karl3:
status: In Progress → Fix Committed
Revision history for this message
JimPGlenn (jpglenn09) wrote :

fixed

Changed in karl3:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Bug attachments

Remote bug watches

Bug watches keep track of this bug in other bug trackers.