(OXFAM) error in tagusers- keyerror self.data['name']

[Jim B. Glenn]

reported via error monitor:

ENTRY
Fri Oct 22 07:09:45 2010
Exception when processing https://karl.oxfam.org.uk/tagusers.html?tag=faq&docid=
-598179427
Traceback (most recent call last):
  File "/opt/karl/oxfam/3.5.4-0/src/karl/karl/errorlog.py", line 15, in middlewa
re
    return app(environ, start_response)
  File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.zodbconn-0.11-py2.5.egg/repoze/zodbc
onn/connector.py", line 21, in __call__ result = self.next_app(environ, start_response)
  File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.retry-0.9.4-py2.5.egg/repoze/retry/_
_init__.py", line 88, in __call__
    app_iter = self.application(environ, replace_start_response)
  File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.tm2-1.0a5-py2.5.egg/repoze/tm/__init
__.py", line 23, in __call__
    result = self.application(environ, save_status_and_headers)
  File "/opt/karl/oxfam/3.5.4-0/eggs/Paste-1.7.2-py2.5.egg/paste/registry.py", l
ine 350, in __call__
    app_iter = self.application(environ, start_response)
  File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.who-1.0.15-py2.5.egg/repoze/who/midd
leware.py", line 107, in __call__
    app_iter = app(environ, wrapper.wrap_start_response)
  File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.browserid-0.3-py2.5.egg/repoze/brows
erid/middleware.py", line 127, in __call__
    return self.app(environ, start_response)
  File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/router.
py", line 130, in __call__
    response = view_callable(context, request)
  File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/configu
ration.py", line 1753, in _secured_view
    return view(context, request)
  File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/configu
ration.py", line 1704, in _rendered_view
    response = view(context, request)
  File "/opt/karl/oxfam/3.5.4-0/src/karl/karl/views/tags.py", line 406, in tag_u
sers_view
    profile = profiles[userid]
  File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.folder-0.4-py2.5.egg/repoze/folder/_
_init__.py", line 89, in __getitem__
    return self.data[name]
KeyError: u'kuhlermckeown'

several more with different urls for the error above

[Paul Everitt (paul-agendaless)]

Over to Tres for triage. We have several that are KeyError related to profile name, some have already been closed, some are getting reported multiply (another one on its way in a moment.)

[Tres Seaver (tseaver)]

Can somebody confirm that Oxfam actually deleted that user, rather than
marking them 'inactive'? And is that a valid use case? Fixing the code
to work around the missing profile is straightforward, but perhaps we
should be fixing the data instead.

[Paul Everitt (paul-agendaless)]

So far, Rossi and Jim have bin/debug access to data. Some choices:

a) Assign this to Jim and give him the lines you'd like typed into bin/debug to get the answer.

b) Assign this to Rossi to get the answer.

c) Get you access to all the production KARLs so you can get the answer.

My guess is (b) is the most pragmatic.

--Paul

On Oct 26, 2010, at 8:19 AM, Tres Seaver wrote:

> Can somebody confirm that Oxfam actually deleted that user, rather than
> marking them 'inactive'? And is that a valid use case? Fixing the code
> to work around the missing profile is straightforward, but perhaps we
> should be fixing the data instead.
>
> ** Changed in: karl3
> Status: New => Confirmed
>
> --
> (OXFAM) error in tagusers- keyerror self.data['name']
> https://bugs.launchpad.net/bugs/666291
> You received this bug notification because you are subscribed to KARL3.
>
> Status in KARL3: Confirmed
>
> Bug description:
> reported via error monitor:
>
> ENTRY
> Fri Oct 22 07:09:45 2010
> Exception when processing https://karl.oxfam.org.uk/tagusers.html?tag=faq&docid=
> -598179427
> Traceback (most recent call last):
> File "/opt/karl/oxfam/3.5.4-0/src/karl/karl/errorlog.py", line 15, in middlewa
> re
> return app(environ, start_response)
> File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.zodbconn-0.11-py2.5.egg/repoze/zodbc
> onn/connector.py", line 21, in __call__ result = self.next_app(environ, start_response)
> File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.retry-0.9.4-py2.5.egg/repoze/retry/_
> _init__.py", line 88, in __call__
> app_iter = self.application(environ, replace_start_response)
> File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.tm2-1.0a5-py2.5.egg/repoze/tm/__init
> __.py", line 23, in __call__
> result = self.application(environ, save_status_and_headers)
> File "/opt/karl/oxfam/3.5.4-0/eggs/Paste-1.7.2-py2.5.egg/paste/registry.py", l
> ine 350, in __call__
> app_iter = self.application(environ, start_response)
> File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.who-1.0.15-py2.5.egg/repoze/who/midd
> leware.py", line 107, in __call__
> app_iter = app(environ, wrapper.wrap_start_response)
> File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.browserid-0.3-py2.5.egg/repoze/brows
> erid/middleware.py", line 127, in __call__
> return self.app(environ, start_response)
> File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/router.
> py", line 130, in __call__
> response = view_callable(context, request)
> File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/configu
> ration.py", line 1753, in _secured_view
> return view(context, request)
> File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.bfg-1.2-py2.5.egg/repoze/bfg/configu
> ration.py", line 1704, in _rendered_view
> response = view(context, request)
> File "/opt/karl/oxfam/3.5.4-0/src/karl/karl/views/tags.py", line 406, in tag_u
> sers_view
> profile = profiles[userid]
> File "/opt/karl/oxfam/3.5.4-0/eggs/repoze.folder-0.4-py2.5.egg/repoze/folder/_
> _init__.py", line 89, in __getitem__
> return self.data[name]
> KeyError: u'kuhlermckeown'
>
> several more with different urls for the error above
>
>

[Tres Seaver (tseaver)]

The immediate data fixup for this bug is to purge users without profiles::

 $ bin/debug
 >>> profiles = root['profiles']
 >>> tags = root.tags
 >>> for userid in tags.getUsers():
 ... if profiles.get(userid) is None:
 ... print 'Removing tags for missing user:', userid
 ... tags.delete(user=userid)

And then, to save any changes::

 >>> import transaction; transaction.commit()

Jim, can you do that and post the output here, please?

[Jim B. Glenn (jimbglenn)]

Hi Tres,

I ran this against STAGING oxfam. check me & i'll be glad to run against oxfam prod:

[oxfam@karlhost01 current]$ bin/debug
Python 2.5.5 (r255:77872, Jun 7 2010, 14:10:57)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2
Type "help" for more information. "root" is the karl root object.
>>> profiles = root['profiles']
>>> tags = root.tags
>>> for userid in tags.getUsers()
  File "<console>", line 1
    for userid in tags.getUsers()

^
SyntaxError: invalid syntax
>>> profiles = root['profiles']
>>> tags = root.tags
>>> for userid in tags.getUsers():
... if profiles.get(userid) is None:
... print 'Removing tags for missing user:', userid
... tags.delete(user=userid)
...
Removing tags for missing user: kuhlermckeown
19
>>> import transaction; transaction.commit()
>>>
>>>

[Tres Seaver (tseaver)]

Looks find, go ahead and run it on production.

[Jim B. Glenn (jimbglenn)]

ran against prod:

[oxfam@karlhost01 current]$ bin/debug
Python 2.5.5 (r255:77872, Jun 7 2010, 14:10:57)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-46)] on linux2
Type "help" for more information. "root" is the karl root object.
>>> profiles = root['profiles']
>>> tags = root.tags
>>> for userid in tags.getUsers():
... if profiles.get(userid) is None:
... print 'Removing tags for missing user:', userid
... tags.delete(user=userid)
...
Removing tags for missing user: kuhlermckeown
19
>>> import transaction; transaction.commit()
>>>
>>> quit()