Multiple problems with thread safe.

[Ivan]

Hi, all.

I've an error on highload uwsgi.

Maybe initialize_globals() should lock threads.

There is single Store object for each thread, according docs:
https://storm.canonical.com/Manual#Working_with_multiple_threads

  File "./community/views/components/sharing.py", line 33, in process
    twitter_profile = store.get(StTwitterProfile, (context_id, site.owner_id))

  File "./.virtualenvs/local/lib/python2.7/site-packages/storm/store.py",
line 181, in get
    variable = column.variable_factory(value=variable)

RuntimeError: initialize_globals() failed the first time it was run

[Ivan (ivzak)]

The problem is stably reproduced on reload uwsgi.

[Ivan (ivzak)]

There are also multiple treadsafe bugs for global dicts and other global objects.

For example, create simple uwsgi application with 5 processes and 5 threads. Run ab util:

ab -n 100 -c 5 -C sessionid='123' https://myhost.loc/

And in process try reload uwsgi. Concurrent threads can reset global dicts and attributes.

Modification of global data should lock other threads.

The most often error is:

  File ".virtualenvs/local/lib/python2.7/site-packages/storm/properties.py", line 60, in __get__
    return obj_info.variables[column].get()

KeyError: <storm.properties.PropertyColumn object at 0x7fe6b141dad0>

[Ivan (ivzak)]

The temporary solution to fix "RuntimeError: initialize_globals() failed the first time it was run" is execute on module level next code:

# Under imp.acquire_lock in django module loader
# see AppCache._populate()
# Fixed bug RuntimeError: initialize_globals() failed the first time it was run
# Create first Variable instance with lock threads.
from storm.variables import Variable
Variable()

Now working well.

[Ivan (ivzak)]

Too many bags for now on reload uwsgi, like this:

  File ".virtualenvs/local/lib/python2.7/site-packages/storm/properties.py", line 60, in __get__
    return obj_info.variables[column].get()

KeyError: <storm.properties.PropertyColumn object at 0x7fe6b141dad0>

[Ivan (ivzak)]

Also too many bugs on uwsgi reload like this:

File "./community/views/renderers.py", line 325, in <setcomp>
    has_twitter = {i.user_id for i in store.find(

  File ".virtualenvs/local/lib/python2.7/site-packages/storm/store.py", line 996, in __iter__
    yield self._load_objects(result, values)

  File ".virtualenvs/local/lib/python2.7/site-packages/storm/store.py", line 989, in _load_objects
    return self._find_spec.load_objects(self._store, result, values)

  File ".virtualenvs/local/lib/python2.7/site-packages/storm/store.py", line 1734, in load_objects
    values[values_start:values_end])

  File ".virtualenvs/local/lib/python2.7/site-packages/storm/store.py", line 714, in _load_object
    values, keep_defined=True)

  File ".virtualenvs/local/lib/python2.7/site-packages/storm/store.py", line 766, in _set_values
    variable = obj_info.variables[column]

KeyError: <storm.properties.PropertyColumn object at 0x7f2c98b35830>

[Ivan (ivzak)]

There is a code to reproduce bug:

from storm_ext import stores
from users.orm import TwitterProfile as StTwitterProfile

def t():
    tp = stores['default_slave'].get(StTwitterProfile, (0, 1))
    a = tp.provider_user_name

for i in range(100):
    threading.Thread(target=t).start()

It's thread-safe bug:

Exception in thread Thread-995:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "seeds/storm_threads.py", line 22, in t
    print tp.provider_user_name
  File "/mnt/local/home/zakrevskyi/.virtualenvs/rebelmouse/local/lib/python2.7/site-packages/storm/properties.py", line 60, in __get__
    return obj_info.variables[column].get()
KeyError: <storm.properties.PropertyColumn object at 0x7f0657ecef30>

Exception in thread Thread-998:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "seeds/storm_threads.py", line 22, in t
    print tp.provider_user_name
  File "/mnt/local/home/zakrevskyi/.virtualenvs/rebelmouse/local/lib/python2.7/site-packages/storm/properties.py", line 60, in __get__
    return obj_info.variables[column].get()
KeyError: <storm.properties.PropertyColumn object at 0x7f0657ecef30>

Exception in thread Thread-999:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "seeds/storm_threads.py", line 22, in t
    print tp.provider_user_name
  File "/mnt/local/home/zakrevskyi/.virtualenvs/rebelmouse/local/lib/python2.7/site-packages/storm/properties.py", line 60, in __get__
    return obj_info.variables[column].get()
KeyError: <storm.properties.PropertyColumn object at 0x7f0657ecef30>

Exception in thread Thread-1000:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "seeds/storm_threads.py", line 22, in t
    print tp.provider_user_name
  File "/mnt/local/home/zakrevskyi/.virtualenvs/rebelmouse/local/lib/python2.7/site-packages/storm/properties.py", line 60, in __get__
    return obj_info.variables[column].get()
KeyError: <storm.properties.PropertyColumn object at 0x7f0657ecef30>

[Ivan (ivzak)]

You can see, that <storm.properties.PropertyColumn object at 0x7fb4c1652050> exists in StTwitterProfile._storm_columns but not exists in tp.__storm_object_info__.variables

ipdb> StTwitterProfile._storm_columns
ipdb> {<storm.properties.Int object at 0x7fb4c1e99810>: <storm.properties.PropertyColumn object at 0x7fb4c1e839f0>,
<storm.properties.Unicode object at 0x7fb4c1e99b10>: <storm.properties.PropertyColumn object at 0x7fb4c1e83de0>,
<storm.properties.Int object at 0x7fb4c1e99a90>: <storm.properties.PropertyColumn object at 0x7fb4c1e83c90>,
<storm.properties.Unicode object at 0x7fb4c1e998d0>: <storm.properties.PropertyColumn object at 0x7fb4c1652050>,
<storm.properties.Int object at 0x7fb4c1e997d0>: <storm.properties.PropertyColumn object at 0x7fb4c1e83d00>,
<storm.properties.Unicode object at 0x7fb4c1e99910>: <storm.properties.PropertyColumn object at 0x7fb4c1e83bb0>,
<storm.properties.Int object at 0x7fb4c1e99950>: <storm.properties.PropertyColumn object at 0x7fb4c1e83f30>,
<storm.properties.Int object at 0x7fb4c1e99890>: <storm.properties.PropertyColumn object at 0x7fb4c1e83e50>,
<storm.properties.DateTime object at 0x7fb4c1e99990>: <storm.properties.PropertyColumn object at 0x7fb4c1e83c20>,
<storm.properties.Unicode object at 0x7fb4c1e99a10>: <storm.properties.PropertyColumn object at 0x7fb4c1e83fa0>,
<storm.properties.DateTime object at 0x7fb4c1e999d0>: <storm.properties.PropertyColumn object at 0x7fb4c1e83d70>}

ipdb> StTwitterProfile.__storm_class_info__.columns
ipdb> (<storm.properties.PropertyColumn object at 0x7fb4c1e83bb0>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83c20>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83c90>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83d00>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83d70>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83de0>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83e50>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83ec0>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83f30>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83fa0>,
<storm.properties.PropertyColumn object at 0x7fb4c1e839f0>)

ipdb> tp.__storm_object_info__.variables
ipdb> {<storm.properties.PropertyColumn object at 0x7fb4c1e83c20>: <storm.variables.DateTimeVariable object at 0x7fb47012d5d0>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83e50>: <storm.variables.IntVariable object at 0x7fb47013e758>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83c90>: <storm.variables.IntVariable object at 0x7fb47013e5f0>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83ec0>: <storm.variables.UnicodeVariable object at 0x7fb47013e7d0>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83d00>: <storm.variables.IntVariable object at 0x7fb47013e668>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83f30>: <storm.variables.IntVariable object at 0x7fb47013e848>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83d70>: <storm.variables.DateTimeVariable object at 0x7fb47012d650>,
<storm.properties.PropertyColumn object at 0x7fb4c1e83fa0>: <storm.variables.UnicodeVariable object at 0x7fb47013e8c0>,
<storm.properties.PropertyColumn object a...

[Ivan (ivzak)]

Property._get_column() should lock threads during creating column, when column does not exists in cls.__dict__["_storm_columns"]

[Ivan (ivzak)]

Maybe class Storm as parent for models can resolve problem KeyError: http://bazaar.launchpad.net/~storm/storm/trunk/view/469/storm/properties.py#L303

[Ivan (ivzak)]

See how resolve similar problem SQLAlchemy: https://github.com/zzzeek/sqlalchemy/blob/b63aae2c232f980a47aa2a635c35dfa45390f451/lib/sqlalchemy/orm/mapper.py#L620