Prefiltering sources in all calculators

[Michele Simionato]

The event based calculator prefilters the source models in the method initialize_sources. This has big performance benefits in the case of small region of interests, since only the relevant sources are sent to the workers via rabbitmq. The same is true for all calculators so it makes sense to extend the approach. However special care must be taken for the cases with 100,000+ sources and 100,000+ points, because the filtering time may become the dominant factor of the computation.

[Michele Simionato (michele-simionato)]

https://github.com/gem/oq-engine/pull/1340

[Michele Simionato (michele-simionato)]

I have decided to implement the following strategy: if there are few sites (i.e. less than 10,000 sites) apply the prefiltering in the method initialize_sources, otherwise filter in the workers. Even in the largest source models we have (250,000+ sites) the prefiltering takes less than 10 minutes for 10,000 points, so it is fast and can be done on a single core. On the other hand for lots of sites the prefiltering could take hours (I tried with the USA model and 150,000 points) so it cannot be enabled.