First getDataSources query after new data is added becomes very slow

[Keith Hughitt]

This issue only appears once you have several million records in the database. What happens is that the back-end currently checks each known datasource to make sure that there is some data available for that datasource. This is done using a SQL COUNT() statement. Each time new data is added to the database, however, the cached results of the relatively slow COUNT statements are discarded, and so the queries must be re-done.

The original purpose of this functionality is to make it so that the Helioviewer.org administrator does not have to be concerned with manually specifying which data sources they have data for; Helioviewer handles that logic.

The solution that will be implemented instead is to add an "enabled" field to the datasource table, which defaults to false. Then, as new data is added, whenever a new datasource is encounterd, the enabled field for the relevant datasource will be set to true. This way no COUNT's are needed to determine data availability at run-time.

[Keith Hughitt (keith-hughitt)]

Moved to 2.2.0 release due to time limitations.