First getDataSources query after new data is added becomes very slow

Bug #666756 reported by Keith Hughitt on 2010-10-26
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Helioviewer.org
High
Keith Hughitt

Bug Description

This issue only appears once you have several million records in the database. What happens is that the back-end currently checks each known datasource to make sure that there is some data available for that datasource. This is done using a SQL COUNT() statement. Each time new data is added to the database, however, the cached results of the relatively slow COUNT statements are discarded, and so the queries must be re-done.

The original purpose of this functionality is to make it so that the Helioviewer.org administrator does not have to be concerned with manually specifying which data sources they have data for; Helioviewer handles that logic.

The solution that will be implemented instead is to add an "enabled" field to the datasource table, which defaults to false. Then, as new data is added, whenever a new datasource is encounterd, the enabled field for the relevant datasource will be set to true. This way no COUNT's are needed to determine data availability at run-time.

Changed in helioviewer.org:
status: In Progress → Fix Committed
Changed in helioviewer.org:
status: Fix Committed → Fix Released
Changed in helioviewer.org:
milestone: 2.1.1 → 2.2.0
Keith Hughitt (keith-hughitt) wrote :

Moved to 2.2.0 release due to time limitations.

To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers