The process by which files are acquired from remote and local data sources is prone to falling down about once per 1-2 weeks, requiring user intervention to enable a restart. The download process should be re-written in order to implement a number of significant changes which will improve the stability. These changes are
(1) implementation of a proper database system to track files ingested from remote and local systems.
(2) ensuring that the pulling process is easily configurable at locations other than GSFC.
(3) a more generalized search for the instrument types and measurements from those instruments, so that the instruments and measurements are defined by what is available at the data source
(4) more flexible launching of pull processes via simple command line options
(5) enabling simpler back-filling processes for missing data.
(6) simpler logging and notification when a pull process falls over, or is no longer acquiring data regularly
(7) Monitoring webpages so that summary information on data pulling can be easily accessed by everyone.
Implementation of these requirements will result in a more stable and useful set of tools to pull data from local and remote data sources.
This capability has now been absorbed by helioviewer.org branch.