udd.sql.gz should get cached once downloaded and should update only when new data gets added or if necessary

[Bhavani Shankar]

Dear Team,

Reg the subject above, It would be beneficial if the udd data thats gets downloaded on get-udd-data and update when new data gets generated in the udd as it would help people running on slow connections.

As of now, the data gets downloaded freshly everytime.

bhavani@bhavani-spagetti-monster:~/d-a-t/dat-overview/overview$ ./manage.py syncdb && ./manage.py get-udd-data
Creating tables ...
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)

bhavani@bhavani-spagetti-monster:~/d-a-t/dat-overview/overview/contributors/temp_data$ ll
total 2854236
drwxrwxr-x 2 bhavani bhavani 4096 Mar 4 19:33 ./
drwxrwxr-x 3 bhavani bhavani 4096 Mar 1 20:56 ../
-rw-rw-r-- 1 bhavani bhavani 160857050 Mar 1 19:52 new-udd.sql
-rw-rw-r-- 1 bhavani bhavani 2228386008 Mar 1 19:38 udd.sql
-rw-rw-r-- 1 bhavani bhavani 476725248 Mar 4 20:49 udd.sql.gz
-rw-rw-r-- 1 bhavani bhavani 56745984 Feb 26 21:30 udd.sql.gz.1
bhavani@bhavani-spagetti-monster:~/d-a-t/dat-overview/overview/contributors/temp_data$ ll
total 2885316
drwxrwxr-x 2 bhavani bhavani 4096 Mar 4 19:33 ./
drwxrwxr-x 3 bhavani bhavani 4096 Mar 1 20:56 ../
-rw-rw-r-- 1 bhavani bhavani 160857050 Mar 1 19:52 new-udd.sql
-rw-rw-r-- 1 bhavani bhavani 2228386008 Mar 1 19:38 udd.sql
-rw-rw-r-- 1 bhavani bhavani 508551168 Mar 4 20:57 udd.sql.gz
-rw-rw-r-- 1 bhavani bhavani 56745984 Feb 26 21:30 udd.sql.gz.1

I merged the latest version of the code while observing this.

Regards,
Bhavani

[Andrew Starr-Bochicchio (andrewsomething)]

This comment from bug #1188634 should help alleviate this.

https://bugs.launchpad.net/dat-overview/+bug/1188634/comments/1