Arg... Was this doing an initial import or just updating?
My quick off hand guess is that it's because I use Uploads.objects.bulk_create(bulk_insert). This speeds things up dramatically as it allows adding a lot of rows to the database while only calling save once instead of saving on each row, but it also means everything is loaded into one variable. Maybe there's a way to do it in batches?
Arg... Was this doing an initial import or just updating?
My quick off hand guess is that it's because I use Uploads. objects. bulk_create( bulk_insert) . This speeds things up dramatically as it allows adding a lot of rows to the database while only calling save once instead of saving on each row, but it also means everything is loaded into one variable. Maybe there's a way to do it in batches?