Implement bulk insert using Django
Affects | Status | Importance | Assigned to | Milestone | |
---|---|---|---|---|---|
OpenQuake (deprecated) |
Fix Released
|
Medium
|
Mattia Barbon |
Bug Description
The current bulk insert implementation uses SQLAlchemy; it should be rewritten to use Django ORM.
Depends from https:/
Related to https:/
There isn't a bulk inserter built-in in Django; some third party solutions:
http://
+ very complete and handles relations (not needed for now but nice)
- not maintained (last updated in 2008)
- might be a pain to debug/update
http://
+ simple and basically what we have now
- does not handle relations (but we do not need them ATM)
http://
Similar to snippet 2362 but uses .executemany rather than bulk insert
- executemany is slower than bulk insert
http://
Uses a custom manager so the bulk insert method is available directly in the model class
- I like the separate bulk insert class better, but it's a personal preference
Changed in openquake: | |
status: | New → Confirmed |
assignee: | nobody → Mattia Barbon (mattia.barbon) |
milestone: | none → 0.4.3 |
Changed in openquake: | |
status: | In Progress → Fix Committed |
tags: | added: database performance |
Changed in openquake: | |
status: | Fix Committed → Fix Released |
Pull request https:/ /github. com/gem/ openquake/ pull/440