Tables are loaded in a random order, loading large ones first is preferable

[Scott Wheeler]

I noticed that per chance my largest table in a database I'm testing with was being loaded last with myloader, which meant that it spent most of its loading time not benefiting from parallelism.

I added a bit of code to sort the table list by decreasing file size, which ensures that the larger tables are loaded first. (File size isn't a perfect proxy for load time, but it's much better than random.)

Patch attached.

[Scott Wheeler (wheeler)]

Ah, meant to point out that while the results weren't amazingly dramatic for my test case, it did at the end shave about 6% of the load time off of the database, which is worthwhile.

[Max Bubenick (max-bubenick)]

Hi Scott, to improve parallelism with large tables you have the chunking options

  -r, --rows Try to split tables into chunks of this many rows. This option turns off --chunk-filesize
  -F, --chunk-filesize Split tables into chunks of this output file size. This value is in MB