for example, if you use --limit 1000 and --sleep 60 instead of fetch 1000 rows and then sleep 60 seconds, it sleeps 60 seconds every row. So for example purge processes needs lot of time:
# pt_archiver:4201 2597 Got another row in this chunk
# pt_archiver:4218 2597 Sleeping 5
# pt_archiver:4201 2597 Got another row in this chunk
# pt_archiver:4218 2597 Sleeping 5
# pt_archiver:4201 2597 Got another row in this chunk
# pt_archiver:4218 2597 Sleeping 5
# pt_archiver:4201 2597 Got another row in this chunk
# pt_archiver:4218 2597 Sleeping 5
# pt_archiver:4112 2597 No more rows in this chunk; doing bulk operations
# pt_archiver:4165 2597 Bulk deleted 5 rows
So, it deletes data in groups of 5 rows, but it stops after every fetched row. If I remove the sleep option:
# pt_archiver:4201 2602 Got another row in this chunk
# pt_archiver:4201 2602 Got another row in this chunk
# pt_archiver:4201 2602 Got another row in this chunk
# pt_archiver:4201 2602 Got another row in this chunk
# pt_archiver:4112 2602 No more rows in this chunk; doing bulk operations
# pt_archiver:4165 2602 Bulk deleted 5 rows
To solve the problem I've moved the sleep section of the code to the end of the "This code is for the bulk archiving functionality" section.
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4112 2110 No more rows in this chunk; doing bulk operations
# pt_archiver:4165 2110 Bulk deleted 5 rows
# pt_archiver:4179 2110 Sleeping 5
# pt_archiver:4191 2110 Fetching rows in next chunk
# pt_archiver:4196 2110 Fetched 5 rows
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4112 2110 No more rows in this chunk; doing bulk operations
# pt_archiver:4165 2110 Bulk deleted 5 rows
# pt_archiver:4179 2110 Sleeping 5
for example, if you use --limit 1000 and --sleep 60 instead of fetch 1000 rows and then sleep 60 seconds, it sleeps 60 seconds every row. So for example purge processes needs lot of time:
pt-archiver \ D=test, t=t,u=root \ tmp/mk- archiver- sessions. pid
--source h=localhost,
--purge \
--where 'i > 1' \
--limit 5 \
--sleep 5 \
--bulk-delete \
--pid=/
# pt_archiver:4201 2597 Got another row in this chunk
# pt_archiver:4218 2597 Sleeping 5
# pt_archiver:4201 2597 Got another row in this chunk
# pt_archiver:4218 2597 Sleeping 5
# pt_archiver:4201 2597 Got another row in this chunk
# pt_archiver:4218 2597 Sleeping 5
# pt_archiver:4201 2597 Got another row in this chunk
# pt_archiver:4218 2597 Sleeping 5
# pt_archiver:4112 2597 No more rows in this chunk; doing bulk operations
# pt_archiver:4165 2597 Bulk deleted 5 rows
So, it deletes data in groups of 5 rows, but it stops after every fetched row. If I remove the sleep option:
pt-archiver \ D=test, t=t,u=root \ tmp/mk- archiver- sessions. pid
--source h=localhost,
--purge \
--where 'i > 1' \
--limit 5 \
--bulk-delete \
--pid=/
# pt_archiver:4201 2602 Got another row in this chunk
# pt_archiver:4201 2602 Got another row in this chunk
# pt_archiver:4201 2602 Got another row in this chunk
# pt_archiver:4201 2602 Got another row in this chunk
# pt_archiver:4112 2602 No more rows in this chunk; doing bulk operations
# pt_archiver:4165 2602 Bulk deleted 5 rows
To solve the problem I've moved the sleep section of the code to the end of the "This code is for the bulk archiving functionality" section.
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4112 2110 No more rows in this chunk; doing bulk operations
# pt_archiver:4165 2110 Bulk deleted 5 rows
# pt_archiver:4179 2110 Sleeping 5
# pt_archiver:4191 2110 Fetching rows in next chunk
# pt_archiver:4196 2110 Fetched 5 rows
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4211 2110 Got another row in this chunk
# pt_archiver:4112 2110 No more rows in this chunk; doing bulk operations
# pt_archiver:4165 2110 Bulk deleted 5 rows
# pt_archiver:4179 2110 Sleeping 5