Additional content for documentation

Bug #1167359 reported by Ovais Tariq on 2013-04-10
This bug affects 4 people
Affects Status Importance Assigned to Milestone
MOVED - Percona Playback
Stewart Smith

Bug Description

-- Connection Leaks in Percona Playback
Both the tcpdump plugin and query_log plugin suffer from what is known as connection leaks.

In the case of tcpdump plugin, PP reads the tcpdump file and creates new MySQL threads whenever it meets new connection creation or any data from new connection, and then terminates the connections when connection breaking is parsed from the tcpdump file. But there can be situations when not all tcp packets are in the tcpdump file (it can be due to additional CPU or network load when the tcpdump data was captured). In this case packets that signal connection termination are lost and PP will never kill MySQL thread which can lead to "too many connections" error on the MySQL server.

In the case of query_log plugin, when query_log parser meets new thread_id it creates new MySQL thread and sends queries using that thread. When there is a quit command in the query log file for a particular thread_id, then PP closes the MySQL connection and terminates the thread. But it can be the case when there are threads in query log without quit command may be due to abnormal session termination (connection break due to session inactivity for example). And this will again lead to connection leaks because such connections won't get closed.

-- Accurate Mode
The option --query-log-preserve-query-time is analogue of tcpdump plugin accurate mode. The difference is tcpdump plugin accurate mode preserves both query execution time and delays between queries, but query_log plugin preserves only query execution time. So --query-log-preserve-query-time is not completely accurate. However, if the load on the host where the tcpdump is captured is high then that could lead to dropped packets which will make the tcpdump plugin less accurate.
Note also that accuracy is only with respect to queries executed within a single connection, for example suppose there are two MySQL threads, thread_id 1 and thread_id 2, then in that case accuracy will only deal with executing queries accurately within the context of the threads. So for example if the query log contains an entry first of a select by thread_id 1 and then an insert by thread_id 2, this order will not be guaranteed, and its possible that when PP replays the query log the insert by thread_id 2 is done before the select by thread_id 1. Therefore PP can only be used for load testing and cannot be used for functional testing as there is no concept of global accuracy.

-- Format of tcpdump capture
I think it should also be emphasized that its necessary to capture tcpdump in raw format, i.e. by using the -w option, if the capture is not done in raw format then PP cannot work with it. This is different for example from the type of tcpdump capture that pt-query-digest expects.

Related branches

Changed in percona-playback:
status: New → Triaged
importance: Undecided → Medium
assignee: nobody → Hrvoje Matijakovic (hrvojem)
Stewart Smith (stewart) on 2013-10-07
Changed in percona-playback:
milestone: none → 0.8
Stewart Smith (stewart) on 2013-10-07
Changed in percona-playback:
assignee: Hrvoje Matijakovic (hrvojem) → Stewart Smith (stewart)
status: Triaged → Fix Committed
Stewart Smith (stewart) on 2013-10-07
Changed in percona-playback:
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  Edit
Everyone can see this information.

Other bug subscribers