Logging MySQL queries from the client instead of the server

The “slow query log” is the single most valuable way to examine query execution on your MySQL server. Queries are logged with timing information, and in the case of Percona Server, a great deal of additional performance and other diagnostic information. But the execution time recorded in the log is the time the query took on the server, and the client that sent the query to the server might see something quite different. Sometimes it’s valuable to be able to see both views of query execution time.

Why would you want to see the query timing information from the application server or other client program? I have run into a handful of scenarios where this was desirable. For example, sometimes there is no access to the database server. I’ve seen this when access was forbidden by management, and when a server was so overloaded that nobody could get into it if they wanted to. Another reason for logging from the client is to log only selected traffic that is of interest, instead of the potentially large and hard-to-analyze amount of traffic that the server might receive in total, much of which might have nothing to do with the task you’re trying to optimize or analyze. You might also be interested in gathering data that you can’t gather on the server with the slow query log, such as information in the TCP protocol that isn’t written into the standard server’s log. Finally, you might be interested in logging the full round-trip time, including network delay, as seen by the application server.

There are several techniques for logging queries with their execution times. I will list three.

One is to make the application time the queries and write a query log. I suggest writing out something in a standard logging format such as the slow-query-log format, so you can analyze it easily with tools such as Maatkit’s mk-query-digest. These tools can do a lot of work for you. Or, you can log queries to a database table and then use SQL to analyze the queries; but this is quite a bit harder, because similar queries need to be “fingerprinted,” which is impractical or impossible in SQL. The main drawback to making the application itself log the queries is that it might not log everything, such as superfluous ‘ping’ commands.

Another option is to make the client connector log the queries. For example, if you are using PHP with mysqlnd, you can create a plugin that logs the queries, such as the query logging demonstrated here. This is likely to be easier and more reliable than changing the application code itself, especially if queries don’t go through a single point in the code but are issued all over the place.

Finally, you can capture TCP traffic. There are several ways to do this with varying levels of accuracy and completeness; you can use tcpdump and some quick shell commands, for example. But with mk-query-digest’s built-in ability to decode the MySQL wire protocol, it is certainly the easiest and best solution for most needs. Just use tcpdump and the “–type=tcpdump” option to mk-query-digest. You can even make it print out a “slow query log file” format with the “–print” option.