Percona-Tookit工具包之pt-query-digest

 

Preface

 

    Performance issues are what DBA most concerned thing.There're always a lot of SQL queries which maybe not using appropriated indexes cause bad perfoemance.Whenever that happens,we have to do some performance diagnosis immediately.In most scenarios,we will do that relies on slow log,but it's not enough or efficient I'm afraid.What's the better way then?Another tool of percona called "pt-query-digest" can solve these kind of issues.

 

Introduce

 

    pt-query-digest is a tool focus on log analysis.It is supported to analyze slow log,general log,binlog and so on in different ways.You can also use it to analyze logs and simultaneously put these informations into tables on remote host(server) that you specified.That's really convenient and flexible for us DBA to work efficiently in performance tuning.

 

Procedure

 

Usage

 Usage: pt-query-digest [OPTIONS] [FILES] [DSN]

Parameters introduce

 --limit -- Limit the output conents by count(defualt ) or percentage(default %).
 --type -- Indicate a type(default "slowlog",else value is "genlog","binlog","tcpdump",etc.) you want to anaylze.
 --processlist -- Analyze from "show processlist" result from specific host(need to input DSN).
 --create-history-table -- Create a table to record infomation if use "--history"(default "true").
 --create-review-table -- Create a table to record infomation if use "--review"(default "true").
 --history -- Save query metrics(such as query time) into a given table which can be checked incremental difference later.
 --review -- Save query classes into a given table for later review.It don't report same kind of class.
 --output -- Specify the format of query result(default "report").
 --since -- Give a specific time with time format of beginning.
 --until -- Give a specific time with time format of end.
 --group-by -- According to the atrribute of envents to group by(default "figerprint",else value is "tables"&"distill").
 --order-by -- According to the atrribute of envents to sort by(default "Query_time:sum").

Examples

 

Make sure the slow log is enabled.

 (root@localhost mysql3306.sock)[(none)]::>show variables like '%slow%';
 +---------------------------+----------+
 | Variable_name             | Value    |
 +---------------------------+----------+
 | log_slow_admin_statements | OFF      |
 | log_slow_slave_statements | ON       |
 | slow_launch_time          |         |
 | slow_query_log            | ON       |
 | slow_query_log_file       | slow.log |
 +---------------------------+----------+
  rows in set (0.00 sec)

 (root@localhost mysql3306.sock)[(none)]::>show variables like '%long_query_time%';
 +-----------------+----------+
 | Variable_name   | Value    |
 +-----------------+----------+
 | long_query_time | 1.000000 |
 +-----------------+----------+
  row in set (0.00 sec)

 (root@localhost mysql3306.sock)[(none)]::>show variables like '%log_output%';
 +---------------+-------+
 | Variable_name | Value |
 +---------------+-------+
 | log_output    | FILE  |
 +---------------+-------+
  row in set (0.00 sec)

Execute a slow query.

 (root@localhost mysql3306.sock)[(none)]::>select sleep();
 +-----------+
 | sleep() |
 +-----------+
 |          |
 +-----------+
  row in set ( min 0.01 sec)

Check slow log for information of above slow query.

 [root@zlm2 :: /data/mysql/mysql3306/data]
 #cat slow.log

 # Time: --23T07::.891778Z
 # User@Host: root[root] @ localhost []  Id:
 # Query_time: 60.001239  Lock_time: 0.000000 Rows_sent:   Rows_examined:
 SET timestamp=;
 select sleep();

Use pt-query-digest analyze slow log file for more details.

 [root@zlm2 :: ~]
 #pt-query-digest /data/mysql/mysql3306/data/slow.log

 # No events processed. -- Only if the slow query has finished,there will be a result of report.

 [root@zlm2 :: ~]
 #pt-query-digest /data/mysql/mysql3306/data/slow.log

 # 180ms user time,  system time, 25.59M rss, 221.68M vsz
 # Current date: Sat Jun  ::
 # Hostname: zlm2
 # Files: /data/mysql/mysql3306/data/slow.log
 # Overall:  total,  unique,  QPS, 0x concurrency ______________________
 # Time range: all events occurred at --23T07::
 # Attribute          total     min     max     avg     %  stddev  median
 # ============     ======= ======= ======= ======= ======= ======= =======
 # Exec time            60s     60s     60s     60s     60s            60s
 # Lock time
 # Rows sent
 # Rows examine
 # Query size                                                 

 # Profile
 # Rank Query ID           Response time  Calls R/Call  V/M   Item
 # ==== ================== ============== ===== ======= ===== ======
 #     0xF9A57DD5A41825CA 60.0012 100.0%      60.0012  0.00 SELECT

 # Query :  QPS, 0x concurrency, ID 0xF9A57DD5A41825CA at byte  ________
 # This item is included in the report because it matches --limit.
 # Scores: V/M = 0.00
 # Time range: all events occurred at --23T07::
 # Attribute    pct   total     min     max     avg     %  stddev  median
 # ============ === ======= ======= ======= ======= ======= ======= =======
 # Count
 # Exec time         60s     60s     60s     60s     60s            60s
 # Lock time
 # Rows sent
 # Rows examine
 # Query size
 # String:
 # Hosts        localhost
 # Users        root
 # Query_time distribution
 #   1us
 #  10us
 # 100us
 #   1ms
 #  10ms
 # 100ms
 #    1s
 #  10s+  ################################################################
 # EXPLAIN /*!50100 PARTITIONS*/
 select sleep()\G

 ###All the meaning of above is self-explanatory,you can check official document for details.###

Other expamples

 

1.Analyze general log.

 pt-query-digest --type=genlog /path/zlm2.log > report1.log

2.Analyze binlog.

 mysqlbinlog -vv --base64-output=decode-rows /path/mysql-bin. > mysql-bin000010.sql
 pt-query-digest --type=binlog /path/mysql-bin000001.sql > report2.log

3.Analyze rawlog(It's a general txt file which contains SQL statement).

 echo "select sleep(10);">rawlog.log pt-query-digest --type=rawlog rawlog.log > report3.log

4.Analyze processlist(DSN is indispensable).

 pt-query-digest --processlist h=192.168.1.101,P=,u=repl,p=repl4slave > report4.log -- If connection failed,it will try every second.

5.Analyze tcpdump.

 tcpdump -s  -x -nn -q -tttt -i any -c  port  > tcpdump.log
 pt-query-digest --type=tcpdump tcpdump.log > report5.log

6.Analyze slow log since last 24 hours.

 pt-query-digest --since=24h slow.log > report6.log

7.Analyze slow log since time until time.

 pt-query-digest --since '2018-06-23 08:30:00' --until '2018-06-23 10:30:00' slow.log > report7.log

8.Analyze slow log into view table("query_review" table will be created if not specify "-t") of remote host.

 pt-query-digest --review h=192.168.1.102,P=,u=repl,p=repl4slave,D=zlm,t=query_review slow.log > report8.log

9.Analyze slow log into history table("query_history" table will be created like above) of remote host.

 pt-query-digest --review h=192.168.1.102,P=,u=repl,p=repl4slave,D=zlm,t=query_history slow.log > report9.log

Summay

• There're still some advanced useages I've not demonstrated such as "--group-by","--order-by" and "--filter",etc.
• pt-query-digest is extraordinarily useful when doing performance diagnosis by different ways.
• It's recommended to analyze logs on another node(maybe slave) instead of master to reduce consumption of CPU,Memory,IO,etc.
• pt-query-digest can help you to find out the specific slow quries.Afterwards you can use MySQL profiling("set profiling=1;" then "show profiles;") to survey the perticular resources which are tremendously consumed.