关于Mysql 的 ICP、MRR、BKA等特性
一、ICP( Index_Condition_Pushdown)
对 where 中过滤条件的处理,根据索引使用情况分成了三种:(何登成)index key, index filter, table filter
如果WHERE条件可以使用索引,MySQL 会把这部分过滤操作放到存储引擎层,存储引擎通过索引过滤,把满足的行从表中读取出。ICP能减少Server层访问存储引擎的次数和引擎层访问基表的次数。
- session级别设置:set optimizer_switch="index_condition_pushdown=on
对于InnoDB表,ICP只适用于辅助索引
当使用ICP优化时,执行计划的Extra列显示Using index condition提示
不支持主建索引的ICP(对于Innodb的聚集索引,完整的记录已经被读取到Innodb Buffer,此时使用ICP并不能降低IO操作)
当 SQL 使用覆盖索引时但只检索部分数据时,ICP 无法使用
ICP的加速效果取决于在存储引擎内通过ICP筛选掉的数据的比例
index_condition_pushdown会大大减少行锁的个数,如select for update, 因为行锁是在引擎层的
例如:
现在的索引
show index from sm_performance_all;
+--------------------+------------+-------------------------------+--------------+----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment |
+--------------------+------------+-------------------------------+--------------+----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
| sm_performance_all | 0 | PRIMARY | 1 | id | A | 40527 | NULL | NULL | | BTREE | | |
| sm_performance_all | 1 | FK_a9t29a4b2af1vfny1j2minc1x | 1 | company_id | A | 316 | NULL | NULL | YES | BTREE | | |
| sm_performance_all | 1 | FK_n3ng4a5qju19fw8qy4uskp4g1 | 1 | bill_id | A | 21532 | NULL | NULL | YES | BTREE | | |
| sm_performance_all | 1 | FK_eb13u3xwslt9t7wwuycg7vha6 | 1 | car_id | A | 16794 | NULL | NULL | YES | BTREE | | |
| sm_performance_all | 1 | FK_2bfhskvklf6mdk557tc3yy3y1 | 1 | commission_entity_id | A | 177 | NULL | NULL | YES | BTREE | | |
| sm_performance_all | 1 | FK_6fr5ib5iyjyu155dncmc48cwr | 1 | member_card_id | A | 34 | NULL | NULL | YES | BTREE | | |
| sm_performance_all | 1 | FK_93p22vcog266wa82i44a6m18b | 1 | user_id | A | 483 | NULL | NULL | YES | BTREE | | |
| sm_performance_all | 1 | FK_p6nc7l6ewnkcpm2y4o3wct81r | 1 | member_card_bill_id | A | 4 | NULL | NULL | YES | BTREE | | |
| sm_performance_all | 1 | billId_userId_memberCarBillId | 1 | bill_id | A | 24194 | NULL | NULL | YES | BTREE | | |
| sm_performance_all | 1 | billId_userId_memberCarBillId | 2 | user_id | A | 25688 | NULL | NULL | YES | BTREE | | |
| sm_performance_all | 1 | billId_userId_memberCarBillId | 3 | member_card_bill_id | A | 25946 | NULL | NULL | YES | BTREE | | |
+--------------------+------------+-------------------------------+--------------+----------------------+-----------+-------------+----------+--------+------+------------+---------+---------------+
11 rows in set (0.00 sec)
现在的语句执行情况
explain select * from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0;
+----+-------------+-------+------------+------+---------------+------+---------+------+-------+----------+-------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+------+-------+----------+-------------+
| 1 | SIMPLE | p | NULL | ALL | NULL | NULL | NULL | NULL | 40527 | 1.11 | Using where |
+----+-------------+-------+------------+------+---------------+------+---------+------+-------+----------+-------------+
1 row in set, 1 warning (0.00 sec)
添加索引后
ALTER TABLE sm_performance_all add index date_created_type(date_created, type );
explain select * from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0;
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+-----------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+-----------------------+
| 1 | SIMPLE | p | NULL | range | date_created_type | date_created_type | 6 | NULL | 1 | 10.00 | Using index condition |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+-----------------------+
1 row in set, 1 warning (0.00 sec)
二、MRR(Multi-Range Read )
随机 IO 转化为顺序 IO 以降低查询过程中 IO 开销的一种手段,这对IO-bound类型的SQL语句性能带来极大的提升。
MRR can be used for InnoDB
and MyISAM
tables for index range scans and equi-join operations.
A portion of the index tuples are accumulated in a buffer.
The tuples in the buffer are sorted by their data row ID.
Data rows are accessed according to the sorted index tuple sequence.
上述的SQL语句需要根据辅助索引date_created_type进行查询,但是由于要求得到的是表中所有的列,因此需要回表进行读取。而这里就可能伴随着大量的随机I/O。这个过程如下图所示:
而MRR的优化在于,并不是每次通过辅助索引就回表去取记录,而是将其rowid给缓存起来,然后对rowid进行排序后,再去访问记录,这样就能将随机I/O转化为顺序I/O,从而大幅地提升性能。这个过程如下所示:
然而,在MySQL当前版本中,基于成本的算法过于保守,导致大部分情况下优化器都不会选择MRR特性。为了确保优化器使用mrr特性,请执行下面的SQL语句:
set optimizer_switch='mrr=on,mrr_cost_based=off';
读取全部字段时
explain select * from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0;
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+
| 1 | SIMPLE | p | NULL | range | date_created_type | date_created_type | 6 | NULL | 1 | 10.00 | Using index condition; Using MRR |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+
1 row in set, 1 warning (0.00 sec)
只读取部分字段时:
读取外键 explain select car_id from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0;
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+
| 1 | SIMPLE | p | NULL | range | date_created_type | date_created_type | 6 | NULL | 1 | 10.00 | Using index condition; Using MRR |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+----------------------------------+
1 row in set, 1 warning (0.00 sec)
读取主键
explain select id from sm_performance_all p where p.date_created>'2018-01-01' and p.date_created< '2018-02-01' and p.type=0;
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+--------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+--------------------------+
| 1 | SIMPLE | p | NULL | range | date_created_type | date_created_type | 6 | NULL | 1 | 10.00 | Using where; Using index |
+----+-------------+-------+------------+-------+-------------------+-------------------+---------+------+------+----------+--------------------------+
1 row in set, 1 warning (0.00 sec)
For MRR, a storage engine uses the value of the read_rnd_buffer_size
system variable as a guideline for how much memory it can allocate for its buffer.
默认256KB
show GLOBAL VARIABLES like '%buffer_size';
+-------------------------+----------+
| Variable_name | Value |
+-------------------------+----------+
| bulk_insert_buffer_size | 8388608 |
| innodb_log_buffer_size | 16777216 |
| innodb_sort_buffer_size | 1048576 |
| join_buffer_size | 262144 |
| key_buffer_size | 8388608 |
| myisam_sort_buffer_size | 8388608 |
| preload_buffer_size | 32768 |
| read_buffer_size | 131072 |
| read_rnd_buffer_size | 262144 |
| sort_buffer_size | 262144 |
+-------------------------+----------+
10 rows in set (0.00 sec)
三、表连接实现方式
3.1 Nested Loop Join
将驱动表/外部表的结果集作为循环基础数据,然后循环该结果集,每次获取一条数据作为下一个表的过滤条件查询数据,然后合并结果,获取结果集返回给客户端。Nested-Loop一次只将一行传入内层循环, 所以外层循环(的结果集)有多少行, 内存循环便要执行多少次,效率非常差。
EXPLAIN SELECT * from sm_performance_all p LEFT JOIN sm_bill b ON p.bill_id > b.car_id where p.company_id>1024;
+----+-------------+-------+------------+-------+------------------------------+------------------------------+---------+------+--------+----------+------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+-------+------------------------------+------------------------------+---------+------+--------+----------+------------------------------------------------+
| 1 | SIMPLE | p | NULL | range | FK_a9t29a4b2af1vfny1j2minc1x | FK_a9t29a4b2af1vfny1j2minc1x | 9 | NULL | 20263 | 100.00 | Using index condition; Using MRR |
| 1 | SIMPLE | b | NULL | ALL | car_id_idx | NULL | NULL | NULL | 738383 | 100.00 | Range checked for each record (index map: 0x2) |
+----+-------------+-------+------------+-------+------------------------------+------------------------------+---------+------+--------+----------+------------------------------------------------+
2 rows in set, 1 warning (0.00 sec)
3.2 Block Nested-Loop Join
将外层循环的行/结果集存入join buffer, 内层循环的每一行与整个buffer中的记录做比较,从而减少内层循环的次数。主要用于当被join的表上无索引。
CREATE TABLE t1 (a int PRIMARY KEY, b int);
CREATE TABLE t2 (a int PRIMARY KEY, b int);
INSERT INTO t1 VALUES (1,2), (2,1), (3,2), (4,3), (5,6), (6,5), (7,8), (8,7), (9,10);
INSERT INTO t2 VALUES (3,0), (4,1), (6,4), (7,5); EXPLAIN
SELECT * FROM t1 LEFT JOIN t2 ON t1.a = t2.a WHERE t2.b <= t1.a AND t1.a <= t1.b; +----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+
| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+
| 1 | SIMPLE | t1 | NULL | ALL | PRIMARY | NULL | NULL | NULL | 9 | 33.33 | Using where |
| 1 | SIMPLE | t2 | NULL | ALL | PRIMARY | NULL | NULL | NULL | 4 | 25.00 | Using where; Using join buffer (Block Nested Loop) |
+----+-------------+-------+------------+------+---------------+------+---------+------+------+----------+----------------------------------------------------+
2 rows in set, 1 warning (0.00 sec)
3.3 Batched Key Access
当被join的表能够使用索引时,就先好顺序,然后再去检索被join的表。对这些行按照索引字段进行排序,因此减少了随机IO。如果被Join的表上没有索引,则使用老版本的BNL策略。
参考:
mysql reference : multi-range read
关于Mysql 的 ICP、MRR、BKA等特性的更多相关文章
- MySQL Index--BNL/ICP/MRR/BKA
MySQL关联查询算法: BNL(Block Nested-Loop) ICP(Index Condition Pushdown) MRR(Multi-Range Read) BKA(Batched ...
- MySQL--BNL/ICP/MRR/BKA
#======================================================##MySQL关联查询算法:BNL(Block Nested-Loop)ICP(Index ...
- MySQL · 特性分析 · 优化器 MRR & BKA【转】
MySQL · 特性分析 · 优化器 MRR & BKA 上一篇文章咱们对 ICP 进行了一次全面的分析,本篇文章小编继续为大家分析优化器的另外两个选项: MRR & batched_ ...
- MySQL 8.0.2复制新特性(翻译)
译者:知数堂星耀队 MySQL 8.0.2复制新特性 MySQL 8 正在变得原来越好,而且这也在我们MySQL复制研发团队引起了一阵热潮.我们一直致力于全面提升MySQL复制,通过引入新的和一些有趣 ...
- PostgreSQL 和 MySQL 在用途、好处、特性和特点上的异同
PostgreSQL 和 MySQL 在用途.好处.特性和特点上的异同. PostgreSQL 和 MySQL 是将数据组织成表的关系数据库.这些表可以根据每个表共有的数据链接或关联.关系数据库使您的 ...
- 【mysql】关于ICP、MRR、BKA等特性
一.Index Condition Pushdown(ICP) Index Condition Pushdown (ICP)是mysql使用索引从表中检索行数据的一种优化方式,从mysql5.6开始支 ...
- MySQL中有关icp mrr和bka的特性
文辉考我的问题,有关这三个的特性,如果在面试过程中,个人见解可以答以下 icp MyQL数据库会在取出索引的同时,判断是否进行WHERE条件过滤,也就是把WHERE的部分过滤操作放在存储引擎层,在某些 ...
- ICP、MRR、BKA等特性
一.Index Condition Pushdown(ICP) Index Condition Pushdown (ICP)是 mysql 使用索引从表中检索行数据的一种优化方式,从mysql5.6开 ...
- MRR,BKA,ICP相关
MRR Multi-Range Read,多范围读,5.6以上版本开始支持 工作原理&优化效果: 将查询到的辅助索引结果放在一个缓冲(read_rnd_buffer_size = 4M)中 将 ...
随机推荐
- Zookeeper系列二:分布式架构详解、分布式技术详解、分布式事务
一.分布式架构详解 1.分布式发展历程 1.1 单点集中式 特点:App.DB.FileServer都部署在一台机器上.并且访问请求量较少 1.2 应用服务和数据服务拆分 特点:App.DB.Fi ...
- (原)阅读Android-Camera2Video的demo源码和调试心得
转载请注明出处:http://www.cnblogs.com/lihaiping/p/6142512.html 最近因为项目需要使用到camera的功能,所以针对官方的demo源码进行一番阅读,并 ...
- Scala学习笔记——简化代码、柯里化、继承、特质
1.简化代码 package com.scala.first import java.io.File import javax.management.Query /** * Created by co ...
- plsql与64位的Oracle关联方法
在这里吐槽一下,plsql居然木有64位的,以前居然不知道,好久没用Oracle了,想练习一下,方法如下: 1.安装Oracle,官网都有,这里不细说了,我选的是64的Oracle安装的. 2.下载p ...
- css背景色半透明的最佳实践
之前项目中遇到纯色的半透明背景,都是这么干: <style> .box {width:300px;height:300px;position:relative;} .mask {width ...
- swoole webSocket服务
socket.php <?php //创建websocket服务器对象,监听0.0.0.0:9502端口 $ws = ); //监听WebSocket连接打开事件 (刚打开的时候会给客户端发送 ...
- [Laravel] 10 - WEB API : wrapper
前言 一.常用的解决方案 React 前端 + PHP (Laravel) 后端 Such as "some exposure to WEB API’s and/or RESTful“. 使 ...
- SpringBoot自定义错误信息,SpringBoot适配Ajax请求
SpringBoot自定义错误信息,SpringBoot自定义异常处理类, SpringBoot异常结果处理适配页面及Ajax请求, SpringBoot适配Ajax请求 ============== ...
- Robot Framework封装的关键字输入参数可以传入多个值的方法
输入参数的最后一个参数可以是一个列表变量,通过@{列表名称}或者${列表名称}的方式实现传入多个值的场景: 或者
- js中的原型继承
<!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8&quo ...