redolog 大小的实验
前言：近日因工作需要，测试postgresql和MySQL在oltp对比测试，因结果差异太多（MySQL测试结果比较差，相同环境），寻求大神帮助，有幸得叶大师和姜大师指点，指出my.cnf配置文件innodb_log_file_size不合理（在以前的资料提示Innodb，redolog大小一般都建议128-512M大小，不然崩溃恢复将会很漫长），redolog越大，checkpoint就越少，tps相对越高，MySQL5.6redolog 大小可以超过4G，下面就测试一下不同redolog大小奔溃恢复时间
测试环境：硬件是VM，系统是centos6.6 4核4G内存，宿主硬盘是Intel SSD，测试表数据为1千万行，测试工具为sysbench0.5版本，测试方法是压力测试工具进行压力测试，然后暴力kill MySQL
跟日志相关参数：
innodb_flush_log_at_trx_commit = 1
innodb_log_buffer_size = 32M
innodb_log_file_size = 4096M（分别测试1G，2G,4G）
innodb_log_files_in_group = 2
innodb_max_dirty_pages_pct = 70
IO性能：因为结果出乎人意料我不得不先把我自己IO能力测试出来
16k随机读写IO，读写比例为：40：60 IOPS=2951
fio -filename=/data/fiotest -direct=1 -rw=randrw -ioengine=libaio -rwmixwrite=60 \
> -bs=16k -size=8G -numjobs=64 -runtime=600 -group_reporting -name=mytest
read : io=27676MB, bw=47231KB/s, iops=, runt=600025msec
8k随机读写IO，读写比例为：40：60 IOPS=2951
fio -filename=/data/fiotest -direct=1 -rw=randrw -ioengine=libaio -rwmixwrite=60 \
> -bs=8k -size=8G -numjobs=64 -runtime=600 -group_reporting -name=mytest
write: io=18155MB, bw=30983KB/s, iops=, runt=600046msec
4K随机读写IO 读写比例为：40：60 IOPS=3590
fio -filename=/data/fiotest -direct=1 -rw=randrw -ioengine=libaio -rwmixwrite=60 \
> -bs=4k -size=8G -numjobs=64 -runtime=600 -group_reporting -name=mytest
read : io=8416.6MB, bw=14363KB/s, iops=3590, runt=600030msec
【1】测试方法
1.1 创建测试表
./sysbench --mysql-host=192.168.80.106 --mysql-user=sysbench --mysql-password=123456 --mysql-db=sysbench --test=/opt/sysbench/tests/db/oltp.lua --oltp_tables_count=10 --oltp-table-size=1000000 --rand-init=on prepare
1.2 进行测试，其中一个会话在做压力测试，一个会话暴力杀掉MySQL sleep 300; pkill -9 mysqld
./sysbench --mysql-host=192.168.80.106 --mysql-port=3306 --mysql-user=sysbench --mysql-password=123456 --test=/opt/sysbench/tests/db/oltp.lua --oltp_tables_count=10 --oltp-table-size=10000000 --num-threads=64-oltp-read-only=off --report-interval=10 --rand-type=uniform --max-time=1200 --mysql-db=sysbench --max-requests=0 --percentile=99 run
【2】测试结果
2.1 innodb_log_file_size = 1024M
:: mysqld_safe Starting mysqld daemon with databases from /data/mydata
-- :: [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
-- :: [Note] Plugin 'FEDERATED' is disabled.
-- :: 7f80332d07e0 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
-- :: [Note] InnoDB: Using atomics to ref count buffer pool pages
-- :: [Note] InnoDB: The InnoDB memory heap is disabled
-- :: [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
-- :: [Note] InnoDB: Memory barrier is not used
-- :: [Note] InnoDB: Compressed tables use zlib 1.2.
-- :: [Note] InnoDB: Using Linux native AIO
-- :: [Note] InnoDB: Using CPU crc32 instructions
-- :: [Note] InnoDB: Initializing buffer pool, size = .0G
-- :: [Note] InnoDB: Completed initialization of buffer pool
-- :: [Note] InnoDB: Highest supported file format is Barracuda.
-- :: [Note] InnoDB: Log scan progressed past the checkpoint lsn
-- :: [Note] InnoDB: Database was not shutdown normally!
-- :: [Note] InnoDB: Starting crash recovery.
-- :: [Note] InnoDB: Reading tablespace information from the .ibd files...
-- :: [Note] InnoDB: Restoring possible half-written data pages
-- :: [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Transaction was in the XA prepared state.
InnoDB: transaction(s) which must be rolled back or cleaned up
InnoDB: in total row operations to undo
InnoDB: Trx id counter is
-- :: [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent:
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position , file name dg6-logbin.
-- :: [Note] InnoDB: rollback segment(s) are active.
-- :: [Note] InnoDB: Waiting for purge to start
InnoDB: Starting in background the rollback of uncommitted transactions
-- :: 7f7f6dde8700 InnoDB: Rolling back trx with id , rows to undo
-- :: [Note] InnoDB: Rollback of trx with id completed
-- :: 7f7f6dde8700 InnoDB: Rolling back trx with id , rows to undo
-- :: [Note] InnoDB: Rollback of trx with id completed
-- :: 7f7f6dde8700 InnoDB: Rolling back trx with id , rows to undo
-- :: [Note] InnoDB: Rollback of trx with id completed
-- :: 7f7f6dde8700 InnoDB: Rollback of non-prepared transactions completed
-- :: [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.21-rel70.1 started; log sequence number 3446339459
-- :: [Note] Recovering after a crash using dg6-logbin
-- :: [Note] Starting crash recovery...
-- :: 7f80332d07e0 InnoDB: Starting recovery for XA transactions...
-- :: 7f80332d07e0 InnoDB: Transaction in prepared state after recovery
-- :: 7f80332d07e0 InnoDB: Transaction contains changes to rows
-- :: 7f80332d07e0 InnoDB: transactions in prepared state after recovery
-- :: [Note] Found prepared transaction(s) in InnoDB
-- :: [Note] Crash recovery finished.
/opt/app/mysql/bin/mysqld: Found a Gtid_log_event or Previous_gtids_log_event when @@GLOBAL.GTID_MODE = OFF.
/opt/app/mysql/bin/mysqld: Found a Gtid_log_event or Previous_gtids_log_event when @@GLOBAL.GTID_MODE = OFF.
-- :: [Note] RSA private key file not found: /data/mydata//private_key.pem. Some authentication plugins will not work.
-- :: [Note] RSA public key file not found: /data/mydata//public_key.pem. Some authentication plugins will not work.
-- :: [Note] Server hostname (bind-address): '*'; port:
-- :: [Note] IPv6 is available.
-- :: [Note] - '::' resolves to '::';
-- :: [Note] Server socket created on IP: '::'.
-- :: [Warning] 'user' entry 'root@dg1' ignored in --skip-name-resolve mode.
-- :: [Warning] 'proxies_priv' entry '@ root@dg1' ignored in --skip-name-resolve mode.
-- :: [Note] Event Scheduler: Loaded events
-- :: [Note] /opt/app/mysql/bin/mysqld: ready for connections.
Version: '5.6.21-70.1-log' socket: '/tmp/mysql.sock' port: Percona Server (GPL), Release 70.1, Revision
看到木有恢复时间才十几秒钟呢
2.2 innodb_log_file_size = 2048M
:: mysqld_safe Starting mysqld daemon with databases from /data/mydata
-- :: [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
-- :: [Note] Plugin 'FEDERATED' is disabled.
-- :: 7f599fd1c7e0 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
-- :: [Note] InnoDB: Using atomics to ref count buffer pool pages
-- :: [Note] InnoDB: The InnoDB memory heap is disabled
-- :: [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
-- :: [Note] InnoDB: Memory barrier is not used
-- :: [Note] InnoDB: Compressed tables use zlib 1.2.
-- :: [Note] InnoDB: Using Linux native AIO
-- :: [Note] InnoDB: Using CPU crc32 instructions
-- :: [Note] InnoDB: Initializing buffer pool, size = .0G
-- :: [Note] InnoDB: Completed initialization of buffer pool
-- :: [Note] InnoDB: Highest supported file format is Barracuda.
-- :: [Note] InnoDB: Log scan progressed past the checkpoint lsn
-- :: [Note] InnoDB: Database was not shutdown normally!
-- :: [Note] InnoDB: Starting crash recovery.
-- :: [Note] InnoDB: Reading tablespace information from the .ibd files...
-- :: [Note] InnoDB: Restoring possible half-written data pages
-- :: [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Transaction was in the XA prepared state.
InnoDB: transaction(s) which must be rolled back or cleaned up
InnoDB: in total row operations to undo
InnoDB: Trx id counter is
-- :: [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent:
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position , file name dg6-logbin.
-- :: [Note] InnoDB: rollback segment(s) are active.
-- :: [Note] InnoDB: Waiting for purge to start
InnoDB: Starting in background the rollback of uncommitted transactions
-- :: 7f58dd9c2700 InnoDB: Rolling back trx with id , rows to undo
-- :: [Note] InnoDB: Rollback of trx with id completed
-- :: 7f58dd9c2700 InnoDB: Rollback of non-prepared transactions completed
-- :: [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.21-rel70.1 started; log sequence number 3505201579
-- :: [Note] Recovering after a crash using dg6-logbin
-- :: [Note] Starting crash recovery...
-- :: 7f599fd1c7e0 InnoDB: Starting recovery for XA transactions...
-- :: 7f599fd1c7e0 InnoDB: Transaction in prepared state after recovery
-- :: 7f599fd1c7e0 InnoDB: Transaction contains changes to rows
-- :: 7f599fd1c7e0 InnoDB: transactions in prepared state after recovery
-- :: [Note] Found prepared transaction(s) in InnoDB
-- :: [Note] Crash recovery finished.
/opt/app/mysql/bin/mysqld: Found a Gtid_log_event or Previous_gtids_log_event when @@GLOBAL.GTID_MODE = OFF.
/opt/app/mysql/bin/mysqld: Found a Gtid_log_event or Previous_gtids_log_event when @@GLOBAL.GTID_MODE = OFF.
-- :: [Note] RSA private key file not found: /data/mydata//private_key.pem. Some authentication plugins will not work.
-- :: [Note] RSA public key file not found: /data/mydata//public_key.pem. Some authentication plugins will not work.
-- :: [Note] Server hostname (bind-address): '*'; port:
-- :: [Note] IPv6 is available.
-- :: [Note] - '::' resolves to '::';
-- :: [Note] Server socket created on IP: '::'.
-- :: [Warning] 'user' entry 'root@dg1' ignored in --skip-name-resolve mode.
-- :: [Warning] 'proxies_priv' entry '@ root@dg1' ignored in --skip-name-resolve mode.
-- :: [Note] Event Scheduler: Loaded events
-- :: [Note] /opt/app/mysql/bin/mysqld: ready for connections.
Version: '5.6.21-70.1-log' socket: '/tmp/mysql.sock' port: Percona Server (GPL), Release 70.1, Revision
不科学呀，还是十几秒，那么我们改成4G，让压测运行时间更长一点，900秒以后在kill
2.3 innodb_log_file_size = 4096M
:: mysqld_safe Starting mysqld daemon with databases from /data/mydata
-- :: [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
-- :: [Note] Plugin 'FEDERATED' is disabled.
-- :: 7fd6f83a67e0 InnoDB: Warning: Using innodb_additional_mem_pool_size is DEPRECATED. This option may be removed in future releases, together with the option innodb_use_sys_malloc and with the InnoDB's internal memory allocator.
-- :: [Note] InnoDB: Using atomics to ref count buffer pool pages
-- :: [Note] InnoDB: The InnoDB memory heap is disabled
-- :: [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins
-- :: [Note] InnoDB: Memory barrier is not used
-- :: [Note] InnoDB: Compressed tables use zlib 1.2.
-- :: [Note] InnoDB: Using Linux native AIO
-- :: [Note] InnoDB: Using CPU crc32 instructions
-- :: [Note] InnoDB: Initializing buffer pool, size = .0G
-- :: [Note] InnoDB: Completed initialization of buffer pool
-- :: [Note] InnoDB: Highest supported file format is Barracuda.
-- :: [Note] InnoDB: Log scan progressed past the checkpoint lsn
-- :: [Note] InnoDB: Database was not shutdown normally!
-- :: [Note] InnoDB: Starting crash recovery.
-- :: [Note] InnoDB: Reading tablespace information from the .ibd files...
-- :: [Note] InnoDB: Restoring possible half-written data pages
-- :: [Note] InnoDB: from the doublewrite buffer...
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Transaction was in the XA prepared state.
InnoDB: Transaction was in the XA prepared state.
InnoDB: transaction(s) which must be rolled back or cleaned up
InnoDB: in total row operations to undo
InnoDB: Trx id counter is
-- :: [Note] InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percent:
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position , file name dg6-logbin.
-- :: [Note] InnoDB: rollback segment(s) are active.
-- :: [Note] InnoDB: Waiting for purge to start
InnoDB: Starting in background the rollback of uncommitted transactions
-- :: 7fd635de8700 InnoDB: Rolling back trx with id , rows to undo
-- :: [Note] InnoDB: Rollback of trx with id completed
-- :: 7fd635de8700 InnoDB: Rollback of non-prepared transactions completed
-- :: [Note] InnoDB: Percona XtraDB (http://www.percona.com) 5.6.21-rel70.1 started; log sequence number 3662227637
-- :: [Note] Recovering after a crash using dg6-logbin
-- :: [Note] Starting crash recovery...
-- :: 7fd6f83a67e0 InnoDB: Starting recovery for XA transactions...
-- :: 7fd6f83a67e0 InnoDB: Transaction in prepared state after recovery
-- :: 7fd6f83a67e0 InnoDB: Transaction contains changes to rows
-- :: 7fd6f83a67e0 InnoDB: transactions in prepared state after recovery
-- :: [Note] Found prepared transaction(s) in InnoDB
-- :: [Note] Crash recovery finished.
/opt/app/mysql/bin/mysqld: Found a Gtid_log_event or Previous_gtids_log_event when @@GLOBAL.GTID_MODE = OFF.
/opt/app/mysql/bin/mysqld: Found a Gtid_log_event or Previous_gtids_log_event when @@GLOBAL.GTID_MODE = OFF.
-- :: [Note] RSA private key file not found: /data/mydata//private_key.pem. Some authentication plugins will not work.
-- :: [Note] RSA public key file not found: /data/mydata//public_key.pem. Some authentication plugins will not work.
-- :: [Note] Server hostname (bind-address): '*'; port:
-- :: [Note] IPv6 is available.
-- :: [Note] - '::' resolves to '::';
-- :: [Note] Server socket created on IP: '::'.
-- :: [Warning] 'user' entry 'root@dg1' ignored in --skip-name-resolve mode.
-- :: [Warning] 'proxies_priv' entry '@ root@dg1' ignored in --skip-name-resolve mode.
-- :: [Note] Event Scheduler: Loaded events
-- :: [Note] /opt/app/mysql/bin/mysqld: ready for connections.
Version: '5.6.21-70.1-log' socket: '/tmp/mysql.sock' port: Percona Server (GPL), Release 70.1, Revision
也才二十几秒钟。
那么我们看看5.5版本是什么情况 因为5.5版本redolog大小不能超过4G，所以单个不能设置成2048M否则会报错
150607 17:15:46 InnoDB: Error: combined size of log files must be < 4 GB
150607 17:15:46 [ERROR] Plugin 'InnoDB' init function returned error.
2.4 innodb_log_file_size = 1024M
:: mysqld_safe Starting mysqld daemon with databases from /data/mydata
:: [Warning] 'THREAD_CONCURRENCY' is deprecated and will be removed in a future release.
:: [Note] /opt/app/mysql/bin/mysqld (mysqld 5.5.-log) starting as process ...
:: [Note] Plugin 'FEDERATED' is disabled.
:: InnoDB: The InnoDB memory heap is disabled
:: InnoDB: Mutexes and rw_locks use GCC atomic builtins
:: InnoDB: Compressed tables use zlib 1.2.
:: InnoDB: Using Linux native AIO
:: InnoDB: Initializing buffer pool, size = .0G
:: InnoDB: Completed initialization of buffer pool
:: InnoDB: highest supported file format is Barracuda.
InnoDB: Log scan progressed past the checkpoint lsn
:: InnoDB: Database was not shut down normally!
InnoDB: Starting crash recovery.
InnoDB: Reading tablespace information from the .ibd files...
InnoDB: Restoring possible half-written data pages from the doublewrite
InnoDB: buffer...
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Doing recovery: scanned up to log sequence number
InnoDB: Transaction 203E1 was in the XA prepared state.
InnoDB: transaction(s) which must be rolled back or cleaned up
InnoDB: in total row operations to undo
InnoDB: Trx id counter is
:: InnoDB: Starting an apply batch of log records to the database...
InnoDB: Progress in percents:
InnoDB: Apply batch completed
InnoDB: Last MySQL binlog file position , file name ./mysql-bin.
InnoDB: Starting in background the rollback of uncommitted transactions
:: InnoDB: Rollback of non-prepared transactions completed
:: InnoDB: Waiting for the background threads to start
:: InnoDB: 5.5. started; log sequence number
:: [Note] Recovering after a crash using mysql-bin
:: [Note] Starting crash recovery...
:: InnoDB: Starting recovery for XA transactions...
:: InnoDB: Transaction 203E1 in prepared state after recovery
:: InnoDB: Transaction contains changes to rows
:: InnoDB: transactions in prepared state after recovery
:: [Note] Found prepared transaction(s) in InnoDB
:: [Note] Crash recovery finished.
:: [Note] Server hostname (bind-address): '0.0.0.0'; port:
:: [Note] - '0.0.0.0' resolves to '0.0.0.0';
:: [Note] Server socket created on IP: '0.0.0.0'.
:: [Warning] 'user' entry 'root@dg1' ignored in --skip-name-resolve mode.
:: [Warning] 'user' entry '@dg1' ignored in --skip-name-resolve mode.
:: [Warning] 'proxies_priv' entry '@ root@dg1' ignored in --skip-name-resolve mode.
:: [Note] Event Scheduler: Loaded events
:: [Note] /opt/app/mysql/bin/mysqld: ready for connections.
Version: '5.5.43-log' socket: '/tmp/mysql.sock' port: MySQL Community Server (GPL)
也不会很慢二十几秒钟
总结：
在5.5版本以上，修改了恢复算法，崩溃恢复快了很多，但是这里面涉及到很多因素，buffer_pool ,脏页比例，TPS等，需要多次测试，因条件限制，没办法进行大buffer_pool，高并发测试，实际物理机测试时，根据自身情况选着合理参数测试
参考资料：来自姜大神的书
在InnoDB存储引擎内部，有两种Checkpoint，分别为：
Sharp Checkpoint
Fuzzy Checkpoint
Sharp Checkpoint发生在数据库关闭时将所有的脏页都刷新回磁盘，这是默认的工作方式，即参数innodb_fast_shutdown=1。
但是若数据库在运行时也使用Sharp Checkpoint，那么数据库的可用性就会受到很大的影响。故在InnoDB存储引擎内部使用Fuzzy Checkpoint进行页的刷新，即只刷新一部分脏页，而不是刷新所有的脏页回磁盘。
这里笔者进行了概括，在InnoDB存储引擎中可能发生如下几种情况的Fuzzy Checkpoint：
Master Thread Checkpoint
FLUSH_LRU_LIST Checkpoint
Async/Sync Flush Checkpoint
Dirty Page too much Checkpoint
对于Master Thread（2.5节会详细介绍各个版本中Master Thread的实现）中发生的Checkpoint，差不多以每秒或每十秒的速度从缓冲池的脏页列表中刷新一定比例的页回磁盘。这个过程是异步的，即此时InnoDB存储引擎可以进行其他的操作，用户查询线程不会阻塞。
FLUSH_LRU_LIST Checkpoint是因为InnoDB存储引擎需要保证LRU列表中需要有差不多100个空闲页可供使用。在InnoDB1.1.x版本之前，需要检查LRU列表中是否有足够的可用空间操作发生在用户查询线程中，显然这会阻塞用户的查询操作。倘若没有100个可用空闲页，那么InnoDB存储引擎会将LRU列表尾端的页移除。如果这些页中有脏页，那么需要进行Checkpoint，而这些页是来自LRU列表的，因此称为FLUSH_LRU_LIST Checkpoint。
而从MySQL 5.6版本，也就是InnoDB1.2.x版本开始，这个检查被放在了一个单独的Page Cleaner线程中进行，并且用户可以通过参数innodb_lru_scan_depth控制LRU列表中可用页的数量，该值默认为1024，如：
mysql> SHOW VARIABLES LIKE 'innodb_lru_scan_depth'\G;
*************************** 1. row ***************************
Variable_name: innodb_lru_scan_depth
Value: 1024
1 row in set (0.00 sec)
Async/Sync Flush Checkpoint指的是重做日志文件不可用的情况，这时需要强制将一些页刷新回磁盘，而此时脏页是从脏页列表中选取的。若将已经写入到重做日志的LSN记为redo_lsn，将已经刷新回磁盘最新页的LSN记为checkpoint_lsn，则可定义：
checkpoint_age = redo_lsn - checkpoint_lsn
再定义以下的变量：
async_water_mark = 75% * total_redo_log_file_size
sync_water_mark = 90% * total_redo_log_file_size
若每个重做日志文件的大小为1GB，并且定义了两个重做日志文件，则重做日志文件的总大小为2GB。那么async_water_mark=1.5GB，sync_water_mark=1.8GB。则：
当checkpoint_age<async_water_mark时，不需要刷新任何脏页到磁盘；
当async_water_mark<checkpoint_age<sync_water_mark时触发Async Flush，从Flush列表中刷新足够的脏页回磁盘，使得刷新后满足checkpoint_age<async_water_mark；
checkpoint_age>sync_water_mark这种情况一般很少发生，除非设置的重做日志文件太小，并且在进行类似LOAD DATA的BULK INSERT操作。此时触发Sync Flush操作，从Flush列表中刷新足够的脏页回磁盘，使得刷新后满足checkpoint_age<async_water_mark。
可见，Async/Sync Flush Checkpoint是为了保证重做日志的循环使用的可用性。在InnoDB 1.2.x版本之前，Async Flush Checkpoint会阻塞发现问题的用户查询线程，而Sync Flush Checkpoint会阻塞所有的用户查询线程，并且等待脏页刷新完成。从InnoDB 1.2.x版本开始——也就是MySQL 5.6版本，这部分的刷新操作同样放入到了单独的Page Cleaner Thread中，故不会阻塞用户查询线程。
MySQL官方版本并不能查看刷新页是从Flush列表中还是从LRU列表中进行Checkpoint的，也不知道因为重做日志而产生的Async/Sync Flush的次数。但是InnoSQL版本提供了方法，可以通过命令SHOW ENGINE INNODB STATUS来观察，如：
mysql> SHOW ENGINE INNODB STATUS\G;
*************************** 1. row ***************************
Type: InnoDB
……
LRU len: 112902, unzip_LRU len: 0
I/O sum[0]:cur[0], unzip sum[0]:cur[0]
Async Flush: 0, Sync Flush: 0, LRU List Flush: 0, Flush List Flush: 111736
……
1 row in set (0.01 sec)
根据上述的信息，还可以对InnoDB存储引擎做更为深入的调优，这部分将在第9章中讲述。
最后一种Checkpoint的情况是Dirty Page too much，即脏页的数量太多，导致InnoDB存储引擎强制进行Checkpoint。其目的总的来说还是为了保证缓冲池中有足够可用的页。其可由参数innodb_max_dirty_pages_pct控制：
mysql>SHOW VARIABLES LIKE 'innodb_max_dirty_pages_pct'\G;
*************************** 1. row ***************************
Variable_name: innodb_max_dirty_pages_pct
Value: 75
1 row in set (0.00 sec)
innodb_max_dirty_pages_pct值为75表示，当缓冲池中脏页的数量占据75%时，强制进行Checkpoint，刷新一部分的脏页到磁盘。在InnoDB 1.0.x版本之前，该参数默认值为90，之后的版本都为75。
redolog 大小的实验的更多相关文章
- C#比较时间大小 1、比较时间大小的实验
1.比较时间大小的实验 string st1="12:13"; string st2="14:14"; DateTime dt1=Convert.ToDa ...
- DATAGUARD 添加修改REDOLOG大小
DG在线日志组大小修改 环境(单实例,Centos 6.5 X64,oracle 10.2.0.5,filesystem存储) REDO ONLINE LOG select * from v$logf ...
- [转帖] Linux 下面栈空间大小的实验
比如局部变量是保存在栈空间中的,今天突然在想栈的上限是多大呢,什么时候才会栈溢出? ulimit 命令 linux下使用ulimit 命令可以查看系统的很多上限值. ulimit -a 查看所有 ul ...
- Hadoop第3周练习--Hadoop2.X编译安装和实验
作业题目 位系统下进行本地编译的安装方式 选2 (1) 能否给web监控界面加上安全机制,怎样实现?抓图过程 (2)模拟namenode崩溃,例如将name目录的内容全部删除,然后通过secondar ...
- C#里面比较时间大小三种方法
1.比较时间大小的实验 string st1="12:13";string st2="14:14";DateTime dt1=Convert.ToDateTim ...
- C++类的大小——sizeof(class)
第一:空类的大小 class CBase { }; 运行cout<<"sizeof(CBase)="<<sizeof(CBase)<<endl; ...
- C#中日期记忆日期的格式化，日期格式化说明
参数format格式详细用法:格式字符 关联属性/说明 d ShortDatePattern D LongDatePattern f 完整日期和时间(长日期和短时间) F FullDateTimePa ...
- 使用caffe训练自己的CNN
现在有这样的一个场景:给一张行人的小矩形框图片, 根据该行人的特征识别出性别. 分析: (1),行人的姿态各异,变化多端.很难提取图像的特定特征 (2),正常人肉眼判别行人的根据是身材比例,头发长度等 ...
- C# 日期格式转【转】
使用:DateTime.ToString的方法(String, IFormatProvider)转换格式 using System; using System.Globalization; Strin ...
随机推荐
- HTML5之 Microdata微数据
- 为何需要微数据 长篇加累版牍,不好理解 微标记来标注其中内容,让其容易识辨 - RDFa Resource Description Framework http://www.w3.org/TR/m ...
- iOS 非ARC基本内存管理系列 2-多对象内存管理(2)
/* 多对象内存管理: 以人拥有车为例涉及到@property底层set方法管理内存的实现 注意:人在换车的时候要进行当前传入的车和人所拥有的车进行判断 */ /******************* ...
- Axure RP 各个版本中文版 汉化包 破解版 下载地址及注册码
导读:Axure RP Pro是一个产品经理必备的交互原型设计工具,能够高效率制作产品原型,快速绘制线框图.流程图.网站架构图.示意图.HTML模版等.Axure RP已被一些大公司采用.Axure ...
- xamarin android——数据绑定到控件（四）
本文为通过自定义列表适配器定义ListView,以上文为基础,基于ListActivity. 定义列表项布局,包含一个图片显示,标题和描述 <LinearLayout xmlns:android ...
- response返回随笔
response.setHeader("Content-type", "text/html;charset=UTF-8");//这句话的意思,是让浏览器用utf ...
- discuz 注册用户用到的几个表
通过开启记录:mysql记录日志,筛出如下信息: INSERT INTO `userclub`.pre_ucenter_members SET secques='', username='pthlp1 ...
- Mysql忘记密码，重新设置
1. 停止mysql 服务 2. 增加参数,启动mysql 服务: mysqld –skip-grant-tables (sudo vi /etc/mysql/my.cnf,在[mysqld]段中 ...
- opencv学习笔记（01）——操作图像的像素
#include <opencv2\core\core.hpp> #include <opencv2\highgui\highgui.hpp> #include <ope ...
- 微软职位内部推荐-SDE
微软近期Open的职位: Organization Summary:Engineering, Community & Online (ECO) is looking for a great & ...
- jq 7种实例化
$(html) ->$(array) $(html,{}||$(...)) $(#id) $(expr,$(...)) $(expr,context) $(dom) $(function(){} ...