1.Redo Log

The redo log is a disk-based data structure used during crash recovery to correct data written by incomplete transactions. During normal operations, the redo log encodes requests to change table data that result from SQL statements or low-level API calls. Modifications that did not finish updating the data files before an unexpected shutdown are replayed automatically during initialization, and before the connections are accepted.

By default, the redo log is physically represented on disk by two files named ib_logfile0 and ib_logfile1. MySQL writes to the redo log files in a circular fashion. Data in the redo log is encoded in terms of records affected; this data is collectively referred to as redo. The passage of data through the redo log is represented by an ever-increasing LSN value.

1.1 Changing the Number or Size of Redo Log Files

To change the number or the size of redo log files, perform the following steps:

  1. Stop the MySQL server and make sure that it shuts down without errors.

  2. Edit my.cnf to change the log file configuration. To change the log file size, configure innodb_log_file_size. To increase the number of log files, configureinnodb_log_files_in_group.

  3. Start the MySQL server again.

If InnoDB detects that the innodb_log_file_size differs from the redo log file size, it writes a log checkpoint, closes and removes the old log files, creates new log files at the requested size, and opens the new log files.

1.2 Group Commit for Redo Log Flushing

InnoDB, like any other ACID-compliant database engine, flushes the redo log of a transaction before it is committed. InnoDB uses group commit functionality to group multiple such flush requests together to avoid one flush for each commit. With group commit, InnoDB issues a single write to the log file to perform the commit action for multiple user transactions that commit at about the same time, significantly improving throughput.

1.3 Redo Log Archiving

Backup utilities that copy redo log records may sometimes fail to keep pace with redo log generation while a backup operation is in progress, resulting in lost redo log records due to those records being overwritten. This issue most often occurs when there is significant MySQL server activity during the backup operation, and the redo log file storage media operates at a faster speed than the backup storage media. The redo log archiving feature, introduced in MySQL 8.0.17, addresses this issue by sequentially writing redo log records to an archive file in addition to the redo log files. Backup utilities can copy redo log records from the archive file as necessary, thereby avoiding the potential loss of data.

If redo log archiving is configured on the server, MySQL Enterprise Backup, available with the MySQL Enterprise Edition, uses the redo log archiving feature when backing up a MySQL server.

Enabling redo log archiving on the server requires setting a value for the innodb_redo_log_archive_dirs system variable. The value is specified as a semicolon-separated list of labeled redo log archive directories. The label:directory pair is separated by a colon (:). For example:

mysql> SET GLOBAL innodb_redo_log_archive_dirs='label1:directory_path1[;label2:directory_path2;…]';

The label is an arbitrary identifier for the archive directory. It can be any string of characters, with the exception of colons (:), which are not permitted. An empty label is also permitted, but the colon (:) is still required in this case. A directory_path must be specified. The directory that is selected for the redo log archive file must exist when redo log archiving is activated, or an error is returned. The path can contain colons (':'), but semicolons (;) are not permitted.

The innodb_redo_log_archive_dirs variable must be configured before the redo log archiving can be activated. The default value is NULL, which does not permit activating redo log archiving.

注意事项:

The archive directories that you specify must satisfy the following requirements. (The requirements are enforced when redo log archiving is activated.):

  • Directories must exist. Directories are not created by the redo log archive process. Otherwise, the following error is returned:

    ERROR 3844 (HY000): Redo log archive directory 'directory_path1' does not exist or is not a directory

  • Directories must not be world-accessible. This is to prevent the redo log data from being exposed to unauthorized users on the system. Otherwise, the following error is returned:

    ERROR 3846 (HY000): Redo log archive directory 'directory_path1' is accessible to all OS users

  • Directories cannot be those defined by datadirinnodb_data_home_dirinnodb_directoriesinnodb_log_group_home_dir,innodb_temp_tablespaces_dirinnodb_tmpdir innodb_undo_directory, or secure_file_priv, nor can they be parent directories or subdirectories of those directories. Otherwise, an error similar to the following is returned:

    ERROR 3845 (HY000): Redo log archive directory 'directory_path1' is in, under, or over server directory 'datadir' - '/path/to/data_directory'

When a backup utility that supports redo log archiving initiates a backup, the backup utility activates redo log archiving by invoking the innodb_redo_log_archive_start() user-defined function.

If you are not using a backup utility that supports redo log archiving, redo log archiving can also be activated manually, as shown:

mysql> SELECT innodb_redo_log_archive_start('label', 'subdir');
+------------------------------------------+
| innodb_redo_log_archive_start('label') |
+------------------------------------------+
| |
+------------------------------------------+

Or:

mysql> DO innodb_redo_log_archive_start('label', 'subdir');
Query OK, rows affected (0.09 sec)

注意事项:

The MySQL session that activates redo log archiving (using innodb_redo_log_archive_start()) must remain open for the duration of the archiving. The same session must deactivate redo log archiving (using innodb_redo_log_archive_stop()). If the session is terminated before the redo log archiving is explicitly deactivated, the server deactivates redo log archiving implicitly and removes the redo log archive file.

where label is a label defined by innodb_redo_log_archive_dirssubdir is an optional argument for specifying a subdirectory of the directory identified by label for saving the archive file; it must be a simple directory name (no slash (/), backslash (\), or colon (:) is permitted). subdir can be empty, null, or it can be left out.

Only users with the INNODB_REDO_LOG_ARCHIVE privilege can activate redo log archiving by invoking innodb_redo_log_archive_start(), or deactivate it usinginnodb_redo_log_archive_stop(). The MySQL user running the backup utility or the MySQL user activating and deactivating redo log archiving manually must have this privilege.

The redo log archive file path is directory_identified_by_label/[subdir/]archive.serverUUID.000001.log, where directory_identified_by_label is the archive directory identified by the label argument for innodb_redo_log_archive_start()subdir is the optional argument used for innodb_redo_log_archive_start().

For example, the full path and name for a redo log archive file appears similar to the following:

/directory_path/subdirectory/archive.e71a47dc-61f8-11e9-a3cb-080027154b4d..log

After the backup utility finishes copying InnoDB data files, it deactivates redo log archiving by calling the innodb_redo_log_archive_stop() user-defined function.

If you are not using a backup utility that supports redo log archiving, redo log archiving can also be deactivated manually, as shown:

mysql> SELECT innodb_redo_log_archive_stop();
+--------------------------------+
| innodb_redo_log_archive_stop() |
+--------------------------------+
| |
+--------------------------------+

Or:

mysql> DO innodb_redo_log_archive_stop();
Query OK, rows affected (0.01 sec)

After the stop function completes successfully, the backup utility looks for the relevant section of redo log data from the archive file and copies it into the backup.

After the backup utility finishes copying the redo log data and no longer needs the redo log archive file, it deletes the archive file.

Removal of the archive file is the responsibility of the backup utility in normal situations. However, if the redo log archiving operation quits unexpectedly beforeinnodb_redo_log_archive_stop() is called, the MySQL server removes the file.

1.4 Performance Considerations

Activating redo log archiving typically has a minor performance cost due to the additional write activity.

On Unix and Unix-like operating systems, the performance impact is typically minor, assuming there is not a sustained high rate of updates. On Windows, the performance impact is typically a bit higher, assuming the same.

If there is a sustained high rate of updates and the redo log archive file is on the same storage media as the redo log files, the performance impact may be more significant due to compounded write activity.

If there is a sustained high rate of updates and the redo log archive file is on slower storage media than the redo log files, performance is impacted arbitrarily.

Writing to the redo log archive file does not impede normal transactional logging except in the case that the redo log archive file storage media operates at a much slower rate than the redo log file storage media, and there is a large backlog of persisted redo log blocks waiting to be written to the redo log archive file. In this case, the transactional logging rate is reduced to a level that can be managed by the slower storage media where the redo log archive file resides.

2. Undo Logs

An undo log is a collection of undo log records associated with a single read-write transaction. An undo log record contains information about how to undo the latest change by a transaction to a clustered index record. If another transaction needs to see the original data as part of a consistent read operation, the unmodified data is retrieved from undo log records. Undo logs exist within undo log segments, which are contained within rollback segments. Rollback segments reside in undo tablespacesand in the global temporary tablespace.

Undo logs that reside in the global temporary tablespace are used for transactions that modify data in user-defined temporary tables. These undo logs are not redo-logged, as they are not required for crash recovery. They are used only for rollback while the server is running. This type of undo log benefits performance by avoiding redo logging I/O.

Each undo tablespace and the global temporary tablespace individually support a maximum of 128 rollback segments. The innodb_rollback_segments variable defines the number of rollback segments.

The number of transactions that a rollback segment supports depends on the number of undo slots in the rollback segment and the number of undo logs required by each transaction.

The number of undo slots in a rollback segment differs according to InnoDB page size.

InnoDB Page Size Number of Undo Slots in a Rollback Segment (InnoDB Page Size / 16)
4096 (4KB) 256
8192 (8KB) 512
16384 (16KB) 1024
32768 (32KB) 2048
65536 (64KB) 4096

A transaction is assigned up to four undo logs, one for each of the following operation types:

  1. INSERT operations on user-defined tables

  2. UPDATE and DELETE operations on user-defined tables

  3. INSERT operations on user-defined temporary tables

  4. UPDATE and DELETE operations on user-defined temporary tables

Undo logs are assigned as needed. For example, a transaction that performs INSERTUPDATE, and DELETE operations on regular and temporary tables requires a full assignment of four undo logs. A transaction that performs only INSERT operations on regular tables requires a single undo log.

A transaction that performs operations on regular tables is assigned undo logs from an assigned undo tablespace rollback segment. A transaction that performs operations on temporary tables is assigned undo logs from an assigned global temporary tablespace rollback segment.

An undo log assigned to a transaction remains tied to the transaction for its duration. For example, an undo log assigned to a transaction for an INSERT operation on a regular table is used for all INSERT operations on regular tables performed by that transaction.

Given the factors described above, the following formulas can be used to estimate the number of concurrent read-write transactions that InnoDB is capable of supporting.

  • If each transaction performs either an INSERT or an UPDATE or DELETE operation, the number of concurrent read-write transactions that InnoDB is capable of supporting is:

    (innodb_page_size / 16) * innodb_rollback_segments * number of undo tablespaces
  • If each transaction performs an INSERT and an UPDATE or DELETE operation, the number of concurrent read-write transactions that InnoDB is capable of supporting is:

    (innodb_page_size / 16 / 2) * innodb_rollback_segments * number of undo tablespaces
  • If each transaction performs an INSERT operation on a temporary table, the number of concurrent read-write transactions that InnoDB is capable of supporting is:

    (innodb_page_size / 16) * innodb_rollback_segments
  • If each transaction performs an INSERT and an UPDATE or DELETE operation on a temporary table, the number of concurrent read-write transactions that InnoDB is capable of supporting is:

(innodb_page_size / 16 / 2) * innodb_rollback_segments

转载、节选于 https://dev.mysql.com/doc/refman/8.0/en/innodb-redo-log.html

                      https://dev.mysql.com/doc/refman/8.0/en/innodb-undo-logs.html

InnoDB On-Disk Structures(五)-- Redo Log & Undo Logs (转载)的更多相关文章

  1. 说说MySQL中的Redo log Undo log都在干啥

        在数据库系统中,既有存放数据的文件,也有存放日志的文件.日志在内存中也是有缓存Log buffer,也有磁盘文件log file,本文主要描述存放日志的文件.     MySQL中的日志文件, ...

  2. 【转】说说MySQL中的Redo log Undo log都在干啥

    阅读目录(Content) 1 undo 1.1 undo是啥 1.2 undo参数 1.3 undo空间管理 2 redo 2.1 redo是啥 2.2 redo 参数 2.3 redo 空间管理 ...

  3. InnoDB存储引擎介绍-(2)redo和undo学习

    01 – Undo LogUndo Log 是为了实现事务的原子性,在MySQL数据库InnoDB存储引擎中,还用Undo Log来实现多版本并发控制(简称:MVCC). - 事务的原子性(Atomi ...

  4. InnoDB存储引擎--学习笔记-redo log

    目录 1. 引言 2. 重做日志文件和相关概念介绍 + 2.1. 重做日志文件和bin log + 2.2. LSN(log squence number) 3. 重做日志文件基本工作原理 4. 重做 ...

  5. 深入学习MySQL 02 日志系统:bin log,redo log,undo log

    上一篇文章中,我们了解了一条查询语句的执行过程,按理说这篇应该讲一条更新语句的执行过程,但这个过程比较复杂,涉及到了好几个日志与事物,所以先梳理一下3个重要的日志,bin log(归档日志).redo ...

  6. InnoDB事务日志(redo log 和 undo log)详解

    数据库通常借助日志来实现事务,常见的有undo log.redo log,undo/redo log都能保证事务特性,undolog实现事务原子性,redolog实现事务的持久性. 为了最大程度避免数 ...

  7. 详细分析MySQL事务日志(redo log和undo log)

    innodb事务日志包括redo log和undo log.redo log是重做日志,提供前滚操作,undo log是回滚日志,提供回滚操作. undo log不是redo log的逆向过程,其实它 ...

  8. 详细分析MySQL事务日志(redo log和undo log) 表明了为何mysql不会丢数据

    innodb事务日志包括redo log和undo log.redo log是重做日志,提供前滚操作,undo log是回滚日志,提供回滚操作. undo log不是redo log的逆向过程,其实它 ...

  9. MySQL日志系统bin log、redo log和undo log

    MySQL日志系统bin log.redo log和undo log   今人不见古时月,今月曾经照古人. 简介:日志是MySQL数据库的重要组成部分,记录着数据库运行期间各种状态信息,主要包括错误日 ...

随机推荐

  1. jQuery实现类似于购物车操作

    先看页面的代码,后面会详细写每一个操作的实现. <html> <head> <title>jQuery操作表格</title> <meta cha ...

  2. A* 算法讲解

    在看下面这篇文章之前,先介绍几个理论知识,有助于理解A*算法. 启发式搜索:启发式搜索就是在状态空间中的搜索对每一个搜索的位置进行评估,得到最好的位置,再从这个位置进行搜索直到目标.这样可以省略大量无 ...

  3. python做单因素方差分析

    方差分析的主要功能就是验证两组样本,或者两组以上的样本均值是否有显著性差异,即均值是否一样. 这里有两个大点需要注意:①方差分析的原假设是:样本不存在显著性差异(即,均值完全相等):②两样本数据无交互 ...

  4. Python爬虫根据关键词爬取知网论文摘要并保存到数据库中【入门必学】

    前言 本文的文字及图片来源于网络,仅供学习.交流使用,不具有任何商业用途,版权归原作者所有,如有问题请及时联系我们以作处理. 作者:崩坏的芝麻 由于实验室需要一些语料做研究,语料要求是知网上的论文摘要 ...

  5. 【Web技术】314- 前端组件设计原则

    点击上方"前端自习课"关注,学习起来~ 译者:@没有好名字了译文:https://github.com/lightningminers/article/issues/36,http ...

  6. Jmeter性能测试配置

    目录 Jmeter检查点/断言 Jmeter事务 Jmeter集合点 Jmeter检查点/断言 在上一章节中,我们通过调试脚本,通过人工验证脚本可以完成业务功能, 但在性能测试中,我们希望能通过自动验 ...

  7. inline以及inline-block行内元素:vertical-align属性

  8. SpringBoot电商项目实战 — ElasticSearch接入实现

    如今在一些中大型网站中,搜索引擎已是必不可少的内容了.首先我们看看搜索引擎到底是什么呢?搜索引擎,就是根据用户需求与一定算法,运用特定策略从互联网检索出制定信息反馈给用户的一门检索技术.搜索引擎依托于 ...

  9. vue路由基础总结

    1.创建项目 为了练习路由 这里没有选择路由 就选了Bable 自己一步一步的配置 加深印象. 然后下载路由 npm install vue-router --save 2.基础配置 src文件下新建 ...

  10. 使用t-SNE做降维可视化

    最近在做一个深度学习分类项目,想看看训练集数据的分布情况,但由于数据本身维度接近100,不能直观的可视化展示,所以就对降维可视化做了一些粗略的了解以便能在低维空间中近似展示高维数据的分布情况,以下内容 ...