bulk_insert_buffer_size and InnoDB
Q:
I read the following on this page
http://dev.mysql.com/doc/mysql/en/server-system-variables.html
bulk_insert_buffer_size MyISAM uses a special tree-like cache to make bulk inserts faster for INSERT ... SELECT, INSERT ... VALUES (...), (...), ..., and LOAD DATA INFILE. This variable limits the size of the cache tree in bytes per thread. Setting it to 0 disables this optimization. Note: This cache is used only when adding data to a non-empty table. The default value is 8MB. This variable was added in MySQL 4.0.3. This variable previously was named myisam_bulk_insert_tree_size.
I take this to mean that bulk_insert_buffer_size has no bearing on InnoDB tables, only on MyISAM tables. Is this correct?
A:
Yes, you are correct. I wrote about this before: How can I optimize this mySQL table that will need to hold 5+ million rows?
WHen it comes to bulk loading InnoDB tables, one of the most frequently ignored settings to disable is innodb_doublewrite.
InnoDB Architecture
Please note the double write buffer inside the system tablespace file ibdata1
. Every time a 16K page is written to an InnoDB table or transaction log, the page is written to the double wrote buffer first. All you need to do is disable it, load your data, and reenable it.
STEP 01 : Disable Fast Shutdown
mysql -uroot -p...-ANe"SET GLOBAL innodb_fast_shutdown = 0"
STEP 02 : Restart mysql without the double write buffer
service mysql restart --innodb-doublewrite=0
STEP 03 : Load the InnoDB data
STEP 04 : Restart mysql
service mysql restart
CAVEAT
You can disable the foreign key checks and the unique checks as well. Any standard mysqldump will do that for you already.
2. For Innodb we need to diable unique_checks and foreign_key_checks using
set unique_checks=0; set foreign_key_checks=0; diable auto_commit.
inserts......
set unique_checks=1; set foreign_key_checks=1; enable auto_commit.
Which improves bulk inserts performance.
bulk_insert_buffer_size:
MyISAM
uses a special tree-like cache to make bulk inserts faster for INSERT ... SELECT
, INSERT ... VALUES (...), (...), ...
, and LOAD DATA INFILE
when adding data to nonempty tables. This variable limits the size of the cache tree in bytes per thread. Setting it to 0 disables this optimization. The default value is 8MB.
innodb_doublewrite
:
If this variable is enabled (the default), InnoDB
stores all data twice, first to the doublewrite buffer, then to the actual data files. This variable can be turned off with --skip-innodb_doublewrite
for benchmarks or cases when top performance is needed rather than concern for data integrity or possible failures.
innodb_fast_shutdown
:
The InnoDB
shutdown mode. If the value is 0, InnoDB
does a slow shutdown, a full purge and an insert buffer merge before shutting down. If the value is 1 (the default), InnoDB
skips these operations at shutdown, a process known as a fast shutdown. If the value is 2, InnoDB
flushes its logs and shuts down cold, as if MySQL had crashed; no committed transactions are lost, but the crash recovery operation makes the next startup take longer.
The slow shutdown can take minutes, or even hours in extreme cases where substantial amounts of data are still buffered. Use the slow shutdown technique before upgrading or downgrading between MySQL major releases, so that all data files are fully prepared in case the upgrade process updates the file format.
Use innodb_fast_shutdown=2
in emergency or troubleshooting situations, to get the absolute fastest shutdown if data is at risk of corruption.
参考:
http://dba.stackexchange.com/questions/49646/bulk-insert-buffer-size-and-innodb
http://dev.mysql.com/doc/refman/5.6/en/innodb-parameters.html#sysvar_innodb_doublewrite
bulk_insert_buffer_size and InnoDB的更多相关文章
- 实战-Mysql5.6.36脚本编译安装及初始化
概述 本文为centos7.3自动化编译安装mysql5.3.6的脚本及后续初始化操作,话不多少,直接上脚本. 安装脚本install.py如下: #coding=utf-8 #!/usr/bin/p ...
- Serv-u FTP迁移(windows_to_windwos)
需求分析 公司服务器要做维护,部分服务器需要进行迁移处理,其中就包括Ser-v FTP服务器. 确认环境信息 角色 ip 系统版本 sql版本 Serv-U版本 ODBC account/passwo ...
- Mysql优化系列(1)--Innodb引擎下mysql自身配置优化
1.简单介绍InnoDB给MySQL提供了具有提交,回滚和崩溃恢复能力的事务安全(ACID兼容)存储引擎.InnoDB锁定在行级并且也在SELECT语句提供一个Oracle风格一致的非锁定读.这些特色 ...
- Mysql优化系列(1)--Innodb重要参数优化
1.简单介绍InnoDB给MySQL提供了具有提交,回滚和崩溃恢复能力的事务安全(ACID兼容)存储引擎.InnoDB锁定在行级并且也在SELECT语句提供一个Oracle风格一致的非锁定读.这些特色 ...
- Innodb引擎下mysql自身配置优化
1.简单介绍InnoDB给MySQL提供了具有提交,回滚和崩溃恢复能力的事务安全(ACID兼容)存储引擎.InnoDB锁定在行级并且也在SELECT语句提供一个Oracle风格一致的非锁定读.这些特色 ...
- Mysql优化系列--Innodb引擎下mysql自身配置优化-转
原文链接:http://www.cnblogs.com/kevingrace/p/6133818.html 谢谢楼主 1.简单介绍 InnoDB给MySQL提供了具有提交,回滚和崩溃恢复能力的事务安全 ...
- mysql innodb cluster 无感知集群
MySQL 8.0.12 innodb cluster 高可用集群部署运维管理手册 Innodb cluster 原理介绍 Innodb cluster 利用组复制的 pxos 协议,保障数据一致性 ...
- MySQL数据库和InnoDB存储引擎文件
参数文件 当MySQL示例启动时,数据库会先去读一个配置参数文件,用来寻找数据库的各种文件所在位置以及指定某些初始化参数,这些参数通常定义了某种内存结构有多大等.在默认情况下,MySQL实例会按照一定 ...
- InnoDB关键特性学习笔记
插入缓存 Insert Buffer Insert Buffer是InnoDB存储引擎关键特性中最令人激动与兴奋的一个功能.不过这个名字可能会让人认为插入缓冲是缓冲池中的一个组成部分.其实不然,Inn ...
随机推荐
- python 用装饰器写登录
# 1.编写装饰器,为多个函数加上认证的功能(用户的账号密码来源于文件), # 要求登录成功一次,后续的函数都无需再输入用户名和密码 # FLAG = False # def login(func): ...
- python基础之初识
一. 计算机是什么 基本组成: 主板+cpu+内存 cpu: 主频, 核数(16) 内存:大小(8G, 16G, 32G) 型号: DDR3, DDR4, DDR5, 主频(海盗船,玩家国度) 显卡: ...
- C++代码理解 (强制指针转换)
#include<iostream> using namespace std; class A { public: A() { a=; b=; c=; f=; } private: int ...
- 用ssh进行git clone出现 fatal: Could not read from remote repository.
问题:在通过MobaXterm进行ssh连接的服务器上用ssh进行git clone出现 fatal: Could not read from remote repository. 解决方法:prox ...
- R语言绘图:时间序列分析 ggplot2绘制ACF PACF
R语言真是博大精深 方法一 Acf(gold[,2], type = "correlation",lag.max = 100) Acf(gold[,2], type = " ...
- stm8编程tips(stvd)
编译完成时显示程序占用的flash和ram大小 将附件压缩包中的mapinfo.exe解压到stvd的安装路径\stvd中 在工程上点右键选settings 右侧的选项卡选择Linker,将categ ...
- 在编程的时候,NotePad++ 中闪烁的光标突然有竖着闪烁的编程蓝色下划线闪烁的--小技巧告诉你-费元星
当在写代码时出现的光标闪烁(横线闪烁) 在键盘上找 Insert ,按这个Insert就可以把横向闪烁光标( _ )修改成竖向闪烁光标样式( | ),横向光标会在你写代码的时候修改前面的代码,把光标移 ...
- 在Go语言里检测内存泄漏
我们先来设定一下数据库,建立一个MySQL数据库表,名为users,里面有login_name.nickname.uid.password.forbidden几个字段,其中uid与forbidden为 ...
- PostgreSQL 10.0 压缩版的 pgAdmin 不能用的问题
PostgreSQL终于发布10.0正式版了!下载压缩版的更新了一下本机的版本,然后打开pgAdmin的时候总是报错“The application server could not be conta ...
- jmeter多用户登录跨线程组操作传值
项目需求: 需要登录两个用户A.B,用户A操作完后会通知B,然后B再操作,B操作完结束或者再通知A. 实现思路: 1. 设置两个线程组Ⅰ.Ⅱ,组Ⅰ添加cookie管理器,里面添加用户A的操作:组Ⅱ添加 ...