This is my first post in 2019, and Im starting with a MySQL solution. In MySQL world, implementing a better backup strategy to meet all of your requirement is still a challenging thing. The complexity depends on your RPO and RTO. Percona has many tools to help DBAs in many scenarios. Xtrabackup is one of the best backup tools to perform a better backup on TeraBytes size of databases. Also, another great feature is it supports Incremental and Differential backup.

There are couple of tools which is available in MySQL world, But due to some restrictions we can’t achieve what we are expecting.

#1 mysqldump

This is a widely used backup tool and most of the DBAs are trust this. This comes with the mysql package. Initially, it was a single thread process but now we have multi-thread in native mysqldump. Its completely a logical backup tool. It has a wide range of parameters to use while taking backups (like include routines, events, triggers).

Right place to use mysqldump

Use if your databases are less than 20GB. Because the restore process will take more time. This will affect your RTO.

Advantages:

  • Easy to use
  • More detailed documentation.
  • Multi-Thread (we need to add parameters to perform this)
  • Comes with MySQL, so they will fix if any bugs and implementing new features.

Drawbacks:

  • Backup is multi-thread, but not restore. Its still a single thread process.
  • No incremental backup feature.
  • Restoration takes more time for large backups.

#2 mydumper/myloader

This is an opensource tool and DBAs are using this for dump the huge databases. It has inbuild multi-thread dump and restore feature. This is the pioneer for mysql’s multi thread dump. It supports snapshot consistency. So it’ll provide the accurate binlog file and its position.

The right place to use mydumper/myloader

If you have the databases grater then 20GB to TB/PB. I have used this to dump TB size. If you used this to take dump for a PB size DB using mysqldumper then please comment below

Advantages:

  • Multi-thread backup and restore.
  • Supports for Incremental backups.
  • Log the binlog info, so we can easily built a new slave.
  • We can control the number of threads.
  • Compression support.
  • Include/Exclude tables in backup.
  • Split the table into chunks.
  • And etc,etc.

Drawbacks:

  • This is also a logical dump. So it needs to scan the complete tables.
  • Performance degrade during the backup.
  • No checksum.

#3 Percona Xtrabackup:

This is my favourite opensource tool from Percona. This is also supports incremental backup. The main reason is its very fast since its logical backup. Instead of reading my story just go through this Doc link and see its features.

You can use this to backup >20GB databases to TB or PB.

Advantages:

  • Multi-thread
  • Super fast
  • Compression
  • Checksum
  • Encryption
  • Master/Slave binlog info.
  • Directly restore the backups to AWS Aurora.

Drawback:

  • Not much, I’ll update this section if I found something

Shell Script to Automate Xtrabackup:

Now coming to the automation part. Im using a shell script to automate FULL and Incremental backup also sync them with GCS and S3. This script is already written by bigzaqui at 2013. But I replaced innobackupex with Xtrabackup and few more changes.

Parameters needs to change:

  • -u sqladmin – Mysql user to take the dump. Replace with -u your_user
  • SECRET='mysql-user-password'– Mysql user’s password.
  • --history – Im tacking backup activity into a database. If you don’t want to track just remove this.
  • --slave-info – If you are taking the backup from a slave, it’ll capture the master’s binlog info. So you can setup a new replica for the Master Server. You can remove this if you are not using slave to take backup.
  • --compress-threads=4,---parallel=4 The number of threads needs to perform compress. Its a best practice to allocate 30%-40% from the total cores. I have 40core cpu, so I used 15 threads.
  • --remove-original – While decompressing a compressed backup, it’ll leave the compressed files. So this will remove compressed files after the extraction process.
  • BACKUP_DIR=/mysqldump/xtrabackup/ – Location for saving the dump.
  • DATA_DIR=/mysqldata – Location of mysql data directory. If you added [xtrabackup]parameters in my.cnf file, then no need to mention this.

Run this script:

  • FULL Backup – ./xtrabackup_full_increment_restore.sh full
  • Incremental Backup – ./xtrabackup_full_increment_restore.sh incremental
  • Restore – ./xtrabackup_full_increment_restore.sh restore

Automation Script For Percona Xtrabackup FULL/Incremental的更多相关文章

  1. Percona XtraBackup 核心文档

    1. 介绍 1.1 MySQL 备份工具特性对比 Features Percona XtraBackup MySQL Enterprise backup License GPL Proprietary ...

  2. (转)CentOS 7 下 MySQL 5.7 配置 Percona Xtrabackup

    CentOS 7 下 MySQL 5.7 配置 Percona Xtrabackup 原文:http://qizhanming.com/blog/2017/05/10/install-percona- ...

  3. Percona XtraBackup User Manual 阅读笔记

    XtraBackup XtraBackup 2 安装XtraBackup 2.1 安装XtraBackup binary版本 2.1.1 yum的安装方法: 2.1.2 直接下载rpm包安装 3 Xt ...

  4. MariaDB之基于Percona Xtrabackup备份大数据库[完整备份与增量备份]

    MariaDB之基于Percona Xtrabackup备份大数据库[完整备份与增量备份] 1.Xtrabackup的安装 percona-xtrabackup-2.2.3-4982.el6.x86_ ...

  5. Percona Xtrabackup备份mysql(转)

    add by zhj:另外,参考了Xtrabackup之innobackupex备份恢复详解,我用的是Xtrabackup2.2.6版本, 可以成功备份和恢复指定的数据库. 原文:http://www ...

  6. Percona Xtrabackup备份mysql全库及指定数据库(完整备份与增量备份)

    原文地址:http://www.tuicool.com/articles/RZRnq2 Xtrabackup简介 Percona XtraBackup是开源免费的MySQL数据库热备份软件,它能对In ...

  7. 通过Percona Xtrabackup实现数据的备份与恢复

    Xtrabackup简介 Percona XtraBackup是一个开源.免费的MySQL热备份软件,能够为InnoDB和XtraDB数据库执行非阻塞备份,特点如下: 1.快速.可靠的完成备份 2.备 ...

  8. Percona XtraBackup 关于 MySQL备份还原的详细测试

    一. Percona XtraBackup 的优点. (1)无需停止数据库进行InnoDB热备: (2)增量备份MySQL: (3)流压缩传输到其它服务器: (4)在线移动表: (5)能够比较容易地创 ...

  9. Percona XtraBackup使用说明(转)

    Percona XtraBackup使用说明 转载出自: https://blog.csdn.net/wfs1994/article/details/80396604 XtraBackup介绍 Per ...

随机推荐

  1. PTA (Advanced Level) 1008 Elevator

    Elevator The highest building in our city has only one elevator. A request list is made up with Npos ...

  2. Qt编写自定义控件12-进度仪表盘

    前言 进度仪表盘主要应用场景是标识一个任务进度完成的状况等,可以自由的设置范围值和当前值,为了美观还提供了四种指示器(圆形指示器/指针指示器/圆角指针指示器/三角形指示器),各种颜色都可以设置,其中的 ...

  3. netty源码解解析(4.0)-4 线程模型-概览

    netty线程体系概览 netty的高并发能力很大程度上由它的线程模型决定的,netty定义了两种类型的线程: I/O线程: EventLoop, EventLoopGroup.一个EventLoop ...

  4. OAuth2.0的理解&基础

    此文章是复制黏贴网上文章的,主要做自己备用着看(也加了自己的一点见解),喜欢的读者也可以看. OAuth是一个关于授权(authorization)的开放网络标准,在全世界得到广泛应用,目前的版本是2 ...

  5. [转]Webpack 入门教程

    本文转自:http://www.runoob.com/w3cnote/webpack-tutorial.html Webpack 是一个前端资源加载/打包工具.它将根据模块的依赖关系进行静态分析,然后 ...

  6. C# 全文搜索Lucene

    全文出自:https://blog.csdn.net/huangwenhua5000/article/details/9341751 1 lucene简介1.1 什么是luceneLucene是一个全 ...

  7. [原创]C#中的堆和栈理解

    引言:程序运行时,它的数据必须存在内存中,一个数据需要多大内存.存储在什么地方以及如何存储都依赖于该数据的数据类型. 1.什么是栈 栈是一个内存数组,是一个LIFO(Last-In-First-Out ...

  8. windows 下 MyEclipse 逆向工程生成hiberate 对应配置文件以及 javaBean。

    步骤1: 右边工具栏 ->  右击你的项目 -> 选中 MyEclipse  -> Project Facets -> install Hibernate Facet -> ...

  9. Linux常用基本命令wc-统计文件的字节,字符,行数

    wc命令 作用:统计文件的字节,单词,行数 用法:wc [option] [file] -c:统计字节 ghostwu@dev:~/linux/uniq$ cat ghostwu.txt 192.16 ...

  10. BOM(JavaScript高程笔记)

    再次编辑于20160115 一.window对象 双重角色 JS访问浏览器窗口的接口 ECAMAscript规定的Global对象 1.全局作用域 所有在全局作用域中声明的变量.函数都会变成windo ...