How Region Split works in Hbase
- A region is decided to be split when store file size goes above hbase.hregion.max.filesize or according to defined region split policy.
- At this point this region is divided into two by region server.
- Region server creates two reference files for these two daughter regions.
- These reference files are stored in a new directory called splits under parent directory.
- Exactly at this point, parent region is marked as closed or offline so no client tries to read or write to it.
- Now region server creates two new directories in splits directory for these daughter regions.
- If steps till 6 are completed successfully, Region server moves both daughter region directories under table directory.
- The META table is now informed of the creation of two new regions, along with an update in the entry of parent region that it has now been split and is offline. (OFFLINE=true , SPLIT=true)
- The reference files are very small files containing only the key at which the split happened and also whether it represents top half or bottom half of the parent region.
- There is a class called “HalfHFileReader”which then utilizes these two reference files to read the original data file of parent region and also to decide as which half of the file has to be read.
- Both regions are now brought online by region server and start serving requests to clients.
- As soon as the daughter regions come online, a compaction is scheduled which rewrites the HFile of parent region into two HFiles independent for both daughter regions.
- As this process in step 12 completes, both the HFiles cleanly replace their respective reference files. The compaction activity happens under .tmp directory of daughter regions.
- With the successful completion till step 13, the parent region is now removed from META and all its files and directories marked for deletion.
- Finally Master server is informed by this Region server about two new regions getting born. Master now decides the fate of the two regions as to let them run on same region server or have them travel to another one.
How Region Split works in Hbase的更多相关文章
- 「从零单排HBase 05」核心特性region split
HBase拥有出色的扩展性,其中最依赖的就是region的自动split机制. 1.split触发时机与策略 前面我们已经知道了,数据写入过程中,需要先写memstore,然后memstore满了以后 ...
- region split流程分析
region split流程分析 splitregion的发起主要通过client端调用regionserver.splitRegion或memstore.flsuh时检查并发起. Client通过r ...
- [HBase]region split流程
1. 简介 HBase 的最小管理单位为region,region会按照region 分裂策略进行分裂. 基于CDH5.4.2 2. 总览
- HBASE 基础命令总结
HBASE基础命令总结 一,概述 本文中介绍了hbase的基础命令,作者既有记录总结hbase基础命令的目的还有本着分享的精神,和广大读者一起进步.本文的hbase版本是:HBase 1.2.0-cd ...
- Hbase学习02
第2章 Apache HBase配置 本章在“入门”一章中进行了扩展,以进一步解释Apache HBase的配置. 请仔细阅读本章,特别是基本先决条件,确保您的HBase测试和部署顺利进行,并防止数据 ...
- hbase集群region数量和大小的影响
1.Region数量的影响 通常较少的region数量可使群集运行的更加平稳,官方指出每个RegionServer大约100个regions的时候效果最好,理由如下: 1)Hbase的一个特性MSLA ...
- HBase如何选取split point
hbase region split操作的一些细节,具体split步骤很多文档都有说明,本文主要关注regionserver如何选取split point 首先推荐web ui查看hbase regi ...
- Hbase split的三种方式和split的过程
在Hbase中split是一个很重要的功能,Hbase是通过把数据分配到一定数量的region来达到负载均衡的.一个table会被分配到一个或多个region中,这些region会被分配到一个或者多个 ...
- hbase日常运维管用命令,region管理
1 Hbase日常运维 1.1 监控Hbase运行状况 1.1.1 操作系统 1.1.1.1 IO 群集网络IO,磁盘IO,HDFS IO IO越大说明文件读 ...
随机推荐
- 【Git】git error记录之 "unpacking the sent packfile failed on the remote"
错误信息: error: cannot open .git/FETCH_HEAD: Permission denied unpacking the sent packfile failed on th ...
- Learning-Python【21】:Python常用模块(4)—— re、logging、hashlib、subprocess
re 模块:与正则相关的模块 在使用 re 模块之前,需要先了解正则表达式(regular expression),描述了一种字符串匹配的模式(pattern),可以用来检查一个字符串是否含有某个子字 ...
- NVIDIA 驱动安装(超详细)
目录 1. 删除原有驱动 2. 安装依赖 3. 禁用nouveau驱动: 4. reboot 5. 获取kernel source (important) 6. 关掉x graphic 服务 7. 安 ...
- Query the tables and index which will caus rebuild index fail
On MSSQL server database, while rebuild index failed, we can use the follow sql statement to see if ...
- redis 序列化存入对象
redis 序列化存入对象 //序列化 public static byte [] serialize(Object obj){ ObjectOutputStream obi=null; ByteAr ...
- CSS设置DIV边框为圆角,添加背景色溢出的问题
这么几天需要做一个类似于层级展示的东西,最后一层需要做一些div框来展示数据,我用css设置了div的边框为圆角,但是添加背景色的时候颜色溢出,覆盖了四个角的圆弧,效果如图所示: css代码如下: . ...
- Windows Server 2008环境下Apache2.4+Tomcat8配置
安装步骤 1. 安装配置JDK2. 安装配置Apache3. 安装配置Tomcat4. 启动服务并测试 一.Apache安装与配置 1.Apache解压在D盘根目录下建立一个文件夹Apache Gro ...
- java基础 (一)之HashMap
HashMap的存储结构是由数组和链表共同完成.Entry<K,V>[] ,Entry是单向链表. 1 HashMap数据结构 HashMap的底层主要是基于数组和链表来实现的,它之所以有 ...
- 在pycharm中运行python程序
安装PyCharm 安装过程取决于您的操作系统: 在Windows上安装PyCharm 运行.exe您已下载的文件,并按照PyCharm安装向导的说明进行操作. 在macOS上安装PyCharm 打开 ...
- 异步核心接口IAsyncResult的实现
要实现异步编程,就需要正确的实现IAsyncResult接口.IAsyncResult共有四个属性: public interface IAsyncResult { object AsyncState ...