How Region Split works in Hbase
- A region is decided to be split when store file size goes above hbase.hregion.max.filesize or according to defined region split policy.
- At this point this region is divided into two by region server.
- Region server creates two reference files for these two daughter regions.
- These reference files are stored in a new directory called splits under parent directory.
- Exactly at this point, parent region is marked as closed or offline so no client tries to read or write to it.
- Now region server creates two new directories in splits directory for these daughter regions.
- If steps till 6 are completed successfully, Region server moves both daughter region directories under table directory.
- The META table is now informed of the creation of two new regions, along with an update in the entry of parent region that it has now been split and is offline. (OFFLINE=true , SPLIT=true)
- The reference files are very small files containing only the key at which the split happened and also whether it represents top half or bottom half of the parent region.
- There is a class called “HalfHFileReader”which then utilizes these two reference files to read the original data file of parent region and also to decide as which half of the file has to be read.
- Both regions are now brought online by region server and start serving requests to clients.
- As soon as the daughter regions come online, a compaction is scheduled which rewrites the HFile of parent region into two HFiles independent for both daughter regions.
- As this process in step 12 completes, both the HFiles cleanly replace their respective reference files. The compaction activity happens under .tmp directory of daughter regions.
- With the successful completion till step 13, the parent region is now removed from META and all its files and directories marked for deletion.
- Finally Master server is informed by this Region server about two new regions getting born. Master now decides the fate of the two regions as to let them run on same region server or have them travel to another one.
How Region Split works in Hbase的更多相关文章
- 「从零单排HBase 05」核心特性region split
HBase拥有出色的扩展性,其中最依赖的就是region的自动split机制. 1.split触发时机与策略 前面我们已经知道了,数据写入过程中,需要先写memstore,然后memstore满了以后 ...
- region split流程分析
region split流程分析 splitregion的发起主要通过client端调用regionserver.splitRegion或memstore.flsuh时检查并发起. Client通过r ...
- [HBase]region split流程
1. 简介 HBase 的最小管理单位为region,region会按照region 分裂策略进行分裂. 基于CDH5.4.2 2. 总览
- HBASE 基础命令总结
HBASE基础命令总结 一,概述 本文中介绍了hbase的基础命令,作者既有记录总结hbase基础命令的目的还有本着分享的精神,和广大读者一起进步.本文的hbase版本是:HBase 1.2.0-cd ...
- Hbase学习02
第2章 Apache HBase配置 本章在“入门”一章中进行了扩展,以进一步解释Apache HBase的配置. 请仔细阅读本章,特别是基本先决条件,确保您的HBase测试和部署顺利进行,并防止数据 ...
- hbase集群region数量和大小的影响
1.Region数量的影响 通常较少的region数量可使群集运行的更加平稳,官方指出每个RegionServer大约100个regions的时候效果最好,理由如下: 1)Hbase的一个特性MSLA ...
- HBase如何选取split point
hbase region split操作的一些细节,具体split步骤很多文档都有说明,本文主要关注regionserver如何选取split point 首先推荐web ui查看hbase regi ...
- Hbase split的三种方式和split的过程
在Hbase中split是一个很重要的功能,Hbase是通过把数据分配到一定数量的region来达到负载均衡的.一个table会被分配到一个或多个region中,这些region会被分配到一个或者多个 ...
- hbase日常运维管用命令,region管理
1 Hbase日常运维 1.1 监控Hbase运行状况 1.1.1 操作系统 1.1.1.1 IO 群集网络IO,磁盘IO,HDFS IO IO越大说明文件读 ...
随机推荐
- Shell 文本处理命令
命令:cut –d’:’ -f1, 文件名 #切割处文件列的参数. -d切割字符. -f列的第几个参数. -c1-10指定字符串范围行的第一个到第十个. 命令:sort 文件名 #根据第一列第一个字符 ...
- springboot跑定时任务
使用@Scheduled注解实现 1.在启动类上加上@EnableScheduling 开启定时任务 2.新建一个任务类,在方法上添加@Scheduled注解 @Componentpublic cla ...
- JS设计模式(3)代理模式
什么是代理模式? 情景:小明追女生 A 非代理模式:小明 =花=> 女生A 代理模式:小明 =花=> 让女生A的好友B帮忙 =花=> 女生A 定义:为其他对象提供一种代理以控制对这个 ...
- Vue常见指令
文本相关指令 <div id="app"> <!-- 插值表达式 --> <p>{{ msg }}</p> <!-- eg:原 ...
- svg(一)
<svg width="100%" height="100%" version="1.1" xmlns="http://ww ...
- English trip EM2-LP-5A Food Teacher:Lamb
课上内容(Lesson) 1. Do you know any of these foods? Yes I do. There are hot pepper(小辣椒), a clove of gar ...
- 来自师兄的Django2.0笔记摘录
翻看一本书,觉得对于Django的日志处理.权限处理包括Django-guardian还没学清楚,然后书上关于mysql深入学习.elasticsearch.redis.celery分布式都没学过. ...
- 根据框架的dtd或xsd生成xml文件
下载Schema文件 首先要下载框架官网下的xsd或者xml文件,例如Spring官网地址,schema里面的就是xsd文件 Myeclipse中配置 我用的Myeclipse纯净版6.5,Windo ...
- Log4Net 添加自定义字段并保存到数据库
Log4Net是常用的功能强大的日志插件,该插件提供了几个默认字段 大家可能都用过Log4Net插件来记录日志,该插件默认提供了这几个字段@log_date, @thread, @log_level, ...
- 『OpenCV3』滤波器实现及使用滤波器降噪
一.滤波器实现 我们实现这样一个基于拉普拉斯算子的滤波器核心,并使用它进行滤波,这可以做到锐化图像的效果, 0 -1 0 -1 5 -1 0 -1 0 首先我们完全手动的进行滤波,依赖指针操作, vo ...