hbase 分页过滤（新老API的差别）
在hbase2.0以前分页过滤必须以上一次的最后一行+空字节数组作为下一次的起始行，
因为scan扫描的时候是包含起始行的，为了既能准确定位起始行，但又不重复把上一次的最末一行加入下一页，
所以，权威指南里才有了加上空字节数组的处理。
hbase2.0以后，新的api是withStartRow(byte[] startRow, boolean inclusive)，可以直接设置是否包含起始行，完美解决问题，但是又保留了对以前api函数的兼容性。
//分页过滤
private static void pageFilterData() throws IOException{
Table table = helper.getConnection().getTable(TableName.valueOf("testtable"));
final byte[] POSTFIX = new byte[] { 0x00 };
Filter filter = new PageFilter(10); int totalRows = 0;
byte[] lastRow = null;
while(true){
Scan scan = new Scan();
scan.setFilter(filter);
if(lastRow!=null){ //为了兼容以前的scan.setStartRow()代码
//在上一次的最后一行加上一个空的byte数据，在下一个分页上，就会以新的key开始，
// 但是实际上这个key并不存在，所以还是从真正的下一行开始扫描
//这么做的原因是scan的扫描会自动包含起始行，如果不加空字节数据，那么定位上就会把上一次的最后一行作为起始行，最后的数据就会多一行。
//而，新的api是withStartRow(byte[] startRow, boolean inclusive)，可以直接设置是否包含起始行，完美解决问题，但是又保留了对
//以前api函数的兼容性
// byte[] startRow = Bytes.add(lastRow,POSTFIX);
// System.out.println("start row: " + Bytes.toStringBinary(startRow));
// scan.withStartRow(startRow,true); System.out.println("start row: " + Bytes.toStringBinary(lastRow));
//不包含起始行，所以可以直接使用上一次的最后一行作为起始行
scan.withStartRow(lastRow,false);
}
ResultScanner scanner = table.getScanner(scan);
int localRows = 0;
Result result;
while ((result=scanner.next())!=null){
System.out.println(localRows++ + ": " + result);
totalRows++;
lastRow = result.getRow();
}
scanner.close();
if(localRows==0)break;
}
System.out.println("total rows: " + totalRows);
}
兼容老API输出如下：
0: keyvalues={rowKey0/ex:addr/1555078771906/Put/vlen=7/seqid=0, rowKey0/info:username/1555078771906/Put/vlen=5/seqid=0, rowKey0/memo:detail/1555078771906/Put/vlen=7/seqid=0}
1: keyvalues={rowKey1/ex:addr/1555078771906/Put/vlen=7/seqid=0, rowKey1/info:username/1555078771906/Put/vlen=5/seqid=0, rowKey1/memo:detail/1555078771906/Put/vlen=7/seqid=0}
2: keyvalues={rowKey10/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey10/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey10/memo:detail/1555078771906/Put/vlen=8/seqid=0}
3: keyvalues={rowKey11/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey11/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey11/memo:detail/1555078771906/Put/vlen=8/seqid=0}
4: keyvalues={rowKey12/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey12/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey12/memo:detail/1555078771906/Put/vlen=8/seqid=0}
5: keyvalues={rowKey13/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey13/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey13/memo:detail/1555078771906/Put/vlen=8/seqid=0}
6: keyvalues={rowKey14/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey14/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey14/memo:detail/1555078771906/Put/vlen=8/seqid=0}
7: keyvalues={rowKey15/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey15/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey15/memo:detail/1555078771906/Put/vlen=8/seqid=0}
8: keyvalues={rowKey16/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey16/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey16/memo:detail/1555078771906/Put/vlen=8/seqid=0}
9: keyvalues={rowKey17/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey17/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey17/memo:detail/1555078771906/Put/vlen=8/seqid=0}
start row: rowKey17\x00
0: keyvalues={rowKey18/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey18/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey18/memo:detail/1555078771906/Put/vlen=8/seqid=0}
1: keyvalues={rowKey19/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey19/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey19/memo:detail/1555078771906/Put/vlen=8/seqid=0}
2: keyvalues={rowKey2/ex:addr/1555078771906/Put/vlen=7/seqid=0, rowKey2/info:username/1555078771906/Put/vlen=5/seqid=0, rowKey2/memo:detail/1555078771906/Put/vlen=7/seqid=0}
3: keyvalues={rowKey20/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey20/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey20/memo:detail/1555078771906/Put/vlen=8/seqid=0}
4: keyvalues={rowKey21/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey21/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey21/memo:detail/1555078771906/Put/vlen=8/seqid=0}
5: keyvalues={rowKey22/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey22/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey22/memo:detail/1555078771906/Put/vlen=8/seqid=0}
6: keyvalues={rowKey23/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey23/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey23/memo:detail/1555078771906/Put/vlen=8/seqid=0}
7: keyvalues={rowKey24/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey24/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey24/memo:detail/1555078771906/Put/vlen=8/seqid=0}
8: keyvalues={rowKey25/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey25/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey25/memo:detail/1555078771906/Put/vlen=8/seqid=0}
9: keyvalues={rowKey26/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey26/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey26/memo:detail/1555078771906/Put/vlen=8/seqid=0}
start row: rowKey26\x00
0: keyvalues={rowKey27/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey27/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey27/memo:detail/1555078771906/Put/vlen=8/seqid=0}
1: keyvalues={rowKey28/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey28/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey28/memo:detail/1555078771906/Put/vlen=8/seqid=0}
2: keyvalues={rowKey29/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey29/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey29/memo:detail/1555078771906/Put/vlen=8/seqid=0}
3: keyvalues={rowKey3/ex:addr/1555078771906/Put/vlen=7/seqid=0, rowKey3/info:username/1555078771906/Put/vlen=5/seqid=0, rowKey3/memo:detail/1555078771906/Put/vlen=7/seqid=0}
4: keyvalues={rowKey30/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey30/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey30/memo:detail/1555078771906/Put/vlen=8/seqid=0}
5: keyvalues={rowKey31/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey31/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey31/memo:detail/1555078771906/Put/vlen=8/seqid=0}
6: keyvalues={rowKey32/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey32/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey32/memo:detail/1555078771906/Put/vlen=8/seqid=0}
7: keyvalues={rowKey33/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey33/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey33/memo:detail/1555078771906/Put/vlen=8/seqid=0}
8: keyvalues={rowKey34/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey34/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey34/memo:detail/1555078771906/Put/vlen=8/seqid=0}
9: keyvalues={rowKey35/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey35/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey35/memo:detail/1555078771906/Put/vlen=8/seqid=0}
start row: rowKey35\x00
... ...省略其他数据
起始行加了空字节数据，形成了新的定位行（相当于真正的两页之间插入了一个虚拟行用于定位），解决了下一页首行的问题。
新的api输出：下一页的定位行就是上一页的末行，采用新的API设置首行的时候把inclusive设置成false（即不包含首行），那么输出结果就是正确的。
0: keyvalues={rowKey0/ex:addr/1555078771906/Put/vlen=7/seqid=0, rowKey0/info:username/1555078771906/Put/vlen=5/seqid=0, rowKey0/memo:detail/1555078771906/Put/vlen=7/seqid=0}
1: keyvalues={rowKey1/ex:addr/1555078771906/Put/vlen=7/seqid=0, rowKey1/info:username/1555078771906/Put/vlen=5/seqid=0, rowKey1/memo:detail/1555078771906/Put/vlen=7/seqid=0}
2: keyvalues={rowKey10/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey10/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey10/memo:detail/1555078771906/Put/vlen=8/seqid=0}
3: keyvalues={rowKey11/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey11/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey11/memo:detail/1555078771906/Put/vlen=8/seqid=0}
4: keyvalues={rowKey12/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey12/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey12/memo:detail/1555078771906/Put/vlen=8/seqid=0}
5: keyvalues={rowKey13/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey13/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey13/memo:detail/1555078771906/Put/vlen=8/seqid=0}
6: keyvalues={rowKey14/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey14/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey14/memo:detail/1555078771906/Put/vlen=8/seqid=0}
7: keyvalues={rowKey15/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey15/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey15/memo:detail/1555078771906/Put/vlen=8/seqid=0}
8: keyvalues={rowKey16/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey16/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey16/memo:detail/1555078771906/Put/vlen=8/seqid=0}
9: keyvalues={rowKey17/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey17/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey17/memo:detail/1555078771906/Put/vlen=8/seqid=0}
start row: rowKey17
0: keyvalues={rowKey18/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey18/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey18/memo:detail/1555078771906/Put/vlen=8/seqid=0}
1: keyvalues={rowKey19/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey19/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey19/memo:detail/1555078771906/Put/vlen=8/seqid=0}
2: keyvalues={rowKey2/ex:addr/1555078771906/Put/vlen=7/seqid=0, rowKey2/info:username/1555078771906/Put/vlen=5/seqid=0, rowKey2/memo:detail/1555078771906/Put/vlen=7/seqid=0}
3: keyvalues={rowKey20/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey20/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey20/memo:detail/1555078771906/Put/vlen=8/seqid=0}
4: keyvalues={rowKey21/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey21/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey21/memo:detail/1555078771906/Put/vlen=8/seqid=0}
5: keyvalues={rowKey22/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey22/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey22/memo:detail/1555078771906/Put/vlen=8/seqid=0}
6: keyvalues={rowKey23/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey23/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey23/memo:detail/1555078771906/Put/vlen=8/seqid=0}
7: keyvalues={rowKey24/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey24/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey24/memo:detail/1555078771906/Put/vlen=8/seqid=0}
8: keyvalues={rowKey25/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey25/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey25/memo:detail/1555078771906/Put/vlen=8/seqid=0}
9: keyvalues={rowKey26/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey26/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey26/memo:detail/1555078771906/Put/vlen=8/seqid=0}
start row: rowKey26
0: keyvalues={rowKey27/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey27/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey27/memo:detail/1555078771906/Put/vlen=8/seqid=0}
1: keyvalues={rowKey28/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey28/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey28/memo:detail/1555078771906/Put/vlen=8/seqid=0}
2: keyvalues={rowKey29/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey29/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey29/memo:detail/1555078771906/Put/vlen=8/seqid=0}
3: keyvalues={rowKey3/ex:addr/1555078771906/Put/vlen=7/seqid=0, rowKey3/info:username/1555078771906/Put/vlen=5/seqid=0, rowKey3/memo:detail/1555078771906/Put/vlen=7/seqid=0}
4: keyvalues={rowKey30/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey30/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey30/memo:detail/1555078771906/Put/vlen=8/seqid=0}
5: keyvalues={rowKey31/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey31/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey31/memo:detail/1555078771906/Put/vlen=8/seqid=0}
6: keyvalues={rowKey32/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey32/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey32/memo:detail/1555078771906/Put/vlen=8/seqid=0}
7: keyvalues={rowKey33/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey33/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey33/memo:detail/1555078771906/Put/vlen=8/seqid=0}
8: keyvalues={rowKey34/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey34/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey34/memo:detail/1555078771906/Put/vlen=8/seqid=0}
9: keyvalues={rowKey35/ex:addr/1555078771906/Put/vlen=8/seqid=0, rowKey35/info:username/1555078771906/Put/vlen=6/seqid=0, rowKey35/memo:detail/1555078771906/Put/vlen=8/seqid=0}
start row: rowKey35
... ...其他省略
hbase 分页过滤（新老API的差别）的更多相关文章
- Hadoop2.2编程：新旧API的区别
Hadoop最新版本的MapReduce Release 0.20.0的API包括了一个全新的Mapreduce JAVA API,有时候也称为上下文对象. 新的API类型上不兼容以前的API,所以, ...
- Android 新老两代 Camera API 大起底
https://blog.csdn.net/Byeweiyang/article/details/80515192 0.背景简介 最近有一部分相机相关的需求,专注于对拍摄的照片.视频的噪点.色温.明暗 ...
- Java 8的新并行API - 魅力与炫目背后
这是一篇译文,原文链接见这里. 本文同时发表在ImportNew上,转载请注明出处. 我很擅长同时处理多项任务.就算是在写这篇博客的此刻,我仍然在为昨天在聚会上发表了一个让大家都感到诧异的评论而觉得尴 ...
- hbase分页查询
为了广大技术爱好者学习netty,在这里帮新浪微博@nettying宣传下他出版的新书 <netty权威指南>@nettying兄在华为NIO实践多年,这本书是他的技术和经验的一个结晶.N ...
- Hadoop日记Day15---MapReduce新旧api的比较
我使用hadoop的是hadoop1.1.2,而很多公司也在使用hadoop0.2x版本,因此市面上的hadoop资料版本不一,为了扩充自己的知识面,MapReduce的新旧api进行了比较研究. h ...
- hbase分页应用场景及分页思路与代码实现
转自:http://www.aboutyun.com/forum.php?mod=viewthread&tid=7030&extra=page=1 可以带着下面问题来阅读1.hbase ...
- Kafka 0.9 新消费者API
kafka诞生之初,它自带一个基于scala的生产者和消费者客户端.但是慢慢的我们认识到这些API有很多限制.比如,消费者有一个“高级”API支持分组和异常控制,但是不支持很多更复杂的应用场景:它也有 ...
- MapReduce简述、工作流程及新旧API对照
什么是MapReduce? 你想数出一摞牌中有多少张黑桃.直观方式是一张一张检查而且数出有多少张是黑桃. MapReduce方法则是: 1. 给在座的全部玩家中分配这摞牌. 2. 让每一个玩家数自己手 ...
- Hbase 分页设计
hbase 数据获取方式 直接根据 rowkey 查找,速度最快 scan,指定 startrowkey.endrowkey 和 limit获取数据,在 rowkey 设计良好的情况下,效率也不错 全 ...
随机推荐
- linux设置禁止ping
linux禁止ping为了服务器的安全, 防止网络攻击(DOS 攻击消耗网络宽带,CPU资源), 需要服务器设置 禁止ping通常有两种方式第一种是通过防火墙 iptables 设置第二种是内核设置 ...
- jenkins使用时出现的问题！
从安装到日常使用中遇到过的问题和解决方法: 背景/问题:安装时是跳过安装插件过程的,安装好后,我发现里面啥也做不了,连个git的插件都无法下载. 方法:在jenkins的主界面,打开系统管理= ...
- Spring AOP 报错org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'XXXXXX' defined in class path resource..........
完整报错如下: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'befo ...
- 出现 org.springframework.beans.factory.BeanCreationException 异常的原因及解决方法
1 异常描述 在从 SVN 检出项目并配置完成后,启动 Tomcat 服务器,报出如下错误: 2 异常原因 通过观察上图中被标记出来的异常信息,咱们可以知道 org.springframework.b ...
- PHP has encountered an Access Violation at 01F4A622解决方法
php搭建的网站出现以下问题的解决方法分享: Z-blog,DedeCMS,Dsicuz!,PhpWind,PhpCMS,帝国CMS等都有可能出现php访问冲突问题. 今天访问网站发现出现了一个错误& ...
- 基本数据结构 -- 栈简介（C语言实现）
栈是一种后进先出的线性表,是最基本的一种数据结构,在许多地方都有应用. 一.什么是栈 栈是限制插入和删除只能在一个位置上进行的线性表.其中,允许插入和删除的一端位于表的末端,叫做栈顶(top),不允许 ...
- 从零开始的Python学习Episode 16——模块
一.模块 在计算机程序的开发过程中,随着程序代码越写越多,在一个文件里代码就会越来越长,越来越不容易维护. 为了编写可维护的代码,我们把很多函数分组,分别放到不同的文件里,这样,每个文件包含的代码就相 ...
- Linux 做网关
首先创建两张路由表,只需要添加到相应的文件中即可,Linux一共支持255个路由表,rt_tables文件中默认已经存在了三张路由表,分别是: 255 local 254 main 2 ...
- 最小费用最大流模板（POJ 2135-Farm Tour）
最近正好需要用到最小费用最大流,所以网上就找了这方面的代码,动手写了写,先在博客里存一下~ 代码的题目是POJ2135-Farm Tour 需要了解算法思想的,可以参考下面一篇文章,个人觉得有最大流基 ...
- sprint2（第二天）
昨天没有想到餐桌的功能,今天加到展板.然后今天我们完成了餐桌模板,可以实现添加桌子的桌号.人数.修改和删除功能.不过由于今天学校网络不是很好,晚上我们clone了很久都没clone下来,所以没有上传代 ...