Lucene 4.6.1 java.lang.IllegalStateException: TokenStream contract violation
这是旧代码在新版本Lucene中出现的异常,异常如下:
Exception in thread "main" java.lang.IllegalStateException: TokenStream contract violation: reset()/close() call missing, reset() called multiple times, or subclass does not call super.reset(). Please see Javadocs of TokenStream class for more information about the correct consuming workflow.
at org.apache.lucene.analysis.Tokenizer$1.read(Tokenizer.java:110)
at java.io.Reader.read(Reader.java:140)
at org.wltea.analyzer.core.AnalyzeContext.fillBuffer(AnalyzeContext.java:124)
at org.wltea.analyzer.core.IKSegmenter.next(IKSegmenter.java:122)
at org.wltea.analyzer.lucene.IKTokenizer.incrementToken(IKTokenizer.java:78)
at com.hankcs.train.IKHelper.parse(IKHelper.java:36)
at com.hankcs.train.AnalysisAdjuster.handleFile(AnalysisAdjuster.java:44)
at com.hankcs.train.AnalysisAdjuster.main(AnalysisAdjuster.java:37)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)Process finished with exit code 1
旧代码:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
|
IKAnalyzer ss = new IKAnalyzer(); StringReader reader = new StringReader(str); try { TokenStream tokenStream = ss.tokenStream( "" , reader); while (tokenStream.incrementToken()) { CharTermAttribute termAttribute = tokenStream.getAttribute(CharTermAttribute. class ); System.out.println(termAttribute.toString()); } } catch (IOException e) { e.printStackTrace(); } |
根据新的API文档,调用TokenStream API的流程必须是:
The workflow of the new
TokenStream
API is as follows:
Instantiation of
TokenStream
/TokenFilter
s which add/get attributes to/from theAttributeSource
.The consumer calls
reset()
.The consumer retrieves attributes from the stream and stores local references to all attributes it wants to access.
The consumer calls
incrementToken()
until it returns false consuming the attributes after each call.The consumer calls
end()
so that any end-of-stream operations can be performed.The consumer calls
close()
to release any resource when finished using theTokenStream
.
所以代码必须在incrementToken()
之前调用一次reset()
:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
|
IKAnalyzer ss = new IKAnalyzer(); StringReader reader = new StringReader(str); try { TokenStream tokenStream = ss.tokenStream( "" , reader); tokenStream.reset(); while (tokenStream.incrementToken()) { CharTermAttribute termAttribute = tokenStream.getAttribute(CharTermAttribute. class ); System.out.println(termAttribute.toString()); } } catch (IOException e) { e.printStackTrace(); } |
转载请注明:码农场 » Lucene 4.6.1 java.lang.IllegalStateException: TokenStream contract violation
Lucene 4.6.1 java.lang.IllegalStateException: TokenStream contract violation的更多相关文章
- Lucene分词报错:”TokenStream contract violation: close() call missing”
Lucene使用IKAnalyzer分词时报错:”TokenStream contract violation: close() call missing” 解决办法是每次完成后必须调用关闭方法. ...
- ES failed to notify ClusterStateListener java.lang.IllegalStateException: environment is not locked
ES出现异常: failed to notify ClusterStateListenerjava.lang.IllegalStateException: environment is not loc ...
- myeclipse 无法启动 java.lang.IllegalStateException: Unable to acquire application service. Ensure that the org.eclipse.core.runtime bundle is resolved and started (see config.ini).
把myeclipse10 按照目录完整拷贝到了另外一台电脑, 另外的目录 原安装目录 D\:\soft\i\myeclipse10 新安装目录 E\:\soft\myeclipse10 双击启动失败, ...
- java.lang.IllegalStateException:Couldn't read row 0, col -1 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.
java.lang.RuntimeException: Unable to start activity ComponentInfo{com.xxx...}: java.lang.IllegalSta ...
- java.lang.IllegalStateException: Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead
java.lang.IllegalStateException: Not allowed to create transaction on sharedEntityManager - use Spri ...
- java.lang.IllegalStateException: getOutputStream() has already been called for this response
ERROR [Engine] StandardWrapperValve[jsp]: Servlet.service() for servlet jsp threw exceptionjava.lang ...
- 用java实现文件下载,提示java.lang.IllegalStateException: getOutputStream() has already been called for this response
1. 用java实现文件下载,提示java.lang.IllegalStateException: getOutputStream() has already been called for this ...
- eclipse启动报错java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' befo
报错: java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' before invo ...
- java.lang.IllegalStateException: Couldn't read row 1, col 0 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data fr
Android中操作Sqlite遇到的错误:java.lang.IllegalStateException: Couldn't read row 1, col 0 from CursorWindow. ...
随机推荐
- 「版本升级」界面控件Kendo UI正式发布R2 2019|附下载
通过70多个可自定义的UI组件,Kendo UI可以创建数据丰富的桌面.平板和移动Web应用程序.通过响应式的布局.强大的数据绑定.跨浏览器兼容性和即时使用的主题,Kendo UI将开发时间加快了50 ...
- MySQL简版(二)
第一章 表的约束 1.1 概念 对表中的数据进行限定,保证数据的正确性.有效性和完整性. 1.2 分类 主键约束:primary key. 非空约束:not null. 唯一约束:unique. 外键 ...
- Linux GDB 程序调试工具使用详解
转自 http://www.codeceo.com/article/linux-gdb-tools.html 整理的挺全的 GDB概述 GDB是GNU开源组织发布的一个强大的UNIX下的程序调试 ...
- @PathVariable注解使用
@PathVariable是spring3.0的一个新功能:接收请求路径中占位符的值 语法: @PathVariable("xxx")通过 @PathVariable 可以将URL ...
- vue路由传参并跳转页面
在vue项目中参数的传递可以使用本地缓存或者Vuex,那么vue能不能像小程序一样路由传参呢,显然是可以的而且非常简单 方式一:query传参 //传参 go(){ that.$router.push ...
- luogu P1036 选数 x
P1036 选数 题目描述 已知 n 个整数 x1,x2,…,xn,以及一个整数 k(k<n).从 n 个整数中任选 k 个整数相加,可分别得到一系列的和.例如当 n=4,k=3,4 个整数分别 ...
- 【ELK】elasticsearch设置密码访问
1.需要在配置文件中开启x-pack验证, 修改config目录下面的elasticsearch.yml文件,在里面添加如下内容,并重启. xpack.security.enabled: true x ...
- IIS新建项目
我在写网站的时候要 经常帮公司的游戏写界面,他们用的都是VS,然后要用IIS创建项目(配置环境)然后页面才能够在浏览器上看效果. 那就要先了解啥是IIS: IIS是Internet Informati ...
- POJ 1380 Equipment Box (暴力枚举)
Equipment Box 题目链接: http://acm.hust.edu.cn/vjudge/contest/130510#problem/B Description There is a la ...
- bootstrap动态调用select下拉框
html代码: <label for="classify" class="col-sm-2 control-label">填报部门:</lab ...