A Go library implementing an FST (finite state transducer)——mark下
https://github.com/couchbaselabs/vellum
Building an FST
To build an FST, create a new builder using the New()
method. This method takes an io.Writer
as an argument. As the FST is being built, data will be streamed to the writer as soon as possible. With this builder you MUST insert keys in lexicographic order. Inserting keys out of order will result in an error. After inserting the last key into the builder, you MUST call Close()
on the builder. This will flush all remaining data to the underlying writer.
In memory:
var buf bytes.Buffer
builder, err := vellum.New(&buf, nil)
if err != nil {
log.Fatal(err)
}
To disk:
f, err := os.Create("/tmp/vellum.fst")
if err != nil {
log.Fatal(err)
}
builder, err := vellum.New(f, nil)
if err != nil {
log.Fatal(err)
}
MUST insert keys in lexicographic order:
err = builder.Insert([]byte("cat"), 1)
if err != nil {
log.Fatal(err)
} err = builder.Insert([]byte("dog"), 2)
if err != nil {
log.Fatal(err)
} err = builder.Insert([]byte("fish"), 3)
if err != nil {
log.Fatal(err)
} err = builder.Close()
if err != nil {
log.Fatal(err)
}
Using an FST
After closing the builder, the data can be used to instantiate an FST. If the data was written to disk, you can use the Open()
method to mmap the file. If the data is already in memory, or you wish to load/mmap the data yourself, you can instantiate the FST with the Load()
method.
Load in memory:
fst, err := vellum.Load(buf.Bytes())
if err != nil {
log.Fatal(err)
}
Open from disk:
fst, err := vellum.Open("/tmp/vellum.fst")
if err != nil {
log.Fatal(err)
}
Get key/value:
val, exists, err = fst.Get([]byte("dog"))
if err != nil {
log.Fatal(err)
}
if exists {
fmt.Printf("contains dog with val: %d\n", val)
} else {
fmt.Printf("does not contain dog")
}
Iterate key/values:
itr, err := fst.Iterator(startKeyInclusive, endKeyExclusive)
for err == nil {
key, val := itr.Current()
fmt.Printf("contains key: %s val: %d", key, val)
err = itr.Next()
}
if err != nil {
log.Fatal(err)
}
How does the FST get built?
A full example of the implementation is beyond the scope of this README, but let's consider a small example where we want to insert 3 key/value pairs.
First we insert "are" with the value 4.
Next, we insert "ate" with the value 2.
Notice how the values associated with the transitions were adjusted so that by summing them while traversing we still get the expected value.
At this point, we see that state 5 looks like state 3, and state 4 looks like state 2. But, we cannot yet combine them because future inserts could change this.
Now, we insert "see" with value 3. Once it has been added, we now know that states 5 and 4 can longer change. Since they are identical to 3 and 2, we replace them.
Again, we see that states 7 and 8 appear to be identical to 2 and 3.
Having inserted our last key, we call Close()
on the builder.
Now, states 7 and 8 can safely be replaced with 2 and 3.
For additional information, see the references at the bottom of this document.
A Go library implementing an FST (finite state transducer)——mark下的更多相关文章
- Finite State Transducers
一, 简介 Finite State Transducers 简称 FST, 中文名:有穷状态转换器.在自然语言处理等领域有很大应用,其功能类似于字典的功能(STL 中的map,C# 中的Dictio ...
- Finite State Machine 是什么?
状态机(Finite State Machine):状态机由状态寄存器和组合逻辑电路构成,能够根据控制信号按照预先设定的状态进行状态转移,是协调相关信号动 作.完成特定操作的控制中心. 类 ...
- Finite State Machine
Contents [hide] 1 Description 2 Components 3 C# - FSMSystem.cs 4 Example Description This is a Dete ...
- 证明与计算(7): 有限状态机(Finite State Machine)
什么是有限状态机(Finite State Machine)? 什么是确定性有限状态机(deterministic finite automaton, DFA )? 什么是非确定性有限状态机(nond ...
- paper:synthesizable finite state machine design techniques using the new systemverilog 3.0 enhancements 之 standard verilog FSM conding styles(二段式)
1.Two always block style with combinational outputs(Good Style) 对应的代码如下: 2段式总结: (1)the combinational ...
- paper:synthesizable finite state machine design techniques using the new systemverilog 3.0 enhancements 之 FSM Coding Goals
1.the fsm coding style should be easily modifiable to change state encoding and FSM styles. FSM 的的 状 ...
- FPGA学习笔记(七)——FSM(Finite State Machine,有限状态机)设计
FPGA设计中,最重要的设计思想就是状态机的设计思想!状态机的本质就是对具有逻辑顺序和时序规律的事件的一种描述方法,它有三个要素:状态.输入.输出:状态也叫做状态变量(比如可以用电机的不同转速作为状态 ...
- paper:synthesizable finite state machine design techniques using the new systemverilog 3.0 enhancements 之 standard verilog FSM conding styles(三段式)
Three always block style with registered outputs(Good style)
- TCP Operational Overview and the TCP Finite State Machine (FSM) http://tcpipguide.com/free/t_TCPOperationalOverviewandtheTCPFiniteStateMachineF.htm
http://tcpipguide.com/free/t_TCPOperationalOverviewandtheTCPFiniteStateMachineF.htm http://tcpipgu ...
随机推荐
- 关于C/C++的一些思考(3)
操作符重载函数(Operator Overload Function)的基本概念: 目的是以与对待内置数据类型相同的方式对待用户自定义类型(程序执行速度会受到影响),限制是不能随意选择函数名和参数个数 ...
- python 1-1模块介绍和使用
1. 什么是模块 1.1 模块就是一系列功能的集合体 1.1.1 模块有三种来源 1.内置的模块 2.第三方的模块 3.自定义模块 1.1.2 模块的格式: 1.使用Python编写的.py文件 2. ...
- [Python3网络爬虫开发实战] 1.5.1-PyMySQL的安装
在Python 3中,如果想要将数据存储到MySQL中,就需要借助PyMySQL来操作,本节中我们介绍一下它的安装方式. 1. 相关链接 GitHub:https://github.com/PyMyS ...
- Android Studio + Genymotion模拟器安装与配置
一.Android studio 下载与安装 https://developer.android.google.cn/studio/index.html 进入谷歌官方链接下载Android studi ...
- 2014-4-5安装python以及基础知识
1.下载安装 从python官网下载python2.7.6 https://www.python.org/download/releases/2.7.6 建议用迅雷下载 会比较快 2.交互式解释器 运 ...
- 关于iphone 微信浏览器编码问题
这个问题最终没有完美的解决,给出的一个解决方法是返回一个html文档.
- 【bzoj3505】[Cqoi2014]数三角形
[bzoj3505][Cqoi2014]数三角形 2014年5月15日3,5230 Description 给定一个nxm的网格,请计算三点都在格点上的三角形共有多少个.下图为4×4的网格上的一个三角 ...
- struts2开发action 的三种方法以及通配符、路径匹配原则、常量
struts2开发action 的三种方法 1.继承ActionSupport public class UserAction extends ActionSupport { // Action中业务 ...
- 【git】Git 提示fatal: remote origin already exists 错误解决办法
今天使用git 添加远程github仓库的时候提示错误:fatal: remote origin already exists. 最后找到解决办法如下: 1.先删除远程 Git 仓库 $ git re ...
- CentOS7下搭建postfix邮箱服务器并实现extmail的web访问
http://blog.51cto.com/zero01/2064693 https://blog.csdn.net/a5nan/article/details/52510887