Extending Markov to Hidden Markov
Extending Markov to Hidden Markov
a tutorial on hidden markov models, Hidden Markov Models, hidden markov models tutorial, markov chains, markov chains examples,markov chains tutorial, markov models
When we talked about Markov Process and training the Markov Model in previous blog posts, we assumed that we knew the states of the process. That’s often true, and hence Markov model is great tool for predicting and modeling systems where discrete events happen in discrete time-steps. There are some special cases though where we are interested in states underlying the events observed, and events do not map to states in one-to-one or one-to-many fashion as had been requirement so far.
Consider example we have described before – about observing customer’s purchase events at a retail store. If we want to predict next purchase product from sequences of past purchases, Markov model may do good job. But what if you want to remind customer about purchase? Reminder is only helpful if customer is out-of-stock on that product. However, observed purchase event doesn’t necessarily directly correlate to out-of-stock state. Sometimes customer will buy when he is out-of-stock, other times he will be buy because he is at the store, or wants to stock up, or has some discount offer. Sometimes customer may not buy even when he is out-of-stock because he didn’t get time to do so. In this example, true state of interest is out-of-stock but observed state is purchase or no purchase.
By way of another example, we compared customer’s clicks on bank’s website to Markov “memoryless” process. This is good enough if want to improve web layout, but not good enough if want to figure out why customer is visiting website in the first place. Intent behind clicks is the state we are interested in, but all we observe is webpage visits. Maybe she is interested in finding interest rates, or maybe looking for nearest ATM, or maybe wants to read up about new pension plan. Cases like these call for Hidden Markov Model (HMM) where unknown hidden states are of interest but correspond to multiple observed states1.
HMM may be represented as directed graph with hidden edges2.
Apart from Transition Matrix which governs probability of state transition from one hidden state to another,HMM also involves an Emission Matrix which governs probability of observation of observed state from underlying hidden state. Goal of HMM learning is estimation of both these matrices.
Learning HMM
Learning HMM isn’t as simple as learning MM is, but we will give schematic overview in this post.
First, like with Markov Process, we need to know number of observed states, which is obvious from data. However, we also need to make assumption about order of Markov process, and number of hidden states – both are not available. Cross-validation and Akaike Information Criteriondiscussed in previous post come handy. Here, we need to train multiple HMMs with varying number of hidden states and varying order of Markov process and select simplest model which explains training data well.
We can gain understanding of HMM training algorithm by following mental exercises:
Exercise 1 – If we knew Emission and Transition Probabilities, and an observed sequence, can we compute probability of observing that sequence?
Let’s say our observed sequence is S1-S2-S3-... and underlying hidden sequence is H1-H2-H3-... , then probability of observing the given observed sequence is
P(H1)*B(S1|H1)*A(H2|H1)*B(S2|H2)*A(H3|H2)*B(S3|H3)*....
Where B(Si|Hj) is Emission Probability of observing state when hidden state is , and is Transition Probability of transitioning to hidden state from hidden state , assuming Markov Process of order one. However, since we don’t know true underlying sequence we can do so over all combinations of underlying sequences3 and sum over computed probability.
Exercise 2 – If we knew Emission and Transition Probabilities, and an observed sequence, can we make best guess about underlying hidden sequence?
If we compute probabilities of observing given state sequence under all possible combinations of hidden state sequences, one of the hidden sequences will correspond to highest probability of observed sequence. In absence of any other information, that is our best guess of underlying sequence under Maximum Likelihood Estimation method.
Exercise 3 – Given number of observed sequences, and assumption on number of observed and hidden states, can we make best estimate of Emission and Transition Probabilities which will explain our sequences?
This, of course, is HMM training. Here we build on previous steps, and starting with random probabilities, compute joint probabilities of observing all observed sequences (as in, again, Maximum Likelihood Estimation), and search for right set of Emission and Transition probabilities which will maximize this joint probability.
We have intentionally skipped mathematics of the algorithm – called Viterbi Algorithm – for HMM training, but interested reader is encouraged to go to classical paper by Lawrence R Rabiner or slightly simpler variant by Mark Stamp. However, many software packages provide easy implementation of HMM training (depmixS4 in R).
In last three posts, we discussed practical cases when sequence modeling through Markov process may come handy, and provided overview of training Markov Models. Markov models are often easy to train, interpret and implement and can be relevant in many business problems with right design and state identification.
1If hidden states correspond to observed states in one-to-one map, what happens?
2This is not completely true representation for this graph, because underlying process graph is very simple, and making anything more realistic means ugly picture. However, idea is that K number of hidden states map to Nnumber of observed state in many-to-many fashion.
3For K hidden states and L length of observed sequence, we will have KL combinations.
Other Articles by the same author
Other Related Links that you may like
Extending Markov to Hidden Markov的更多相关文章
- [Bayesian] “我是bayesian我怕谁”系列 - Markov and Hidden Markov Models
循序渐进的学习步骤是: Markov Chain --> Hidden Markov Chain --> Kalman Filter --> Particle Filter Mark ...
- [综]隐马尔可夫模型Hidden Markov Model (HMM)
http://www.zhihu.com/question/20962240 Yang Eninala杜克大学 生物化学博士 线性代数 收录于 编辑推荐 •2216 人赞同 ×××××11月22日已更 ...
- PRML读书会第十三章 Sequential Data(Hidden Markov Models,HMM)
主讲人 张巍 (新浪微博: @张巍_ISCAS) 软件所-张巍<zh3f@qq.com> 19:01:27 我们开始吧,十三章是关于序列数据,现实中很多数据是有前后关系的,例如语音或者DN ...
- 隐马尔可夫模型(Hidden Markov Model,HMM)
介绍 崔晓源 翻译 我们通常都习惯寻找一个事物在一段时间里的变化规律.在很多领域我们都希望找到这个规律,比如计算机中的指令顺序,句子中的词顺序和语音中的词顺序等等.一个最适用的例子就是天气的预测. 首 ...
- 理论沉淀:隐马尔可夫模型(Hidden Markov Model, HMM)
理论沉淀:隐马尔可夫模型(Hidden Markov Model, HMM) 参考链接:http://www.zhihu.com/question/20962240 参考链接:http://blog. ...
- Hidden Markov Model
Markov Chain 马尔科夫链(Markov chain)是一个具有马氏性的随机过程,其时间和状态参数都是离散的.马尔科夫链可用于描述系统在状态空间中的各种状态之间的转移情况,其中下一个状态仅依 ...
- NLP —— 图模型(一)隐马尔可夫模型(Hidden Markov model,HMM)
本文简单整理了以下内容: (一)贝叶斯网(Bayesian networks,有向图模型)简单回顾 (二)隐马尔可夫模型(Hidden Markov model,HMM) 写着写着还是写成了很规整的样 ...
- 隐马尔可夫模型(Hidden Markov Model)
隐马尔可夫模型(Hidden Markov Model) 隐马尔可夫模型(Hidden Markov Model, HMM)是一个重要的机器学习模型.直观地说,它可以解决一类这样的问题:有某样事物存在 ...
- Tagging Problems & Hidden Markov Models---NLP学习笔记(原创)
本栏目来源于对Coursera 在线课程 NLP(by Michael Collins)的理解.课程链接为:https://class.coursera.org/nlangp-001 1. Taggi ...
随机推荐
- CentOS7使用winbind加入AD
https://ishm.idv.tw/?p=336 CentOS 7 使用 winbind 加入 AD 需求:已經熟悉 CentOS 6 的 AD 加入方式,CentOS 7 已將 winbind ...
- 金蝶盘点机PDA仓库条码管理:仓库如何盘点
1.1. 仓库盘点 传统的仓库盘点,需要人工手工抄写盘点单,时候再去电脑上一行行的录入盘点单,操作非常耗时费力,往往需要盘点好几天,最终盘点效果还不好,在抄写过程中容易出现错误,从而造成盘点结果不准确 ...
- 记录Appium-desktop踩过的坑could not find devices
最近了解到一个自动化入门的新工具appium-desktop,看网上各种文章,感觉这个工具是很简单的一个入门级工具,下载试用了一下. 官网下载,输入网址appium.io,点击下载. 一路傻瓜式安装, ...
- 1080. Graduate Admission (30)-排序
先对学生们进行排序,并且求出对应排名. 对于每一个学生,按照志愿的顺序: 1.如果学校名额没满,那么便被该学校录取,并且另vis[s][app[i].ranks]=1,表示学校s录取了该排名位置的学生 ...
- 20135337朱荟潼 Linux第一周学习总结——计算机是如何工作的
朱荟潼 + 原创作品转载请注明出处 + <Linux内核分析>MOOC课http://mooc.study.163.com/course/USTC-1000029000 1.冯诺依曼体系结 ...
- Alpha版使用说明
1引言 1 .1编写目的 针对我们发布的alpha版本做出安装和使用说明,使参与内测的人员及用户了解软件的使用方法和相关内容. 1 .2参考资料 <javaWeb程序设计基础><di ...
- C与C++的编程风格区别
c和c++都是在实践中发展起来的语言.实用性极强.c是与UNIX/Linux的发展相辅相成的.而C++是B.S为了摆脱c与硬件以及底层过于紧密的苦恼,而开发的语言.C++可以说就是c语言的超集.任何c ...
- XCODE 6.1.1 配置GLFW
最近在学习opengl的相关知识.第一件事就是配环境(好烦躁).了解了一下os x下的OpenGL开源库,主要有几个:GLUT,freeglut,GLFW等.关于其详细的介绍可以参考opengl网站( ...
- spring mvc自定义注解--访问时验证
作用:在访问controller的方法时,判断用户是否是登陆状态. step1:定义注解 import java.lang.annotation.ElementType; import java.la ...
- Linux命令(十) 在文件或目录之间创建链接 ln
命令简介 ln 命令用于连接文件或目录,如同时指定两个以上的文件或目录,且最后的目的地是一个已经存在的目录,则会把前面指定的所有文件或目录复制到该目录中.若同时指定多个文件或目录,且最后的目的地是一个 ...