似然和对数似然Likelihood & LogLikelihood
One of the most fundamental concepts of modern statistics is that of likelihood. In each of the discrete random variables we have considered thus far, the distribution depends on one or more parameters that are, in most statistical applications, unknown. In the Poisson distribution, the parameter is λ. In the binomial, the parameter of interest is p (since n is typically fixed and known).
Likelihood is a tool for summarizing the data’s evidence about unknown parameters. Let us denote the unknown parameter(s) of a distribution generically by θ. Since the probability distribution depends on θ, we can make this dependence explicit by writing f(x) as f(x ; θ). For example, in the Bernoulli distribution the parameter is θ = π , and the distribution is
f(x;π)=πx(1−π)1−xx=0,1f(x;π)=πx(1−π)1−xx=0,1 (2)
Once a value of X has been observed, we can plug this observed value x into f(x ; π ) and obtain a function of π only. For example, if we observe X = 1, then plugging x = 1 into (2) gives the function π . If we observe X = 0, the function becomes 1 − π .
Whatever function of the parameter we get when we plug the observed data x into f(x ; θ), we call that function thelikelihood function.
We write the likelihood function as L(θ;x)=∏ni=1f(Xi;θ)L(θ;x)=∏i=1nf(Xi;θ) or sometimes just L(θ). Algebraically, the likelihoodL(θ ; x) is just the same as the distribution f(x ; θ), but its meaning is quite different because it is regarded as a function of θ rather than a function of x. Consequently, a graph of the likelihood usually looks very different from a graph of the probability distribution.
For example, suppose that X has a Bernoulli distribution with unknown parameter π . We can graph the probability distribution for any fixed value of π . For example, if π = .5 we get this:
Now suppose that we observe a value of X, say X = 1. Plugging x = 1 into the distribution πx(1−π)1−xπx(1−π)1−x gives the likelihood function L(π ; x) = π , which looks like this:
For discrete random variables, a graph of the probability distribution f(x ; θ) has spikes at specific values of x, whereas a graph of the likelihood L(θ ; x) is a continuous curve (e.g. a line) over the parameter space, the domain of possible values for θ.
L(θ ; x) summarizes the evidence about θ contained in the event X = x. L(θ ; x) is high for values of θ that make X =x more likely, and small for values of θ that make X = x unlikely. In the Bernoulli example, observing X = 1 gives some (albeit weak) evidence that π is nearer to 1 than to 0, so the likelihood for x = 1 rises as p moves from 0 to 1.
For example, if we observe xx from Bin(n,π)Bin(n,π), the likelihood function is
Any multiplicative constant which does not depend on θθ is irrelevant and may be discarded, thus,
Loglikelihood
In most cases, for various reasons, but often computational convenience, we work with the loglikelihood
which is defined up to an arbitrary additive constant.
For example, the binomial loglikelihood is
In many problems of interest, we will derive our loglikelihood from a sample rather than from a single observation. If we observe an independent sample x1,x2,...,xnx1,x2,...,xn from a distribution f(x|θ)f(x|θ), then the overall likelihood is the product of the individual likelihoods:
and the loglikelihood is:
Binomial loglikelihood examples:
Plot of binomial loglikelihood function if n = 5 and we observe x = 0, x = 1, and x = 2 (see the lec1fig.R code on ANGEL on how to produce these figures):
In regular problems, as the total sample size nn grows, the loglikelihood function does two things:
- it becomes more sharply peaked around its maximum, and
- its shape becomes nearly quadratic (i.e. a parabola, if there is a single parameter).
This is important since the tests such as Wald test based on z=statisticSE of statisticz=statisticSE of statistic only works if the logL approximates well to quadratic form. For example, the loglikelihood for a normal-mean problem is exactly quadratic. As the sample size grows, the inference comes to resemble the normal-mean problem. This is true even for discrete data. The extent to which normal-theory approximations work for discrete data does not depend on how closely the distribution of responses resembles a normal curve, but on how closely the loglikelihood resembles a quadratic function.
Transformations may help us to improve the shape of loglikelihood. More on this in Section 1.6 on Alternative Parametrizations. Next we will see how we use the likelihood, that is the corresponding loglikelihood, to estimate the most likely value of the unknown parameter of interest.
from: https://onlinecourses.science.psu.edu/stat504/node/27
似然和对数似然Likelihood & LogLikelihood的更多相关文章
- 负对数似然(negative log-likelihood)
negative log likelihood文章目录negative log likelihood似然函数(likelihood function)OverviewDefinition离散型概率分布 ...
- 挑子学习笔记:对数似然距离(Log-Likelihood Distance)
转载请标明出处:http://www.cnblogs.com/tiaozistudy/p/log-likelihood_distance.html 本文是“挑子”在学习对数似然距离过程中的笔记摘录,文 ...
- 二次代价函数、交叉熵(cross-entropy)、对数似然代价函数(log-likelihood cost)(04-1)
二次代价函数 $C = \frac{1} {2n} \sum_{x_1,...x_n} \|y(x)-a^L(x) \|^2$ 其中,C表示代价函数,x表示样本,y表示实际值,a表示输出值,n表示样本 ...
- 最大似然预计(Maximum likelihood estimation)
一.定义 最大似然预计是一种依据样本来预计模型參数的方法.其思想是,对于已知的样本,如果它服从某种模型,预计模型中未知的參数,使该模型出现这些样本的概率最大.这样就得到了未知參数的预计值. 二 ...
- 朴素贝叶斯-对数似然Python实现-Numpy
<Machine Learning in Action> 为防止连续乘法时每个乘数过小,而导致的下溢出(太多很小的数相乘结果为0,或者不能正确分类) 训练: def trainNB0(tr ...
- 【MLE】最大似然估计Maximum Likelihood Estimation
模型已定,参数未知 已知某个随机样本满足某种概率分布,但是其中具体的参数不清楚,参数估计就是通过若干次试验,观察其结果,利用结果推出参数的大概值.最大似然估计是建立在这样的思想上:已知某个参数能使这个 ...
- 回归——线性回归,Logistic回归,范数,最大似然,梯度,最小二乘……
写在前面:在本篇博客中,旨在对线性回归从新的角度考虑,然后引入解决线性回归中会用到的最大似然近似(Maximum Likelihood Appropriation-MLA) 求解模型中的参数,以及梯度 ...
- EM 最大似然概率估计
转载请注明出处 Leavingseason http://www.cnblogs.com/sylvanas2012/p/5053798.html EM框架是一种求解最大似然概率估计的方法.往往用在存在 ...
- LR为什么用极大似然估计,损失函数为什么是log损失函数(交叉熵)
首先,逻辑回归是一个概率模型,不管x取什么值,最后模型的输出也是固定在(0,1)之间,这样就可以代表x取某个值时y是1的概率 这里边的参数就是θ,我们估计参数的时候常用的就是极大似然估计,为什么呢?可 ...
随机推荐
- ARKit:增强现实技术在美团到餐业务的实践
前言 增强现实(Augmented Reality)是一种在视觉上呈现虚拟物体与现实场景结合的技术.Apple 公司在 2017 年 6 月正式推出了 ARKit,iOS 开发者可以在这个平台上使用简 ...
- JAVAEE——BOS物流项目02:学习计划、动态添加选项卡、ztree、项目底层代码构建
1 学习计划 1.jQuery easyUI中动态添加选项卡 2.jquery ztree插件使用 n 下载ztree n 基于标准json数据构造ztree n 基于简单json数据构造ztree( ...
- [代码审计]eml企业通讯录管理系统v5.0 存在sql注入
0x00 前言 上周五的时候想练练手,随便找了个系统下载下来看看. 然后发现还有VIP版本,但是VIP要钱,看了一下演示站,貌似也没有什么改变,多了个导入功能?没细看. 搜了一下发现这个系统,压根就没 ...
- Visual Studio 2017强制更新方法
Visual Studio 2017强制更新方法 Visual Studio 2017更新时候,用户都是根据消息提示,进行更新.这样做的好处,就是微软可以分批下发升级包,避免集中更新.不过为了早点 ...
- thinkphp5使用redis
1.设置应用配置文件config.php type可以是很多分类File.Redis等等 2.thinkphp5使用redis新建application/index/controller/index. ...
- 下载 ....aar jitpack.io 打不开。
下载 ....aar aar 是 安卓的 打包. 相对与jar 就是可以打包android的资源 比如res下的 . ------ jitpack.io 打不开. ====== 这个是jcenter ...
- [BZOJ3507][CQOI2014]通配符匹配(DP+Hash)
显然f[i][j]表示S匹配到第i个通配符,T匹配到第j个字符,是否可行. 一次一起转移两个通配符之间的所有字符,Hash判断. 稍微有点细节.常数极大卡时过排名倒数,可能是没自然溢出的原因. #in ...
- 关于void main()的误区
很多人甚至市面上的一些书籍,都使用了void main( ) ,其实这是错误的.C/C++ 中从来没有定义过void main( ) .C++ 之父 Bjarne Stroustrup 在他的主页上的 ...
- oracle复杂查询是sql
一.over()分析函数 分组查前几条:select * from test t where (select count(*) from test a where t.type=a.type and ...
- Codeforces Round #360 (Div. 1) D. Dividing Kingdom II 暴力并查集
D. Dividing Kingdom II 题目连接: http://www.codeforces.com/contest/687/problem/D Description Long time a ...