翻译 | Improving Distributional Similarity with Lessons Learned from Word Embeddings
翻译 | Improving Distributional Similarity with Lessons Learned from Word Embeddings
叶娜老师说:“读懂论文的最好方法是翻译它”。我认为这是很好的科研训练,更加适合一个陌生领域的探索。因为论文读不懂,我总结无非是因为这个领域不熟悉。如果是自己熟悉的领域,那么读起来肯定会比较顺畅。
原文
摘要
[1] Recent trends suggest that neural-network-inspired word embedding models outperform traditional count-based distributional models on word similarity and analogy detection tasks.
[2] We reveal that much of the performance gains of word embeddings are due to certain system design choices and hyper-parameter optimizations, rather than the embedding algorithms themselves.
[3] Furthermore, we show that these modifications can be transferred to traditional distributional models, yielding similar gains.
[4] In contrast to prior reports, we observe mostly local or insignificant performance differences between the methods, with no global advantage to any single approach over the others.
结论
[1] Recent embedding methods introduce a plethora of design choices beyond network architecture and optimization algorithms.
[2] We reveal that these seemingly minor variations can have a large impact on the success of word representation methods.
[3] By showing how to adapt and tune these hyper-parameters in traditional methods, we allow a proper comparison between representations, and challenge various claims of superiority from the word embedding literature.
(下启第二段)
[4] This study also exposes the need for more controlled-variable experiments, and extending the concept of “variable” from the obvious task, data, and method to the often ignored preprocessing steps and hyper-parameter settings.
[5] We also stress the need for transparent and reproducible experiments, and commend authors such as Mikolov, Pennington, and others for making their code publicly available.
[6] In this spirit, we make our code available as well.
译文
摘要
[1] 最近的趋势表明,神经网络启发的嵌入词模型在词语相似度和词语类比检测任务上优于传统的基于计数的分布式模型。
[2] 我们发现,词嵌入的性能提高很大程度上是由于特定系统设计选择和超参数优化,而不是词嵌入算法本身(带来的性能提升)。
[3] 此外,我们还表明,这些修改可以转移到传统的分布模型,从而产生类似的增益。
[4] 与之前的报告相比,我们观察到方法之间主要存在局部或微小的性能差异,与其他方法相比,没有任何整体优势。
结论
[1] 最近的嵌入方法引入了过剩的网络体系结构和优化算法之外的设计选择。
[2] 我们发现,这些看似微小的变化可能会对单词表达方法的效果产生很大的影响。
[3] 通过展示如何在传统方法中采纳以及调整这些超参数,我进行了在各种表示方法之间的适当比较,并从词嵌入文献中挑战各种主张。
[4] 这项研究还暴露了对更多可控变量实验的需求,并将“变量”的概念从明显的任务、数据和方法扩展到经常被忽略的预处理步骤和超参数设置。
[5] 我们还强调需要透明和可重复的实验,并赞扬 Mikolov,Pennington 等作者公开提供其代码。
[6] 本着这种精神,我们也提供了代码。
感悟
这篇文章是一项对比研究,旨在揭示基于神经网络的词表示学习方法所带来的效果提升,在于超参数的设置,而不是网络结构的改进。
翻译 | Improving Distributional Similarity with Lessons Learned from Word Embeddings的更多相关文章
- Lessons learned developing a practical large scale machine learning system
原文:http://googleresearch.blogspot.jp/2010/04/lessons-learned-developing-practical.html Lessons learn ...
- Lessons learned from manually classifying CIFAR-10
Lessons learned from manually classifying CIFAR-10 Apr 27, 2011 CIFAR-10 Note, this post is from 201 ...
- Lessons Learned from Developing a Data Product
Lessons Learned from Developing a Data Product For an assignment I was asked to develop a visual ‘da ...
- Elasticsearch Mantanence Lessons Learned Today
Today I troubleshooted an Elasticsearch-cluster-down issue. Several lessons were learned: When many ...
- Lessons Learned 1(敏捷项目中的变更影响分析)
问题/现象: 业务信息流转的某些环节,会向相关人员发送通知邮件,邮件中附带有链接,供相关人员进入察看或处理业务.客户要求邮件中的链接,需要进行限制,只有特定人员才能进入处理或察看.总管想了想,应道没问 ...
- Paper Reading - Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge
Link of the Paper: https://arxiv.org/abs/1609.06647 A Correlative Paper: Show and Tell: A Neural Ima ...
- 【机器学习Machine Learning】资料大全
昨天总结了深度学习的资料,今天把机器学习的资料也总结一下(友情提示:有些网站需要"科学上网"^_^) 推荐几本好书: 1.Pattern Recognition and Machi ...
- 机器学习(Machine Learning)&深度学习(Deep Learning)资料【转】
转自:机器学习(Machine Learning)&深度学习(Deep Learning)资料 <Brief History of Machine Learning> 介绍:这是一 ...
- 机器学习(Machine Learning)与深度学习(Deep Learning)资料汇总
<Brief History of Machine Learning> 介绍:这是一篇介绍机器学习历史的文章,介绍很全面,从感知机.神经网络.决策树.SVM.Adaboost到随机森林.D ...
随机推荐
- 安装Nginx到linux服务器(Ubuntu)详解
先去下载一个nginx放到服务器. 然后解压(可参考前面安装tomcat)编译(./configure --prefix=/usr/local/nginx/server/ && mak ...
- python webbrowser模块(浏览器操作)
webbrowser模块提供了一个高级接口来显现基于Web的文档,大部分情况下只需要简单的调用open()办法. webbrowser定义了如下的反常: exception webbrowser.Er ...
- boost中打印python中的变量
p::extract<char const *>(p::str(py_variable))
- Spring 基于Session的创建实例
需求 提供一个网页,根据导入的Excel数据计算结果. 第一版本设计 Controller层 @RestController public class QuoteController { privat ...
- java生成二维码的几个方法
1: 使用SwetakeQRCode在Java项目中生成二维码 http://swetake.com/qr/ 下载地址 或着http://sourceforge.jp/projects/qrcode/ ...
- 2019年新软件发布分享HanGi.IT.AStrutTie.v2017 1CD
Steelray Project Viewer 2019.1.69 1CDIAR Embedded Workbench for Renesas M16C-R8C v3.71.1 1CD Mentor ...
- 尝试启动 ADB 服务器时出错 解决方法
启动CMD→ adb kill-server → adb start-server;
- 基数排序-LSD
这个最低位优先的基数排序,非常适合移植到硬件设备中,所以,我们提供一个C源码 —————————————————————————————————————— #include <stdio.h&g ...
- Xamarin.Forms 未能找到路径“x:\platforms”的一部分
https://stackoverflow.com/questions/45500269/xamarin-android-common-targets-error-could-not-find-a-p ...
- “QObject调用moveToThread()后 该如何释放”及QThread 的启动关闭
1 QThread *thread = new QThread( ); 2 Task *task = new Task(); 3 task->moveToThread(thread); 4 co ...