1. 基础模型 A. Sequence to sequence model:机器翻译.语音识别.(1. Sutskever et. al., 2014. Sequence to sequence learning with neural networks. 2. Cho et. al., 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation.) B…
1. Word representation One-hot representation的缺点:把每个单词独立对待,导致对相关词的泛化能力不强.比如训练出“I want a glass of orange juice”后,面对“I want a glass of apple ”,由于任何两个不同单词的one-hot vector的内积都为0,算法不知道orange和apple是一类词,所以没办法泛化出在apple后面填“juice”. Featurized represent…