【udacity】机器学习-神经网络

Evernote Export

1.神经网络

神经元

细胞的主体称为细胞体，然后有轴突、突触
他们构建的方式是可以调整的
我们会有一些输入的放电信号视为放电频率或输入的强度
X1w1X2w2X3w3
权值w基本相当于神经元对每个输入的增益或敏感性
计算这些输入的和∑i=1kxiwi≥θ
输出的结果如果是1或0，我们称这样的模型为神经网络中的感知器。

2.感知器函数

阶跃函数

在数学中，如果实数域上的某个函数可以用半开区间上的指示函数的有限次线性组合来表示，那么这个函数就是阶跃函数，或者叫赫维赛德函数。换一种不太正式的说法就是，阶跃函数是有限段分段常数函数的组合。
感知器在目前的情况下就是一系列的输入，返回值为0或1

8.感知器训练

1.感知器规则-->感知器使用的是阈值机制
2.梯度下降(德尔塔定律)-->不使用阈值机制
感知器最终只要得到的是一个线性的函数并且最终能够实现分类就能完成任务

9.梯度下降

10.学习规则比较

1.对于感知器来说，线性的变化更适合，因为这是一个线性的规则，具有更好的收敛性
2.对于梯度下降来说，更适合非线性的操作，因为使用的是微积分导数

11.sigmod函数

σ(a)=1+e−a1
a−>−∞σ(a)−>0
a−>+∞σ(a)−>1
sigmod函数的特性就是随着激活函数趋向于负无穷，函数值趋向于0，随着激活函数趋向于正无穷，函数值趋向于1，并且整个过程是平滑曲线
sigmod函数

13.神经网络草图

反向传播，从输入到输出，再回到输入来验证结果

15.限定偏差

限定偏差所用的任意一张数据结构，也可以告诉你将要考虑的假设集合
感知器规则一定是线性的，sigmod函数结合起来可以表示更多问题
我们训练神经网络的时候，一般会指定有限数量的隐藏单元以及有限数量的分层
因此，并不是任何的神经网络真的都可以得到任意函数，任何固定的网络只能得到可以得到的结果
网络的复杂性不仅仅由节点和分层决定，也和权值的大小有关

16.喜好偏差

选择偏置：我们会优先选择节点靠近根节点，有较高信息增益的树，我们优先选择正确的树

17.小结

1.感知器：一个阈值单元或者线性阈值单元
2.感知器网络连接构成网络
3.感知器的学习规则->针对学习规则

%23%23%23%23%201.%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%0A**%E7%A5%9E%E7%BB%8F%E5%85%83**%0A!%5B1f4de4d9c8aa96fd9a3f931d13c28842.png%5D(en-resource%3A%2F%2Fdatabase%2F1236%3A1)%0A%E7%BB%86%E8%83%9E%E7%9A%84%E4%B8%BB%E4%BD%93%E7%A7%B0%E4%B8%BA%E7%BB%86%E8%83%9E%E4%BD%93%EF%BC%8C%E7%84%B6%E5%90%8E%E6%9C%89%E8%BD%B4%E7%AA%81%E3%80%81%E7%AA%81%E8%A7%A6%0A%E4%BB%96%E4%BB%AC%E6%9E%84%E5%BB%BA%E7%9A%84%E6%96%B9%E5%BC%8F%E6%98%AF%E5%8F%AF%E4%BB%A5%E8%B0%83%E6%95%B4%E7%9A%84%0A%E6%88%91%E4%BB%AC%E4%BC%9A%E6%9C%89%E4%B8%80%E4%BA%9B%E8%BE%93%E5%85%A5%E7%9A%84%E6%94%BE%E7%94%B5%E4%BF%A1%E5%8F%B7%E8%A7%86%E4%B8%BA%E6%94%BE%E7%94%B5%E9%A2%91%E7%8E%87%E6%88%96%E8%BE%93%E5%85%A5%E7%9A%84%E5%BC%BA%E5%BA%A6%0A%24X_1w_1%24%24X_2w_2%24%24X_3w_3%24%0A%E6%9D%83%E5%80%BC%24w%24%E5%9F%BA%E6%9C%AC%E7%9B%B8%E5%BD%93%E4%BA%8E%E7%A5%9E%E7%BB%8F%E5%85%83%E5%AF%B9%E6%AF%8F%E4%B8%AA%E8%BE%93%E5%85%A5%E7%9A%84%E5%A2%9E%E7%9B%8A%E6%88%96%E6%95%8F%E6%84%9F%E6%80%A7%0A%E8%AE%A1%E7%AE%97%E8%BF%99%E4%BA%9B%E8%BE%93%E5%85%A5%E7%9A%84%E5%92%8C%24%5Csum%5Ek_%7Bi%3D1%7Dx_iw_i%5Cgeq%5Ctheta%24%0A%E8%BE%93%E5%87%BA%E7%9A%84%E7%BB%93%E6%9E%9C%E5%A6%82%E6%9E%9C%E6%98%AF1%E6%88%960%EF%BC%8C%E6%88%91%E4%BB%AC%E7%A7%B0%E8%BF%99%E6%A0%B7%E7%9A%84%E6%A8%A1%E5%9E%8B%E4%B8%BA%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E4%B8%AD%E7%9A%84%E6%84%9F%E7%9F%A5%E5%99%A8%E3%80%82%0A%0A%23%23%23%23%202.%E6%84%9F%E7%9F%A5%E5%99%A8%E5%87%BD%E6%95%B0%0A**%5B%E9%98%B6%E8%B7%83%E5%87%BD%E6%95%B0%5D(https%3A%2F%2Fzh.wikipedia.org%2Fzh-cn%2F%25E9%2598%25B6%25E8%25B7%2583%25E5%2587%25BD%25E6%2595%25B0)**%0A%3E%E5%9C%A8%E6%95%B0%E5%AD%A6%E4%B8%AD%EF%BC%8C%E5%A6%82%E6%9E%9C%E5%AE%9E%E6%95%B0%E5%9F%9F%E4%B8%8A%E7%9A%84%E6%9F%90%E4%B8%AA%E5%87%BD%E6%95%B0%E5%8F%AF%E4%BB%A5%E7%94%A8%E5%8D%8A%E5%BC%80%E5%8C%BA%E9%97%B4%E4%B8%8A%E7%9A%84%E6%8C%87%E7%A4%BA%E5%87%BD%E6%95%B0%E7%9A%84%E6%9C%89%E9%99%90%E6%AC%A1%E7%BA%BF%E6%80%A7%E7%BB%84%E5%90%88%E6%9D%A5%E8%A1%A8%E7%A4%BA%EF%BC%8C%E9%82%A3%E4%B9%88%E8%BF%99%E4%B8%AA%E5%87%BD%E6%95%B0%E5%B0%B1%E6%98%AF%E9%98%B6%E8%B7%83%E5%87%BD%E6%95%B0%EF%BC%8C%E6%88%96%E8%80%85%E5%8F%AB%E8%B5%AB%E7%BB%B4%E8%B5%9B%E5%BE%B7%E5%87%BD%E6%95%B0%E3%80%82%E6%8D%A2%E4%B8%80%E7%A7%8D%E4%B8%8D%E5%A4%AA%E6%AD%A3%E5%BC%8F%E7%9A%84%E8%AF%B4%E6%B3%95%E5%B0%B1%E6%98%AF%EF%BC%8C%E9%98%B6%E8%B7%83%E5%87%BD%E6%95%B0%E6%98%AF%E6%9C%89%E9%99%90%E6%AE%B5%E5%88%86%E6%AE%B5%E5%B8%B8%E6%95%B0%E5%87%BD%E6%95%B0%E7%9A%84%E7%BB%84%E5%90%88%E3%80%82%0A%E6%84%9F%E7%9F%A5%E5%99%A8%E5%9C%A8%E7%9B%AE%E5%89%8D%E7%9A%84%E6%83%85%E5%86%B5%E4%B8%8B%E5%B0%B1%E6%98%AF%E4%B8%80%E7%B3%BB%E5%88%97%E7%9A%84%E8%BE%93%E5%85%A5%EF%BC%8C%E8%BF%94%E5%9B%9E%E5%80%BC%E4%B8%BA0%E6%88%961%0A%0A%23%23%23%23%208.%E6%84%9F%E7%9F%A5%E5%99%A8%E8%AE%AD%E7%BB%83%0A1.%E6%84%9F%E7%9F%A5%E5%99%A8%E8%A7%84%E5%88%99--%3E%E6%84%9F%E7%9F%A5%E5%99%A8%E4%BD%BF%E7%94%A8%E7%9A%84%E6%98%AF%E9%98%88%E5%80%BC%E6%9C%BA%E5%88%B6%0A2.%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D(%E5%BE%B7%E5%B0%94%E5%A1%94%E5%AE%9A%E5%BE%8B)--%3E%E4%B8%8D%E4%BD%BF%E7%94%A8%E9%98%88%E5%80%BC%E6%9C%BA%E5%88%B6%0A%E6%84%9F%E7%9F%A5%E5%99%A8%E6%9C%80%E7%BB%88%E5%8F%AA%E8%A6%81%E5%BE%97%E5%88%B0%E7%9A%84%E6%98%AF%E4%B8%80%E4%B8%AA%E7%BA%BF%E6%80%A7%E7%9A%84%E5%87%BD%E6%95%B0%E5%B9%B6%E4%B8%94%E6%9C%80%E7%BB%88%E8%83%BD%E5%A4%9F%E5%AE%9E%E7%8E%B0%E5%88%86%E7%B1%BB%E5%B0%B1%E8%83%BD%E5%AE%8C%E6%88%90%E4%BB%BB%E5%8A%A1%0A%0A%23%23%23%23%209.%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%0A%0A%23%23%23%23%2010.%E5%AD%A6%E4%B9%A0%E8%A7%84%E5%88%99%E6%AF%94%E8%BE%83%0A1.%E5%AF%B9%E4%BA%8E%E6%84%9F%E7%9F%A5%E5%99%A8%E6%9D%A5%E8%AF%B4%EF%BC%8C%E7%BA%BF%E6%80%A7%E7%9A%84%E5%8F%98%E5%8C%96%E6%9B%B4%E9%80%82%E5%90%88%EF%BC%8C%E5%9B%A0%E4%B8%BA%E8%BF%99%E6%98%AF%E4%B8%80%E4%B8%AA%E7%BA%BF%E6%80%A7%E7%9A%84%E8%A7%84%E5%88%99%EF%BC%8C%E5%85%B7%E6%9C%89%E6%9B%B4%E5%A5%BD%E7%9A%84%E6%94%B6%E6%95%9B%E6%80%A7%0A2.%E5%AF%B9%E4%BA%8E%E6%A2%AF%E5%BA%A6%E4%B8%8B%E9%99%8D%E6%9D%A5%E8%AF%B4%EF%BC%8C%E6%9B%B4%E9%80%82%E5%90%88%E9%9D%9E%E7%BA%BF%E6%80%A7%E7%9A%84%E6%93%8D%E4%BD%9C%EF%BC%8C%E5%9B%A0%E4%B8%BA%E4%BD%BF%E7%94%A8%E7%9A%84%E6%98%AF%E5%BE%AE%E7%A7%AF%E5%88%86%E5%AF%BC%E6%95%B0%0A%0A%23%23%23%23%2011.sigmod%E5%87%BD%E6%95%B0%0A%24%5Csigma(a)%3D%5Cfrac%7B1%7D%7B1%2Be%5E%7B-a%7D%7D%24%0A%24a-%3E-%5Cinfty%20%5Csigma(a)-%3E0%24%0A%24a-%3E%2B%5Cinfty%20%5Csigma(a)-%3E1%24%0Asigmod%E5%87%BD%E6%95%B0%E7%9A%84%E7%89%B9%E6%80%A7%E5%B0%B1%E6%98%AF%E9%9A%8F%E7%9D%80%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0%E8%B6%8B%E5%90%91%E4%BA%8E%E8%B4%9F%E6%97%A0%E7%A9%B7%EF%BC%8C%E5%87%BD%E6%95%B0%E5%80%BC%E8%B6%8B%E5%90%91%E4%BA%8E0%EF%BC%8C%E9%9A%8F%E7%9D%80%E6%BF%80%E6%B4%BB%E5%87%BD%E6%95%B0%E8%B6%8B%E5%90%91%E4%BA%8E%E6%AD%A3%E6%97%A0%E7%A9%B7%EF%BC%8C%E5%87%BD%E6%95%B0%E5%80%BC%E8%B6%8B%E5%90%91%E4%BA%8E1%EF%BC%8C%E5%B9%B6%E4%B8%94%E6%95%B4%E4%B8%AA%E8%BF%87%E7%A8%8B%E6%98%AF%E5%B9%B3%E6%BB%91%E6%9B%B2%E7%BA%BF%0A**%5Bsigmod%E5%87%BD%E6%95%B0%5D(http%3A%2F%2Fmathworld.wolfram.com%2FSigmoidFunction.html)**%0A%0A%23%23%23%23%2013.%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E8%8D%89%E5%9B%BE%0A%E5%8F%8D%E5%90%91%E4%BC%A0%E6%92%AD%EF%BC%8C%E4%BB%8E%E8%BE%93%E5%85%A5%E5%88%B0%E8%BE%93%E5%87%BA%EF%BC%8C%E5%86%8D%E5%9B%9E%E5%88%B0%E8%BE%93%E5%85%A5%E6%9D%A5%E9%AA%8C%E8%AF%81%E7%BB%93%E6%9E%9C%0A%0A%23%23%23%23%2015.%E9%99%90%E5%AE%9A%E5%81%8F%E5%B7%AE%0A%3E%E9%99%90%E5%AE%9A%E5%81%8F%E5%B7%AE%E6%89%80%E7%94%A8%E7%9A%84%E4%BB%BB%E6%84%8F%E4%B8%80%E5%BC%A0%E6%95%B0%E6%8D%AE%E7%BB%93%E6%9E%84%EF%BC%8C%E4%B9%9F%E5%8F%AF%E4%BB%A5%E5%91%8A%E8%AF%89%E4%BD%A0%E5%B0%86%E8%A6%81%E8%80%83%E8%99%91%E7%9A%84%E5%81%87%E8%AE%BE%E9%9B%86%E5%90%88%0A%E6%84%9F%E7%9F%A5%E5%99%A8%E8%A7%84%E5%88%99%E4%B8%80%E5%AE%9A%E6%98%AF%E7%BA%BF%E6%80%A7%E7%9A%84%EF%BC%8Csigmod%E5%87%BD%E6%95%B0%E7%BB%93%E5%90%88%E8%B5%B7%E6%9D%A5%E5%8F%AF%E4%BB%A5%E8%A1%A8%E7%A4%BA%E6%9B%B4%E5%A4%9A%E9%97%AE%E9%A2%98%0A%E6%88%91%E4%BB%AC%E8%AE%AD%E7%BB%83%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%9A%84%E6%97%B6%E5%80%99%EF%BC%8C%E4%B8%80%E8%88%AC%E4%BC%9A%E6%8C%87%E5%AE%9A%E6%9C%89%E9%99%90%E6%95%B0%E9%87%8F%E7%9A%84%E9%9A%90%E8%97%8F%E5%8D%95%E5%85%83%E4%BB%A5%E5%8F%8A%E6%9C%89%E9%99%90%E6%95%B0%E9%87%8F%E7%9A%84%E5%88%86%E5%B1%82%0A%E5%9B%A0%E6%AD%A4%EF%BC%8C%E5%B9%B6%E4%B8%8D%E6%98%AF%E4%BB%BB%E4%BD%95%E7%9A%84%E7%A5%9E%E7%BB%8F%E7%BD%91%E7%BB%9C%E7%9C%9F%E7%9A%84%E9%83%BD%E5%8F%AF%E4%BB%A5%E5%BE%97%E5%88%B0%E4%BB%BB%E6%84%8F%E5%87%BD%E6%95%B0%EF%BC%8C%E4%BB%BB%E4%BD%95%E5%9B%BA%E5%AE%9A%E7%9A%84%E7%BD%91%E7%BB%9C%E5%8F%AA%E8%83%BD%E5%BE%97%E5%88%B0%E5%8F%AF%E4%BB%A5%E5%BE%97%E5%88%B0%E7%9A%84%E7%BB%93%E6%9E%9C%0A**%E7%BD%91%E7%BB%9C%E7%9A%84%E5%A4%8D%E6%9D%82%E6%80%A7%E4%B8%8D%E4%BB%85%E4%BB%85%E7%94%B1%E8%8A%82%E7%82%B9%E5%92%8C%E5%88%86%E5%B1%82%E5%86%B3%E5%AE%9A%EF%BC%8C%E4%B9%9F%E5%92%8C%E6%9D%83%E5%80%BC%E7%9A%84%E5%A4%A7%E5%B0%8F%E6%9C%89%E5%85%B3**%0A%0A%23%23%23%23%2016.%E5%96%9C%E5%A5%BD%E5%81%8F%E5%B7%AE%0A%E9%80%89%E6%8B%A9%E5%81%8F%E7%BD%AE%EF%BC%9A%E6%88%91%E4%BB%AC%E4%BC%9A%E4%BC%98%E5%85%88%E9%80%89%E6%8B%A9%E8%8A%82%E7%82%B9%E9%9D%A0%E8%BF%91%E6%A0%B9%E8%8A%82%E7%82%B9%EF%BC%8C%E6%9C%89%E8%BE%83%E9%AB%98%E4%BF%A1%E6%81%AF%E5%A2%9E%E7%9B%8A%E7%9A%84%E6%A0%91%EF%BC%8C%E6%88%91%E4%BB%AC%E4%BC%98%E5%85%88%E9%80%89%E6%8B%A9%E6%AD%A3%E7%A1%AE%E7%9A%84%E6%A0%91%0A%0A%23%23%23%23%2017.%E5%B0%8F%E7%BB%93%0A1.%E6%84%9F%E7%9F%A5%E5%99%A8%EF%BC%9A%E4%B8%80%E4%B8%AA%E9%98%88%E5%80%BC%E5%8D%95%E5%85%83%E6%88%96%E8%80%85%E7%BA%BF%E6%80%A7%E9%98%88%E5%80%BC%E5%8D%95%E5%85%83%0A2.%E6%84%9F%E7%9F%A5%E5%99%A8%E7%BD%91%E7%BB%9C%E8%BF%9E%E6%8E%A5%E6%9E%84%E6%88%90%E7%BD%91%E7%BB%9C%0A3.%E6%84%9F%E7%9F%A5%E5%99%A8%E7%9A%84%E5%AD%A6%E4%B9%A0%E8%A7%84%E5%88%99-%3E%E9%92%88%E5%AF%B9%E5%AD%A6%E4%B9%A0%E8%A7%84%E5%88%99