tensorflow2.0学习笔记第二章第四节

2.4损失函数
损失函数（loss）:预测值（y）与已知答案（y_）的差距
    nn优化目标：loss最小->-mse
                        -自定义
                        -ce(cross entropy)
均方误差mse:MSE(y_,y)=E^n~i=1(y-y_)^2/n
    loss_mse = tf.reduce_mean(tf.square(y_-y))

import tensorflow as tf
import numpy as np

SEED = 23455

rdm = np.random.RandomState(seed=SEED)
x = rdm.rand(32,2)
y_ = [[x1 + x2 + (rdm.rand()/10.0 - 0.05)] for (x1,x2) in x] # 生成【0，1】/10-0.05的噪声
x = tf.cast(x,dtype = tf.float32)

w1 = tf.Variable(tf.random.normal([2,1],stddev=1, seed = 1)) # 创建一个2行一列的参数矩阵

epoch = 15000
lr = 0.002

for epoch in range(epoch):
    with tf.GradientTape() as tape:
        y = tf.matmul(x,w1)
        loss_mse = tf.reduce_mean(tf.square(y_-y))

    grads = tape.gradient(loss_mse,w1) # loss_mse对w1求导
    w1.assign_sub(lr*grads) # 在原本w1上减去lr(学习率)*求导结果

    if epoch % 500 == 0:
        print('After %d training steps,w1 is'%(epoch))
        print(w1.numpy(),"\n")
print("Final w1 is:",w1.numpy())

结果：

After 0 training steps,w1 is
[[-0.8096241]
[ 1.4855157]]

After 500 training steps,w1 is
[[-0.21934733]
[ 1.6984866 ]]

After 1000 training steps,w1 is
[[0.0893971]
[1.673225 ]]

After 1500 training steps,w1 is
[[0.28368822]
[1.5853055 ]]

........

........

After 14000 training steps,w1 is
[[0.9993659]
[0.999166 ]]

After 14500 training steps,w1 is
[[1.0002553 ]
[0.99838644]]

Final w1 is: [[1.0009792]
[0.9977485]]

自定义损失函数
    如预测商品销量，预测多了，损失成本，预测少了损失利润
    若利润！=成本，则mse产生的loss无法利益最大化
    自定义损失函数 loss(y_-y)=Ef(y_-y)

    f(y_-y)={profit*(y_-y) ,y<y_ 预测少了，损失利润
            {cost*(y_-y)   ,y>y_ 预测多了，损失成本

    写出函数：
            loss_zdy = tf.reduce_sum(tf.where(tf.greater(y_,y),(profit*(y_-y),cost*(y-y_) )))

    假设商品成本1元，利润99元，则预测后的参数偏大，预测销量较高，反之成本为99利润为1则参数小，销售预测较小

import tensorflow as tf
import numpy as np

profit = 1
cost = 99
SEED = 23455
rdm = np.random.RandomState(seed=SEED)
x = rdm.rand(32,2)
x = tf.cast(x,tf.float32)
y_ = [[x1+x2 + rdm.rand()/10.0-0.05] for x1,x2 in x]
w1 = tf.Variable(tf.random.normal([2,1],stddev=1,seed=1))

epoch = 10000
lr = 0.002

for epoch in range(epoch):
    with tf.GradientTape() as tape:
        y = tf.matmul(x,w1)
        loss_zdy = tf.reduce_sum(tf.where(tf.greater(y_,y),(y_-y)*profit,(y-y_)*cost))

        grads = tape.gradient(loss_zdy,w1)
        w1.assign_sub(lr*grads)

    if epoch % 500 == 0:
        print("after %d epoch w1 is:"%epoch)
        print(w1.numpy(),'\n')
        print('--------------')
print('final w1 is',w1.numpy())

# 当成本=1，利润=99模型的两个参数[[1.1231122][1.0713713]] 均大于1模型在往销量多的预测
# 当成本=99，利润=1模型的两个参数[[0.95219666][0.909771  ]] 均小于1模型在往销量少的预测

交叉熵损失函数CE（cross entropy）,表示两个概率分布之间的距离
    H(y_,y)= -Ey_*lny
    如：二分类中标准答案y_=(1,0),预测y1=(0.6,0.4),y2=(0.8,0.2)
    哪个更接近标准答案？
    H1((1,0),(0.6,0.4))=-(1*ln0.6 + 0*ln0.4) =0.511
    H2((1,0),(0.8,0.2))=0.223
    因为h1>H2,所以y2预测更准
    tf中交叉熵的计算公式：
    tf.losses.categorical_crossentropy(y_,y)

import tensorflow as tf
loss_ce1 = tf.losses.categorical_crossentropy([1,0],[0.6,0.4])
loss_ce2 = tf.losses.categorical_crossentropy([1,0],[0.8,0.2])
print("loss_ce1",loss_ce1)
print("loss_ce2",loss_ce2)
#loss_ce1 tf.Tensor(0.5108256, shape=(), dtype=float32)
#loss_ce2 tf.Tensor(0.22314353, shape=(), dtype=float32)
# 结果loss_ce2数值更小更接近

softmax与交叉熵结合
    输出先过softmax,再计算y_和y的交叉损失函数
    tf.nn.softmax_cross_entroy_with_logits(y_,y)