Network Initialization: Fan-in and Fan-out

https://github.com/pytorch/pytorch/blob/master/torch/nn/init.py

@weak_script
def _calculate_fan_in_and_fan_out(tensor):
    dimensions = tensor.dim()
    if dimensions < 2:
        raise ValueError("Fan in and fan out can not be computed for tensor with fewer than 2 dimensions")

    if dimensions == 2:  # Linear
        fan_in = tensor.size(1)
        fan_out = tensor.size(0)
    else:
        num_input_fmaps = tensor.size(1)
        num_output_fmaps = tensor.size(0)
        receptive_field_size = 1
        if tensor.dim() > 2:
            receptive_field_size = tensor[0][0].numel()
        fan_in = num_input_fmaps * receptive_field_size
        fan_out = num_output_fmaps * receptive_field_size

    return fan_in, fan_out

Fan-in: 影响到该Neuron的输入数目。

Fan-out: 该Neron所能影响的输出的数目。

https://stackoverflow.com/questions/42670274/how-to-calculate-fan-in-and-fan-out-in-xavier-initialization-for-neural-networks

fan_in = n_feature_maps_in * receptive_field_height * receptive_field_width
fan_out = n_feature_maps_out * receptive_field_height * receptive_field_width / max_pool_area

https://blog.csdn.net/wumo1556/article/details/86583946

https://blog.csdn.net/dss_dssssd/article/details/83992701