Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

Sergey Ioffe and Christian Szegedy submitted Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift on February 11, 2015. The paper proposed normalizing layer inputs as part of the model architecture using statistics computed for each training mini-batch, and reported that the method enabled higher learning rates and reduced sensitivity to initialization.