The idea in plain language.
Compute normalization statistics inside each example so behavior is less dependent on batch size.
How it works
For a layer and a single example, the method computes a mean and variance across the layer's inputs, then normalizes those values. Learned gains and biases preserve the model's ability to adjust the representation. Because the statistics belong to each example, the same operation can be used during training and inference. The paper also applies the method to recurrent networks at each time step.
What to keep in mind
Normalization changes optimization behavior; it does not remove the need to evaluate architecture choices, training settings, and task performance.
Source: Layer Normalization. The original manuscript contains the methods, experiments, figures, and references. An arXiv posting date may follow an earlier conference publication. Read the linked record for version history.