Bar 02 · The Gaussian

The Gaussian normal distribution

The normal law is the central object of probability theory. Its density has a single smooth hump, its tails thin faster than any power, and it appears as the limiting law of sums. This page sets out the density, the standardised form, the empirical rule and the central limit theorem.

The density

A random variable is normal with mean μ and variance σ² when its density has the form given by a constant times the exponential of a quadratic expression in (x - μ). The constant is chosen so that the total integral equals one; it involves both the standard deviation and the reciprocal of the square root of two, and it is a direct consequence of the Gaussian integral.

The density is symmetric about its mean, rises to a single peak there, and falls away on both sides. Its inflection points lie exactly one standard deviation from the centre. Two parameters, and only two, fix the whole curve: the mean slides it along the axis and the standard deviation widens or narrows it.

Figure 1. The normal density. The shaded region marks the central band one standard deviation either side of the mean.

Standardisation

Subtracting the mean and dividing by the standard deviation turns any normal variable into the standard normal variable, whose mean is zero and whose variance is one. The transformation is reversible, so every question about the original variable can be answered through the standard curve. Cumulative values for the standard normal are tabulated or obtained by an integral that has no elementary closed form; the error function supplies the same values in a convenient notation.

Tails and the empirical rule

Almost all of the measure lies within a few standard deviations of the centre. About two thirds of the measure lies within one standard deviation, about ninety-five parts in a hundred within two, and about ninety-nine point seven within three. These portions, known as the empirical rule, make the normal law easy to reason about in words.

The tails thin extremely quickly. A value four standard deviations from the mean is very unlikely, and a value six standard deviations away is rarer still, by many orders of magnitude. This thinning is faster than any power law and is the source of both the usefulness and the occasional misuse of the normal model.

The central limit theorem

The central limit theorem states that a sum of many independent variables of finite variance, once centred and scaled, approaches the standard normal law. The remarkable part is the universality: the shapes of the individual variables matter very little, provided that their variances are finite and no single term dominates the sum. This is why the normal curve appears in contexts where no single cause is itself normal.

A classical consequence is the normal approximation to a binomial count, which holds when the count is large and the success probability is neither very small nor very near one.

Historical note

The curve was studied by de Moivre in the eighteenth century and developed by Laplace and later by Gauss, whose name it now carries. Its role as a limiting law was placed on a firm footing in the early twentieth century. The history is instructive because it shows how a single formula can migrate from error analysis into the general theory of sums.

The next page extends the same idea to several variables at once, where the exponent becomes a quadratic form and the variance becomes a matrix of density.