Bar 04 · Series

Stochastic processes and time series

A stochastic process is a family of random variables indexed by time. This page introduces the idea, examines white noise as the building block, sketches moving-average and autoregressive constructions, and follows the random walk as the simplest non-stationary case.

What a process is

Fix an index set, usually the integers or the whole line, and to each index attach a random variable. The resulting family is a stochastic process. A realised trajectory, recorded for one run of the underlying mechanism, is a time series. The theory lives on two levels at once: the joint law of all the variables, which is fixed and abstract, and the single record, which is what an observer actually sees.

Describing a process in full would require the joint law of every finite collection of its values. In practice one settles for a summary: the mean function, the covariance function, and perhaps a few higher features. For the wide and useful class of Gaussian processes, those first two objects are enough, because a normal law is fixed entirely by its mean and its covariance.

White noise

The simplest process is a sequence of uncorrelated variables of constant mean and constant variance. Such a sequence is called white noise. Its covariance between distinct times is zero, so each value carries no linear information about any other. White noise is the raw material from which richer processes are assembled, and it plays a role in series much like the role of independent draws in the study of sums.

Moving-average processes

A moving-average process forms each value as a weighted combination of the current shock and a fixed number of past shocks. Dependence is injected by the weights. With a single weight the process has a memory only one step long, and its autocorrelation vanishes beyond that step. With more weights the memory stretches further, always over a fixed and finite span.

Autoregressive processes

An autoregressive process forms each value from its own past values plus a fresh shock. With a single lag the process oscillates about its mean when the coefficient is negative and decays smoothly when it is positive. The whole family is stationary precisely when its coefficients satisfy a stability condition, under which the influence of any shock fades geometrically as time passes.

Random walks

A random walk accumulates the shocks of a white-noise sequence by addition rather than by weighted combination. Its variance grows without bound as time advances, and its covariance between two times depends only on the earlier of the two. The walk is therefore not stationary; it wanders, and its long-run level is never pinned down. Random walks are the archetype of a series whose mean structure must be removed by a differencing step before the stationary machinery of this reference can be applied.

Figure 1. A wandering walk that drifts without a fixed level, above a steady noisy series that oscillates about its mean.

The next page examines the condition that separates these two behaviours, stationarity, and studies the autocorrelation function that measures how quickly memory fades.