De Finetti's theorem
One of the differences between Bayesian and frequentist methods in statistical inference is that frequentists often treat observations as independent that Bayesians treat as exchangeable. A Bayesian statistician will often seek the conditional probability distribution of that unobservable quantity given the observable data. The concept of exchangeability (see below) was introduced by de Finetti. De Finetti's theorem explains the mathematical relationship between independence and exchangeability.
An infinite sequence
A random variable X has a "Bernoulli distribution" if P(X = 0 or X = 1) = 1. De Finetti's theorem states that the probability distribution of any infinite exchangeable sequence of Bernoulli random variables is a "mixture" of the probability distributions of independent and identically distributed sequences of Bernoulli random variables. "Mixture", in this sense, means a weighted average, but this need not mean a finite or countably infinite (i.e., discrete) weighted average: it can be an integral rather than a sum.
Here is a concrete example. Suppose p = 2/3 with probability 1/2 and p = 9/10 with probability 1/2. Suppose the conditional distribution of the sequence
Another way of stating the conclusion of de Finetti's theorem is that the Bernoulli random variables are conditionally independent given the tail sigma-field.
The conclusion of the first version of the theorem above makes sense if the sequence of exchangeable Bernoulli random variables is finite, but the theorem is not generally true in that case. It is true if the sequence can be extended to an exchangeable sequence that is infinitely long. The very simplest example of an exchangeable sequence of Bernoulli random variables that cannot be so extended is the one in which X1 = 1 - X2 and X1 is either 0 or 1, each with probability 1/2. This sequence is exchangeable, but cannot be extended to an exchangeable sequence of length 3, let alone an infinitely long one.