Principle of indifference

The principle of indifference is a rule for assigning epistemic probabilities amongst n mutually exclusive possibilities, where n is a positive integer. It states that if there is no reason to favor a particular possibility, then each possibility is to be assigned a probability of 1/n. Naturally, this rule is meaningless to those who espouse the frequency interpretation of probability, for whom probabilities are relative frequencies rather than degrees of belief in uncertain propositions, conditional upon a state of information.

In layman's terms, the principle of indifference states that, if we have a list of several independent events and have no reason to believe that any are more or less likely to occur than others, then we should assume that each has an equal chance of occurring.

Table of contents

1 Examples

1.1 Coins
1.2 Dice
1.3 Cards

2 Misuse

2.4 Dependent Events
2.5 Ranges

3 History of the principle of indifference

Examples

The textbook examples for the application of the principle of indifference are coins, dice, and cards. Another example of the use of the principle of indifference can be found in the derivation of the partition function.

Coins

A symmetric coin has two sides, arbitrarily labeled "heads" and "tails". We toss the coin; applying the principle of indifference, we assign each of the possible outcomes a probability of 1/2.

To be somewhat pedantic, just the condition of symmetry is not enough to justify the use of the principle of indifference. Several other factors could affect the trajectory and landing of the coin and cause one side to become more likely to appear than the other, such as fluctuations in air pressure and gravity. However, in almost every single set of conditions that a coin flip would be used in on the Earth, these forces act symmetrically on the coin and hence have no significant effect on the outcome.

There is also a third possible outcome: the coin could land on its edge. If the principle of indifference is applied to a coin with three possible outcomes, the probability of each outcome is then 1/3. Common experience differs with the expected probability of 1/3 for an edge landing, which actually has an extremely low probability of occurring on a flat surface due to static equilibrium and the low surface area of the edge of a coin. If the landing surface were not flat, though, the expected probability of an edge landing could be made higher (if the landing surface were a steep funnel with a slit it the bottom only wide enough to admit a coin edgewise, for example).

Dice

A symmetric die has n faces, arbitrarily labeled from 1 to n (n is usually 6, but not always). We toss the die; applying the principle of indifference, we assign each of the possible outcomes a probability of 1/n.

Just as with coins, the condition of symmetry alone is not enough to justify the application of the principle of indifference. However, just as with coins, in almost every case a die would be used all external forces either cancel out or act symmetrically over the coin, exerting no net force, and the unlikely event of an edge landing (even less likely on a die roll than on a coin flip due to the smaller surface area of a die edge) is essentially zero, providing the landing surface is flat. Think of this: if the faces of a (six-sided) die did not have equal chances of being rolled, why would casinos use them for many years without noticing a probability disparity?

Cards

A standard deck contains 52 cards, each given a unique label in an arbitrary fashion, i.e. arbitrarily ordered. We draw a card from the deck; applying the principle of indifference, we assign each of the possible outcomes a probability of 1/52.

This example, more than the others, shows the difficulty of actually applying the principle of indifference in real situations. What we really mean by the phrase "arbitrarily ordered" is simply that we don't have any information that would lead us to favor a particular card. In actual practice, this is rarely the case: a new deck of cards is certainly not in arbitrary order, and neither is a deck immediately after a hand of cards. In practice, we therefore shuffle the cards; this does not destroy the information we have, but instead (hopefully) renders our information practically unusable, although it is still usable in principle. In fact, some expert blackjack players can track aces through the deck; for them, the condition for applying the principle of indifference is not satisfied.

Misuse

The principle of indifference is normally misused by being applied to dependent events. It is also used as part of a faulty argument involving ranges of values.

Dependent Events

The principle of indifference only applies to independent events, not dependent ones. Take the following example, based on one provided by Martin Gardner in the book aha! Gotcha.

What is the probability of life on Europa (or some other moon or planet)?

Well, either there is life or there isn't.

We have no reason to assume one or the other is more or less likely.

Therefore, we apply the principle of indifference, giving the existence of life a probability of 1/2.

There is, however, another way we could reason this:

What is the probability of no single-celled life? There either is or there isn't, so using the principle of indifference, the probability of single-celled life is 1/2.
What is the probability of no multi-celled life? There either is or there isn't, so using the principle of indifference, the probability of multi-celled life is 1/2.
What is the probability of no single-celled or multi-celled life? Using the laws of probability for independent evens, we multiply the probabilities together: 1/2 * 1/2 = 1/4.
What is the probability of either single-celled or multi-celled life? This probability is the opposite of the probability of the probability in the last step: 1 - 1/4 = 3/4.

This probability of 3/4 contradicts the previous probability of 1/2. These contradictory results occur because (assuming evolution is valid) the probability of multi-celled life is not independent of the probability of single-celled life; multi-celled life would develop from single-celled life. The principle of indifference does not apply to dependent events.

Ranges

The principle of indifference is often misapplied to continuous variables of non-uniform distribution. The difficulty is best explained through an example:

Suppose there is a cube hidden in a box. A sign on the box says the cube has a side length between 3 and 5 inches.
There is no reason to favour any side particular side length between 3 inches and 5 inches, so we might believe that it's reasonable to assign a uniform probability distribution to lengths between those limits.
The surface area of the cube must be between 54 square inches and 180 square inches.
There is no reason to favour any side particular surface area between 54 square inches and 150 square inches, so we might believe that it's reasonable to assign a uniform probability distribution to surface areas between those limits.
The volume of the cube must be between 27 cubic inches and 125 cubic inches.
There is no reason to favour any side particular surface area between 27 cubic inches and 125 cubic inches, so we might believe that it's reasonable to assign a uniform probability distribution to volumes between those limits.
If we consider the mean of the distributions to be reasonable point estimates of the variables, then we are in the curious position of estimating that the cube has a side length of 4 inches, a surface area of 102 square inches, and a volume of 76 cubic inches!

In this example, mutually contradictory estimates of the length, surface area, and volume of the cube arise because we have assumed three mutually contradictory distributions for these parameters: a uniform distribution for any one of the variables implies a non-uniform distribution for the other two. In general, for continuous variables, the principle of indifference does not indicate the which variable (e.g. in this case, length, surface area, or volume) is to have a uniform epistemic probability distribution.

History of the principle of indifference

The original writers on probability, primarily Jacob Bernoulli and Pierre Simon Laplace, considered the principle of indifference to be intuitively obvious and did not bother to give it a name. Laplace wrote:

The theory of chance consists in reducing all the events of the same kind to a certain number of cases equally possible, that is to say, to such as we may be equally undecided about in regard to their existence, and in determining the number of cases favorable to the event whose probability is sought. The ratio of this number to that of all the cases possible is the measure of this probability, which is thus simply a fraction whose numerator is the number of favorable cases and whose denominator is the number of all the cases possible.

These earlier writers, Laplace in particular, naively generalized the principle of indifference to the case of continuous parameters, giving the so-called "uniform prior probability distribution", a function which is constant over all real numbers. He used this function to express a complete lack of knowledge as to the value of a parameter.

The principle of insufficient reason was its first name, given to it by later writers, possibly as a play on Leibniz's Principle of Sufficient Reason. These later writers (George Boole, John Venn, and others) objected to the use of the uniform prior for two reasons. The first reason is that the constant function is not normalizable, and thus is not a proper probability distribution. The second reason is its inapplicability to continuous variables, as described above.

The "Principle of insufficient reason" was renamed the "Principle of Indifference" by the economist John Maynard Keynes, who was careful to note that it applies only when there is no knowledge indicating unequal probabilities. It turns out to be a special case of the Principle of maximum entropy, and can be given a deeper logical justification by the Principle of transformation groups.