Gauss-Markov theorem

This articlenot about Gauss-Markov processes.

---

In statistics,Gauss-Markov theorem states that inlinear modelwhicherrors have expectation zero anduncorrelatedhave equal variances,best linear unbiased estimators ofcoefficients areleast-squares estimators. More generally,best linear unbiased estimatorany linear combination ofcoefficientsits least-squares estimator. The errorsnot assumedbe normally distributed, northey assumedbe independent (but only uncorrelated ---weaker condition), northey assumedbe identically distributed (but only homoscedastic ---weaker condition, defined below).

More explicitly,more concretely, suppose we have

for i = 1, . . . , n, where β₀β₁non-random but unobservable parameters, x_inon-randomobservable, ε_irandom,so Y_irandom. (We set xlower-case because itnot random,Ycapital because itrandom.) The random variables ε_icalled"errors". The Gauss-Markov assumptions state that

(i.e., all errors havesame variance; that"homoscedasticity"), and

for ; that"uncorrelatedness." A linear unbiased estimatorβ₁ islinear combination

in whichcoefficients c_inot allowed depend onearlier coefficients β_i, since thosenot observable, butalloweddepend on x_i, since thoseobservable,whose expected value remains β₁ even ifvaluesβ_i change. (The dependence ofcoefficients onx_itypically nonlinear;estimatorlinearthat whichrandom; thatwhy this"linear" regression.) The mean squared errorsuch an estimator is

i.e.,isexpectation ofsquare ofdifference betweenestimator andparameterbe estimated. (The mean squared erroran estimator coincides withestimator's variance ifestimatorunbiased;biased estimatorsmean squared error issum ofvariance andsquare ofbias.) The best linear unbiased estimator isone withsmallest mean squared error. The "least-squares estimators"β₀β₁ arefunctions ofYs andxs that makesumsquaresresiduals

as small as possible.

The main idea ofproofthatleast-squares estimatorsuncorrelatedevery linear unbiased estimatorzero, i.e.,every linear combination

whose coefficients do not depend uponunobservable β_i but whose expected value remains zero regardlesshowvaluesβ₁β₂ change.

• See also linear regression.

External links

Forbrief history oftheoreman explanationits name seeentry onGauss-Markov theorem in

• Earliest Known UsesSome ofWordsMathematics: G