Gram-Schmidt process

In linear algebra,Gram-Schmidt processmethodorthogonalizingsetvectorsan inner product space, most commonlyEuclidean space Rⁿ. Orthogonalizationthis context meansfollowing: we startvectors v₁,...,v_k whichlinearly independentwe wantfind mutually orthogonal vectors u₁,...,u_k which generatesame subspace asvectors v₁,...,v_k.

We denoteinner (dot) product oftwo vectors uv by (u . v). The Gram-Schmidt process works as follows:

u₁ = v₁

u₂ = v₂ - [(v₂ . u₁)/(u₁ . u₁)]u₁

u₃ = v₃ - [(v₃ . u₁)/(u₁ . u₁)]u₁ - [(v₃ . u₂)/(u₂ . u₂)]u₂

...

u_k = v_k - [(v_k . u₁)/(u₁ . u₁)]u₁ - [(v_k . u₂)/(u₂ . u₂)]u₂ - ... - [(v_k . u_k-1)/(u_k-1 . u_k-1)]u_k-1

To check that these formulas work, first compute (u₁ . u₂) by substitutingabove formulau₂: you will get zero. Then use thiscompute (u₁ . u₃) again by substitutingformulau₃: you will get zero. The general proof proceeds by mathematical induction.

Geometrically, this method proceeds as follows:compute u_i,projects v_i orthogonally ontosubspace U generated by u₁,...,u_i-1, which issame assubspace generated by v₁,...,v_i-1. u_ithen definedbedifference between v_ithis projection, guaranteedbe orthogonalall ofvectors insubspace U.

If oneinterestedan orthonormal system u₁,...,u_k (i.e.vectorsmutually orthogonalall have norm 1), then one can divide u_i by its norm (u_i . u_i).

When performing orthogonalization oncomputer,Householder transformationusually preferred overGram-Schmidt process since itmore numerically stable, i.e. rounding errors tendhave less serious effects.