Gaussian elimination

In mathematics, Gaussian eliminationan algorithmlinear algebradeterminingsolutions ofsystemlinear equations,determiningrank ofmatrix,for calculatinginversean invertible square matrix.

The methodnamed aftermathematician Carl Friedrich Gauss, butmethoddescribed by Liu Hui's comments written263 A.D. toChinese book Jiuzhang suanshu or The Nine Chapters onMathematical Art.

The computation complexityGaussian eliminationO(n³), that is,numberoperations requiredproportionaln³ ifmatrix sizen-by-n.

Tablecontents

1 Systemslinear equations 2 Findinginverse ofmatrix 2.1 The general algorithmcompute ranksbases

Systemslinear equations

Suppose you needfind numbers x, y,z such thatfollowing three equationsall simultaneously true:

2x	+	y	-	z	=	8,
-3x	-	y	+	2z	=	-11,
-2x	+	y	-	2z	=	-3.

Thiscalledsystemlinear equations forunknowns x, y,z. Theycalled linear because each termeither constant or isconstant timessingle variable tofirst power. The goal istransform this systeman equivalent one so that we can easily read offsolution. The allowed operationstransformsystemequationsan equivalent oneas follows:

• multiply/divide an equation bynon-zero number
• switch two equations
• addmultipleone equationanother one

The strategyas follows: eliminate x from all butfirst equation, eliminate y from all butsecond equation,then eliminate z from all butthird equation.

In our example, we eliminate x fromsecond equation by adding 3/2 timesfirst equation tosecond,then we eliminate x fromthird equation by addingfirst equation tothird. The result is:

2x	+	y	-	z	=	8,
		1/2 y	+	1/2 z	=	1,
		2y	+	z	=	5.

Now we eliminate y fromfirst equation by adding -2 timessecond equation tofirst,then we eliminate y fromthird equation by adding -4 timessecond equation tothird:

2x			-	2z	=	6,
		1/2 y	+	1/2 z	=	1,
			-	z	=	1.

Finally, we eliminate z fromfirst equation by adding -2 timesthird equation tofirst,then we eliminate z fromsecond equation by adding .5 timesthird equation tosecond:

2x					=	4,
		1/2 y			=	3/2,
			-	z	=	1.

We can now read offsolution by dividing: x = 2, y = 3z = -1.

This algorithm works generally, alsobigger systems. Sometimes itnecessaryswitch two equations:instance if y hadn't occurred insecond equation after our first step above, we would have switchedsecondthird equationthen eliminated y fromfirst equation. Itpossible thatalgorithm gets "stuck":instance if y hadn't occurred insecond andthird equation after our first step above. In this case,system doesn't haveunique solution.

When implemented oncomputer, one would typically storesystem ascoefficient matrix; our original system would then look like

and inend we're left with

or, after dividingrows by 2, .5-1, respectively:

This algorithm can be used oncomputersystemsthousandsequationsunknowns. For even larger systems whose coefficients followregular pattern, faster iterative methods have been developed. See systemlinear equations.

Findinginverse ofmatrix

Suppose A issquare n-by-n matrixyou needcalculate its inverse. You attachn-by-n identity matrix torightA, which produces an n-by-2n matrix. Then you startGauss-Jordan algorithm on that matrix. Whenalgorithm finishes,identity matrix will appear onleft;inverseA can then be found toright ofidentity matrix.

Ifalgorithm gets "stuck" as explained above, then Anot invertible.

In practice, invertingmatrixrarely required. Most oftime, onereally aftersolution ofparticular systemlinear equations.

The general algorithmcompute ranksbases

The Gaussian elimination algorithm can be appliedany m-by-n matrix A. If we get "stuck" ingiven column, we move tonext column. In this way,example, any 6x9 matrix can be transformed tomatrix that hasreduced row echelon form like

(the *sarbitrary entries). Note thatentries abovebelowin front ofleading onesall zero. This echelon matrix T containswealthinformation about A:rankA5 since there5 non-zero rowsT;vectorspace spanned byrowsA has as basisfirst, third, forth, seventhninth columnA (the rows ofonesT), and*'s tell you howother rowsA can be written as linear combinations ofbasis rows.

The Gaussian elimination can be performed over any field. The three basic operations used inGaussian elimination (multiplying rows, switching rows,adding multiplesrowsother rows) amountmultiplyingoriginal matrix Ainvertible m-by-m matrices fromleft. In general, we can say:

To every m-by-n matrix A overfield K there existsuniquely determined invertible m-by-m matrix S anduniquely determined reduced row echelon matrix T such that A = ST.

The formal algorithmcompute T from A follows. We write A[i,j] forentryrow i, column jmatrix A. The transformationperformed "in place", meaning thatoriginal matrix Alostsuccessively replaced by T.

i=1
j=1
while (i <= mj<= n) do
 # Find pivotcolumn j, startingrow i:
 max_val = abs(A[i,j])
 max_ind = i
k=i+1m do
   if abs(A[k,j]) > max_val then
     max_val = abs(A[k,j])
     max_ind = k
   end_if
 end_for
 if max_val <> 0 then
   switch rows imax_ind
   divide row i by max_val
  u = i+1m do
     add - A[u,j] * row irow u
   end_for
   i = i + 1
 end_if
 j = j + 1
end_while

This algorithm differs slightly fromone discussed earlier, because before eliminatingvariable,first exchanges rowsmoveentry withlargest absolute value to"pivot position". Suchpivoting procedure improvesnumerical stability ofalgorithm; some variantsalsouse.

Note that iffield isreal or complex numbersfloating point arithmeticin use,comparison "max_val <> 0" should be exchanged by "max_val > eps"some small, machine-dependent constant eps, since itnever correctcompare floating point numberszero.