The Gram-Schmidt Algorithm

https://www.math.hmc.edu/calculus/tutorials/gramschmidt/

The Gram-Schmidt Algorithm In any inner product space, we can choose the basis in which to work. It often greatly simplifies calculations to work in an orthogonal basis. For one thing, if S=v1v2vn is an orthogonal basis for an inner product space V, then it is a simple matter to express any vector wV as a linear combination of the vectors in S:

w=v12wv1v1+v22wv2v2++vn2wvnvn Given an arbitrary basis u1u2un for an n-dimensional inner product space V, the Gram-Schmidt algorithm constructs an orthogonal basis v1v2vn for V:

That is, w has coordinates v12wv1v1v22wv2v2vn2wvnvn relative to the basis S.

Step 1 Let v1=u1.
Step 2 Let v2=u2−projW1u2=u2−v12u2v1v1 where W1 is the space spanned by v1, and projW1u2 is the orthogonal projection of u2 on W1.
Step 3 Let v3=u3−projW2u3=u3−v12u3v1v1−v22u3v2v2 where W2 is the space spanned by v1 and v2.
Step 4 Let v4=u4−projW3u4=u4−v12u4v1v1−v22u4v2v2−v32u4v3v3 where W3 is the space spanned by v1v2 and v3.
     
Continue this process up to vn. The resulting orthogonal set v1v2vn consists of n linearly independent vectors in V and so forms an orthogonal basis for V.





Notes

• To obtain an orthonormal basis for an inner product space V, use the Gram-Schmidt algorithm to construct an orthogonal basis. Then simply normalize each vector in the basis.
  
  
• For Rn with the Eudlidean inner product (dot product), we of course already know of the orthonormal basis (1000)(0100)(001). For more abstract spaces, however, the existence of an orthonormal basis is not obvious. The Gram-Schmidt algorithm is powerful in that it not only guarantees the existence of an orthonormal basis for any inner product space, but actually gives the construction of such a basis.

Example

Let V=R3 with the Euclidean inner product. We will apply the Gram-Schmidt algorithm to orthogonalize the basis (1−11)(101)(112).
Step 1 v1=(1−11).
Step 2 v2===(101)−(1−11)2(101)(1−11)(1−11)(101)−32(1−11)(313231)
Step 3 v3===(112)−(1−11)2(112)(1−11)(1−11)−(313231)2(112)(313231)(313231)(112)−32(1−11)−25(313231)(2−1021)
You can verify that (1−11)(313231)(2−1021)  forms an orthogonal basis for R3. Normalizing the vectors in the orthogonal basis, we obtain the orthonormal basis

333−3336636662−2022 

---

Key Concepts

Given an arbitrary basis u1u2un for an n-dimensional inner product space V, the Gram-Schmidt algorithm constructs an orthogonal basis v1v2vn for V: Step 1 Let v1=u1.
Step 2 Let v2=u2−v12u2v1v1.
Step 3 Let v3=u3−v12u3v1v1−v22u3v2v2.