MAT223: 3_Linear Dependence

Spans, Translated Spans, and Linear Independence/Dependence §

Span defintion §

• Span - the span of a set of vectors $V$ is the set of all linear combinations of vectors in $V$.

  $$\text{span}V=\{\vec{v}:\vec{v}={\alpha }_1{\vec{v}}_1+{\alpha }_2{\vec{v}}_2...+{\alpha }_n{\vec{v}}_n\text{for some}{\alpha }_1,{\alpha }_2,...,{\alpha }_n\in \mathbb{R}\}$$

  • span$\{\}=\{\vec{0}\}$

  Let $\vec{a}=\left[\begin{array}{c}1\\ 2\\ 1\end{array}\right],\vec{b}=\left[\begin{array}{c}0\\ 1\\ 0\end{array}\right],\vec{c}=\left[\begin{array}{c}1\\ 1\\ 2\end{array}\right]$. Show that ${\mathbb{R}}^3=$ span$\{\vec{a},\vec{b},\vec{c}\}$ §

  If the equation of these 3 vectors is always consistent, then any vector in ${\mathbb{R}}^3$ can be obtained as a linear combination of $\vec{a},\vec{b},\vec{c}$.

  $$\vec{x}=\left[\begin{array}{c}x\\ y\\ x\end{array}\right]={\alpha }_1\left[\begin{array}{c}1\\ 2\\ 1\end{array}\right]+{\alpha }_2\left[\begin{array}{c}0\\ 1\\ 0\end{array}\right]+{\alpha }_3\left[\begin{array}{c}1\\ 1\\ 2\end{array}\right]={\alpha }_1\vec{a}+{\alpha }_2\vec{b}+{\alpha }_3\vec{c}$$

  Get the system and solve it

  $$\left[\begin{array}{cccc}{\alpha }_1& 0& {\alpha }_3& x\\ 2{\alpha }_1& {\alpha }_2& {\alpha }_3& y\\ {\alpha }_1& 0& 2{\alpha }_3& z\end{array}\right]\Rightarrow \{\begin{array}{ccc}& 2x-z& ={\alpha }_1\\ & -3x+y+z& ={\alpha }_2\\ & -x+z& ={\alpha }_3\end{array}$$

  It, always have a solution (no matter the values of $x,y,z$). Therefore, span$\{\vec{a},\vec{b},\vec{c}\}={\mathbb{R}}^3$

• Lines and planes that passes through the origin can be expressed as spans ($\vec{p}=\vec{0}$)

  • Lines
    $$\ell =\{\vec{x}\mid \vec{x}=t\vec{d}+\vec{0}\text{for some}t\in \mathbb{R}\}=\text{span}\{\vec{d}\}$$

• Planes

  $$\mathcal{P}=\{\vec{x}\mid \vec{x}=t{\vec{d}}_1+s{\vec{d}}_2+\vec{0}\text{for some}t,s\in \mathbb{R}\}=\text{span}\{{\vec{d}}_1,{\vec{d}}_2\}$$

Sets Addition §

• If $A$ and $B$ are sets of vectors, then the set sum of $A$ and $B$ denoted $A+B$ is

  $$A+B=\{\vec{x}:\vec{x}=\vec{a}+\vec{b}\text{for some}a\in A\text{and}b\in B\}$$

  Let $C=\{\vec{x}\in {\mathbb{R}}^2:\Vert x\Vert \}$, $Y=\{C+\{3{\vec{e}}_1,{\vec{e}}_2\}\}$ §

  $$Y=\{\vec{x}+\vec{v}:\Vert x\Vert =1\text{and}\vec{v}=3{\vec{e}}_1,\text{or}\vec{v}={\vec{e}}_2\}$$

  In another words, $Y$ is the union of two translated copies of $C$

  $$Y=(C+\{{\vec{e}}_1\})\cup (C+\{{\vec{e}}_2\})$$

Span Transformation §

• Every point in ${\ell }_2$ can be obtained by adding $\vec{p}$ to corresponding point in ${\ell }_1$: ${\ell }_2={\ell }_1+\{\vec{p}\}$
  • Do not write ${\ell }_2={\ell }_1+\vec{p}$ because ${\ell }_1$ is a set while $\vec{p}$ is a single vector

Linear Independence & Linear dependence §

• Redundant vector - not needed for the span (can be removed without changing the span)

• When writing an object in vector form, the directional vectors must always be linearly independent

• Geometric defintion

  • Linearly Independent for vectors ${\vec{v}}_1,{\vec{v}}_2...{\vec{v}}_n$
  $$\forall i\in \{1,2,...n\},{\vec{v}}_i\notin \text{span}\{{\vec{v}}_1,{\vec{v}}_2,...,{\vec{v}}_n\}$$

  • Linearly Dependent for vectors ${\vec{v}}_1,{\vec{v}}_2...{\vec{v}}_n$

    $$\exists i\in \{1,2,...n\}\text{s.t.}{\vec{v}}_i\in \text{span}\{{\vec{v}}_1,{\vec{v}}_2,...,{\vec{v}}_{i-1},{\vec{v}}_{i+1},...,{\vec{v}}_n\}$$

  The planes $\mathcal{P}:\vec{x}=t\left[\begin{array}{c}1\\ 2\\ 1\end{array}\right]+s\left[\begin{array}{c}2\\ 2\\ 1\end{array}\right]$ and $\mathcal{Q}:\vec{x}=t\left[\begin{array}{c}3\\ 4\\ 2\end{array}\right]+s\left[\begin{array}{c}2\\ 2\\ 1\end{array}\right]$ are given determine if they are the same plane §

  Let ${\vec{a}}_1=\left[\begin{array}{c}1\\ 2\\ 1\end{array}\right]$ ${\vec{a}}_2=\left[\begin{array}{c}2\\ 2\\ 1\end{array}\right]$ (from $\mathcal{P}$) and ${\vec{b}}_1=b\left[\begin{array}{c}3\\ 4\\ 2\end{array}\right]$ ${\vec{b}}_2=\left[\begin{array}{c}2\\ 2\\ 1\end{array}\right]$ (from $\mathcal{Q}$)

  For $\mathcal{P}=\mathcal{Q}$, all direction vector in $\mathcal{Q}$ must be linear combination of directional vectors in $\mathcal{P}$. In other words, $\{{\vec{a}}_1,{\vec{a}}_2,{\vec{b}}_1\}$ and $\{{\vec{a}}_1,{\vec{a}}_2,{\vec{b}}_2\}$ must all be linearly dependent sets.

  Since ${\vec{a}}_2={\vec{b}}_2$, so only need to verify if $\{{\vec{a}}_1,{\vec{a}}_2,{\vec{b}}_1\}$ is linearly dependent. (If ${\vec{b}}_1$ can be written as a linear combination of ${\vec{a}}_1+{\vec{a}}_2$)

  $$\left[\begin{array}{c}3\\ 4\\ 2\end{array}\right]=x\left[\begin{array}{c}1\\ 2\\ 1\end{array}\right]+y\left[\begin{array}{c}2\\ 2\\ 1\end{array}\right]$$

  ${\vec{b}}_1={\vec{a}}_1+{\vec{a}}_2$, so $\{{\vec{a}}_1,{\vec{a}}_2,{\vec{b}}_1\}$ is linearly dependent. Therefore $\mathcal{P}=\mathcal{Q}$

• Algebraic Defintion

  • Trivial Linear Combination - The linear combination ${\alpha }_1{\vec{v}}_1+{\alpha }_2{\vec{v}}_2...+{\alpha }_n{\vec{v}}_n$ is called a trivial if ${\alpha }_1=...={\alpha }_n=0$, if $\exists {\alpha }_i\ne 0$, then the linear combination is called non-trivial

  • Linearly Independent for vectors ${\vec{v}}_1,{\vec{v}}_2...{\vec{v}}_n$ if all linear combinations are trivial

  • Linearly Dependent - vectors ${\vec{v}}_1,{\vec{v}}_2...{\vec{v}}_n$ if there is a non-trivial linear combination of ${\vec{v}}_1,...{\vec{v}}_n$ that equals the zero vector.

  Let $\vec{u}=\left[\begin{array}{c}1\\ 2\end{array}\right]$, $\vec{v}=\left[\begin{array}{c}2\\ 3\end{array}\right]$, $\vec{w}=\left[\begin{array}{c}4\\ 5\end{array}\right]$. Use the algebraic definition fo linear independence to determine whether $\{\vec{u},\vec{v},\vec{w}\}$ is linearly independent or not §

  Need to determine if there is a non-trivial solution to $x\vec{u}+y\vec{v}+z\vec{w}=\vec{0}$

  This vector equation is equivalent to he system of equations

  $$\{\begin{array}{ccc}& x+2y+4z& =0\\ & 2x+3y+5z& =0\end{array}$$

  By substitution, $y=-3z$ and $x=2z$, let $z=t$ for $t\in \mathbb{R}$

  $$\left[\begin{array}{c}x\\ y\\ z\end{array}\right]=t\left[\begin{array}{c}2\\ -3\\ 1\end{array}\right]$$

  This is a non-trivial solution. Therefore $\vec{u},\vec{v},\vec{w}\}$ is linearly dependent

• Homogeneous - system of linear equations for a vector equation in the variables in the form

  $${\alpha }_1{\vec{v}}_1+{\alpha }_2{\vec{v}}_2...+{\alpha }_n{\vec{v}}_n=\vec{0}$$
  • Vector ${\vec{v}}_1,...{\vec{v}}_n$ are linearly independent iff the homogeneous equation has a unique solution

Proof §

• (Geometric $\Rightarrow$ Algebraic) Assume ${\vec{v}}_1,...{\vec{v}}_n$ are linearly dependent by the geometric definition. This means that for some $i$, ${\vec{v}}_i\in \text{span}\{{\vec{v}}_1,...,{\vec{v}}_{i-1},{\vec{v}}_{i+1},...,{\vec{v}}_n\}$.

  • Fix such $i$. Then, by the definition of span:
  $$\begin{array}{rl}{\vec{v}}_i& ={\alpha }_1{\vec{v}}_1+...+{\alpha }_{i-1}{\vec{v}}_{i-1}+\vec{0}+{\alpha }_{i+1}{\vec{v}}_{i+1}+...+{\alpha }_n{\vec{v}}_n\\ \vec{0}& ={\alpha }_1{\vec{v}}_1+...+{\alpha }_{i-1}{\vec{v}}_{i-1}-{\vec{v}}_i+{\alpha }_{i+1}{\vec{v}}_{i+1}+...+{\alpha }_n{\vec{v}}_n\end{array}$$
  • This must be a non-trivial linear combination because the coefficient of ${\vec{v}}_i$ is $-1\ne 0$, and so it is linearly dependent by the algebraic definition

• (Algebraic $\Rightarrow$ Geometric) Assume ${\vec{v}}_1,...{\vec{v}}_n$ are linearly dependent by the geometric definition. This means that there exist ${\alpha }_1,...{\alpha }_n$ not all zero, so $\vec{0}={\alpha }_1{\vec{v}}_1+...+{\alpha }_n{\vec{v}}_n$

  • Fix $i$ so that ${\alpha }_i\ne 0$, rearrange

    $$-{\alpha }_i{\vec{v}}_i={\alpha }_1{\vec{v}}_1+...+{\alpha }_{i-1}{\vec{v}}_{i-1}+{\alpha }_{i+1}{\vec{v}}_{i+1}+...+{\alpha }_n{\vec{v}}_n$$

  • Since ${\alpha }_i\ne 0$, multiply both sides by $\frac{-1}{{\alpha }_i}$

    $${\vec{v}}_i=\frac{-{\alpha }_1}{{\alpha }_i}{\vec{v}}_1+...+\frac{-{\alpha }_{i-1}}{{\alpha }_i}{\vec{v}}_{i-1}+\frac{-{\alpha }_{i-1}}{{\alpha }_i}{\vec{v}}_{i+1}+...+\frac{-{\alpha }_1}{{\alpha }_n}{\vec{v}}_n$$

  • This shows that ${\vec{v}}_i\in \text{span}\{{\vec{v}}_1,...,{\vec{v}}_{i-1},{\vec{v}}_{i+1},...,{\vec{v}}_n\}$, and so it is linearly dependent by the geometric definition.

Testing if a set of vector is linearly dependent or not §

1. Making them into a matrix, perform RREF, if its trivial, then its linearly independent
2. If the rank of the matrix is equal to the number of columns, then its linearly independent, if the rank is smaller, then its a linearly dependent set.

---

Reduced Row Echelon Form §

• Pivots - learning ones (first non-zero entry)
• Pivot columns - columns with leading ones

A matrix is in RREF if it satisfies these 3 properties §

1. The pivots in every row is 1
2. Above and below each leading one are zeros
3. The leading ones are arranged in an echelon (sarcasm pattern)

Free variables and complete solutions §

• System with an unique solution

  $\{\begin{array}{ccc}& x+2y& =9\\ & 3x+5y& =20\end{array}$. Has a complete unique solution. $\left[\begin{array}{c}x\\ y\end{array}\right]=\left[\begin{array}{c}-5\\ 7\end{array}\right]$

• System with not unique (infinitely many solutions) solutions

  $\{\begin{array}{ccc}& x+2y& =9\\ & 2x+4y& =18\end{array}$ Assigning $t=y$ to the free variable $\{\begin{array}{ccc}& x+2y& =9\\ & y& =t\end{array}$

  Therefore every solution in this system can be written as $\left[\begin{array}{c}x\\ y\end{array}\right]=\left[\begin{array}{c}9\\ 0\end{array}\right]+t\left[\begin{array}{c}-1\\ 1\end{array}\right]$ for $t\in \mathbb{R}$.