\documentclass{article}
\usepackage[left=2.5cm,right=2.5cm, top=2cm, bottom=2cm]{geometry}
\usepackage{enumerate,amsmath,amssymb,amsthm,graphicx}
\newenvironment{amatrix}[1]{%
\left(\begin{array}{@{}*{#1}{c}|c@{}}
}{%
\end{array}\right)
}
\newif\ifsinglecharacter
\def\lengthone#1{\isSingleCharacterHelper(#1\empty)!!end!!\ifsinglecharacter}
\def\isSingleCharacterHelper(#1#2)!!end!!{\ifx#2\empty\singlecharactertrue\else\singlecharacterfalse\fi}
\def\splitcommainto(#1#2)!!end!!#3{\ifx#1,#3\else #1\fi\ifx#2\empty\else\splitcommainto(#2)!!end!!{#3}\fi}
\def\det(#1){\operatorname{det}(#1)}
\def\trans#1{\lengthone{#1} #1^T\else\left(#1\right)^T\fi}
\def\inv#1{\lengthone{#1} #1^{-1}\else\left(#1\right)^{-1}\fi}
\def\dotprod#1#2{\vec{#1}\cdot\vec{#2}}
\def\vec#1{\mathbf{#1}}
\def\vecsub #1_#2{\mathbf{#1_{#2}}}
\def\coord #1_#2{\left[#1\right]_#2}
\def\norm#1{\left|\!\left|#1\right|\!\right|}
\def\R{\mathbb{R}}
\def\spn{\operatorname{span}}
\def\rank{\operatorname{rank}}
\def\nullity{\operatorname{nullity}}
\def\row{\operatorname{row}}
\def\col{\operatorname{col}}
\def\ker{\operatorname{null}}
\def\set#1{\left\{\,#1\,\right\}}
\def\s#1{\left\{#1\right\}}
\def\st{\text{\huge{.}}}
\def\colvec#1{\begin{pmatrix}\splitcommainto(#1\empty)!!end!!{\\}\end{pmatrix}}
\def\mlist#1_#2{#1_1, \ldots, #1_{#2}}
\def\veclist#1_#2{\mlist\vec{#1}_{#2}}
\def\lincomb#1#2_#3{#1_1\vec{#2_1} + \cdots + #1_{#3}\vec{#2_{#3}}}
\def\sbs{\subseteq}
\def\df#1{\textbf{#1}}
\def\theorem{\par\noindent{\bf \underline{Theorem}} }
\def\lemma{\par\noindent{\bf \underline{Lemma}} }
\def\proof{\par\noindent{\sl Proof.} }
\def\example{\par\noindent{\bf \underline{Example}} }
\def\soln{\par\noindent{\sl Solution.} }
\def\remark{\par\noindent{\bf \underline{Remark}} }
\title{Summary of Day 11}
\author{William Gunther}
\date{June 4, 2014}
\begin{document}
\maketitle
\section{Objectives}
\begin{itemize}
\item Define dimension, and give geometric interpretation.
\item Be able to write a vector in a different coordinate system relative to a different basis.
\item Begin talking about linear transformations.
\end{itemize}
\section{Summary}
\begin{itemize}
\item In linear algebra, we often want to capture invariants. That is, things that stay the same even when you alter them. We know lots of them: for instance, the row space is invariant under row operations (so if you do row operations, the row space does not change).
Along with invariants are particular parameters or characteristics. For instance, we know (sort of-we haven't prove it) that the number of nonzero rows in a matrix's row echelon form does not dependent on the row echelon form you chose. Therefore, this is a parameter that we called rank.
Now we will learn a new parameter called dimension. The definition will not make sense (as with rank) until we prove a certain invariance.
\item The \df{dimension} of a subspace is the size of a basis. We denote the dimension of $V$ by $\dim V$
\theorem If $V$ is a subspace of $\R^n$, then any two bases of $V$ have the same number of vectors.
\proof This is theorem 3.23 in the book. We will prove it in class.
\remark What should the dimension of the trivial subspace be?
\item Intuitively, the dimension is exactly what we want to capture. 1 dimensional subspaces of $\R^n$ are lines since their basis is of size $1$, so it's all vectors that are $c\vec v$. 2 dimensional objects are like planes, as it's vectors like $s\vec v + t\vec u$. 3 dimensional objects are...well, whatever they are called, which is a 3 dimensional hyperplanes, but that's not so important. Just like planes look like a copy of $\R^2$ in $\R^3$, so does a 3 dimensional subspace look like a copy of $\R^3$ in higher dimensions, like $\R^4$, it's just harder to picture.
\item
\theorem The row and column space of a matrix have the same dimension
\proof Let $A$ be the matrix. Clearly $\dim(\row(A)) = \dim(\row(A'))$ where $A'$ is in rref. $\dim(\row(A'))$ is the number of nonzero rows of the matrix which, by the many connections we've made, is the same as the rank of the matrix (i.e. the number of zero rows is the same as the number of leading entries of $A'$).
Now, although $\col(A')\neq\col(A)$ they have a fundamental relationship: $A$ is linearly independent if and only if $A'$ is, and moreover, the dependency is demonstrated in the same way (e.g. if the first column is a multiple of the second in $A'$ then it is so in $A$ as well).
We now make a claim that a moments thought makes clear: in $A'$ all the columns without leading entries can we written as a linear combination of columns with leading entries. Why? Because all the columns with leading entries are elements of the standard basis of $\R^m$ and all the columns not containing leading entries only have nonzero components in places that have a leading entry!
Therefore, the dimension of $A'$ (which as noted is the same as the dimension of $A$) is equal to the number of leading entries, which is the rank, which is the number of nonzero rows, which is the dimension of the row space! Wow, cool. \qed
\item We can then give an equivalent meaning for rank (which actually is not the last one we will see) which is more important when thinking about matrices rather than systems: The \df{rank} of a matrix is the dimension of the row and column spaces.
\item What can you say about the connection between $\rank(A)$ and $\rank(\trans{A})$?
\item With this new version of rank, one would expect that we could say more about The Rank Theorem, and rephrase it in this new matrix-centric way. You'd be right.
With systems there is a tug of war: The more the rank has, the less free variables there will be. What is the tug of war between with regards to the dimension of the column and row spaces?
\item The \df{nullity} of a matrix is the dimension of its null space.
\theorem If $A$ is $m\times n$ then:
\[
\rank(A) + \nullity(A) = n
\]
\proof View $A$ as a homogeneous system of equations. Then if the rank of $A$ is $r$ then there are $n-r$ total free variables by the rank theorem. The solution set of this system is exactly the nullspace, and as there are $n-r$ free variables, it has dimension $n-r$. Therefore, the nullity is $n-r$, and as we knew the rank is $r$, which makes the sum $n$.
\item We can now add some things to the fundamental theorem of invertible matrices:
\theorem (The fundamental theorem of invertible matrices version 2 (thm 3.27)) If $A$ is $n\times n$ then the following are equivalent:
\begin{enumerate}
\item A is invertible.
\item $\rank(A) = n$.
\item $\nullity(A) = 0$
\item The column space of $A$ is a basis for $\R^n$.
\item The row space of $A$ is a basis for $\R^n$.
\end{enumerate}
\proof This is proved in the book with lots of other results filled in too. You should try to work out the proofs by yourself as practice.
\item Now, why is all this basis and dimension stuff important?
\theorem Let $V$ be a $k$ dimensional subspace of $\R^n$ with bases $B=\s{\veclist v_k}$. Then for every $\vec v\in V$ there exists a {\bf unique} way to write $\vec v$ as a linear combination of the elements of $B$.
\proof Clearly there is a way to write $\vec v$ as a linear combination, since $B$ spans the space (that is part of the definition of basis).
If there were two ways then:
\[
\vec v = \lincomb c v_k
\]
\[
\vec v = \lincomb d v_k
\]
Take their difference:
\[
\vec 0 = (c_1 - d_1)\vecsub v_1 + \cdots + (c_k - d_k)\vecsub v_k
\]
As $B$ is linearly independent (as it is a basis) $c_i = d_i$ for all $i$.\qed
\item Now we can formalize something I mentioned long ago. When you take the span of a set of vectors we now see that you get a subspace. The subspace has some basis, which has some size (no matter what basis you choose it will be the same) called the dimension. The basis then creates a coordinate system for the subspace. Just like $\veclist e_n$ is a coordinate system for $\R^n$ so is any basis by the last theorem.
Formally: If $B = \s{\veclist v_n}$ is a basis for an $n$ dimensional subspace of $\R^m$, and $\vec v$ is a element of this subspace then we already know we can write $v$ uniquely as:
\[
\vec v = \lincomb c v_n
\]
So we define the \df{coordinates of $\vec v$ with respect to the basis $B$} to be the $mlist c_n$ that are the coefficents. We can write this as a vector:
\[ \coord {\vec v}_B = \colvec{ c_1, \vdots, c_n} \]
which we call the \df{coordinate vector of $\vec v$ with respect to $B$}.
\example The vectors \[ B = \s{ \colvec{1,1}, \colvec{-1,1} }\] are a basis for $\R^2$. This means that every vector in $\R^2$ can be written uniquely as a linear combination of these two vectors. You already know how to determine how to figure out how: we can set up an augmented matrix:
\[
\begin{amatrix}{2}
1 & -1 & a\\
1 & 1 & b
\end{amatrix}
\]
Then you see that to write $[a,b]$ as a linear combination of these vectors, you take:
\[
\frac{a + b}{2}\colvec{1,1} + \frac{b-a}{2}\colvec{-1,1} = \colvec{a,b}
\]
Thus, for instance, the vector $[5,7]$ can be written as:
\[
\coord{ \colvec{5,7} }_B = \colvec{ 6, 1 }
\]
\item The moral of the story: there is nothing special about the way we write vectors. When we write a vector as $[5,2]$ we are just writing with with respect to the \emph{standard basis} $\vecsub e_1, \vecsub e_2$. We could similarly write this vector with respect to any other basis for $\R^2$.
\item So, matrices are (as I've said repeatedly) a special type of function. Let's unlock exactly what that means. First we need to review what a function is:
A \df{function} $f$ is a mapping from a \df{domain} $A$ to a \df{codomain} $B$. A function has a few guarantees, namely:
\begin{itemize}
\item Every element from $A$ gets mapped to somewhere.
\item Every element gets mapped somewhere in $B$.
\item There is only one thing in $B$ that each element of $A$ gets mapped to.
\end{itemize}
The last condition actually implies the rest, but it's nice to say them all separately.
The \df{range} of the function is the stuff in the codomain that actually gets hit.
\example $f:\R \to \R$ defined by $f(x) = x^2$ has domain $\R$, codomain $\R$, and range $\set{ x\in \R \mid x\geq 0}$
\item $T:\R^n \to \R^m$ is called a \df{linear transformation} if:
\begin{itemize}
\item $T(\vec u + \vec v) = T(\vec u) + T(\vec v)$ for all vectors $\vec u, \vec v\in \R^n$.
\item $T(c\vec u) = cT(\vec u)$ for all scalars $\vec u$ and all vectors $\vec u\in \R^m$.
\end{itemize}
As with subspaces, we can abbreviate this to say:
\begin{itemize}
\item $T(0) = 0$
\item $T(\vec u + c\vec v) = T(\vec u) + cT(\vec v)$ for all $\vec u, \vec v\in \R^n$ and all scalars $c$.
\end{itemize}
\item Matrices are important because$\ldots$
\theorem Let $A$ be a $m\times n$ matrix. Defined $T_A:\R^n\to\R^m$ by:
\[
T_A(\vec x) = A\vec x
\]
this is a linear transformation.
\end{itemize}
\end{document}