Central limit theorem

Theorem 1 (Central Limit Theorem).

Let $\set{X_n}_{n \in \N}$ be i.i.d. random variables with $\E X_1 = \mu$ and $\var(X_1) = \sigma^2$. Then $$ \frac{1}{\sqrt{N}} \sum_{n = 1}^N (X_n - \mu) \xrightarrow[N \to \infty]{d} \mathcal N(0, \sigma^2) \,. $$

Here the $\xrightarrow{d}$ notation above means convergence in distribution, which will be defined shortly.

Normal distribution

Definition 2. We say $X$ is a standard normal if $X \sim \mathcal N(0, 1)$.

Proposition 3. If $X \sim \mathcal N(\mu, \sigma^2)$, and $a, b \in \R$ with $a \neq 0$, then $$ aX + b \sim \mathcal N(a\mu+b, a^2\sigma^2) \,. $$ In particular, $(X-\mu)/\sigma$ is a standard normal distribution.

Proposition 4. Suppose $X_1, \dots, X_N$ are independent random variables with $X_n \sim \mathcal N(\mu_n, \sigma_n^2)$. Then $$ \sum_{n=1}^N X_n \sim \mathcal N\paren[\Big]{ \sum_{n=1}^N \mu_n,\, \sum_{n=1}^N \sigma_n^2 } \,. $$

Proof sketch. Follows immediately from this problem.

Remark 5. Suppose $X_n$ are i.i.d. $\mathcal N(\mu,\sigma^2)$ random variables. Then for every $N \in \N$, $$ \frac{1}{\sqrt N}\sum_{n=1}^N(X_n-\mu) \sim \mathcal N(0,\sigma^2) \,. $$

Proposition 6. If $X \sim \mathcal N(\mu, \sigma^2)$, then

Multivariate normal distribution

Definition 7. We say $Z=(Z_1,\dots,Z_d)$ is a standard normal random vector if $Z_1,\dots,Z_d$ are independent standard normal random variables.

Remark 8. The joint density of $Z$ is $$ f_Z(z) = \frac{1}{(2\pi)^{d/2}} \exp\paren[\Big]{-\frac{\abs{z}^2}{2}} \,, \qquad z \in \R^d \,, $$ where $\abs{z} = (z_1^2+\cdots+z_d^2)^{1/2}$ is the standard Euclidean norm.