Previously… (1/2)

The variance of a random variable measures the spread of the variable around its expected value.

[PSDR] Definition 3.31 Let X be a random variable with expected value \(\mu = E[X]\). The variance of \(X\) is defined as

\[Var(X) = E\left[(X−\mu)^2\right]\]

The standard deviation of \(X\) is written \(\sigma(X)\) and is the square root of the variance:

\[\sigma(X) = \sqrt{Var(X)}.\]

[PSDR] Theorem 3.9 \[\begin{align*} Var(X) = & E\left[(X - E[X])^2\right] \\ = & E\left[X^2\right] − E[X]^2 \end{align*}\]

Previously… (2/2)

[PSDR] Theorem 3.10

1. Let \(X\) be a random variable and \(c\) a constant. Then

\[Var(cX) = c^2 Var(X)\] \[\sigma(cX) = |c|\sigma(X)\]

2. Let \(X_1, X_2, \cdots, X_n\) be independent random variables. Then

\[Var(X_1 + X_2 + \cdots + X_n) = Var(X_1) + Var(X_2) + \dots + Var(X_n)\]

Covariance (1/3)

Suppose \(X\) and \(Y\) are two independent random variables. Here, we are showing that \(Var(X + Y) = Var(X) + Var(Y)\).

\[\begin{align*} Var(X + Y) = & E\left[ \left(X + Y\right)^2\right] - \left(E\left[ X + Y \right]\right)^2 \\ = & E\left[ X^2 + 2XY + Y^2 \right] - \left( E[X] + E[Y] \right)^2 \\ = & E[X^2] + 2E[XY] + E[Y^2] - \left((E[X])^2 + E[X]E[Y] + (E[Y])^2 \right) \\ = & E[X^2] - (E[X])^2 + E[Y^2] - (E[Y])^2 + 2\left( E[XY] - E[X]E[Y] \right) \\ = & Var(X) + Var(Y) + 2\left( E[XY] - E[X]E[Y] \right) \\ \end{align*}\]

The term \(E[XY] - E[X]E[Y]\) is the covariance. If \(X\) and \(Y\) are independent, then \(E[XY] - E[X]E[Y] = 0\).

If \(X\) and \(Y\) are independent, then \(Cov(X,Y) = E(XY) - E(X)E(Y) = 0\). The converse is not generally true.

Covariance (2/3)

The covariance between \(X\) and \(Y\) is defined as

\[Cov(X,Y)=E[(X-EX)(Y-EY)]=E[XY]-E[X]E[Y].\]

The covariance between X and Y indicates how the values of \(X\) and \(Y\) move relative to each other.

• If large values of \(X\) tend to happen with large values of \(Y\), then \((X−E[X])(Y−E[Y])\) is positive on average. In this case, the covariance is positive and we say \(X\) and \(Y\) are positively correlated.
• If \(X\) tends to be small when \(Y\) is large, then \((X−E[X])(Y−E[Y])\) is negative on average. In this case, the covariance is negative and we say \(X\) and \(Y\) are negatively correlated.

Covariance (3/3)

The covariance has the following properties:

• \(Cov(X,X) = Var(X)\);
• if \(X\) and \(Y\) are independent then \(Cov(X,Y)=0\);
• \(Cov(X,Y) = Cov(Y,X)\);
• \(Cov(aX,Y) = aCov(X,Y)\);
• \(Cov(X+c,Y) = Cov(X,Y)\);
• \(Cov(X+Y,Z) = Cov(X,Z) + Cov(Y,Z)\);
• more generally, \[Cov(\sum_{i=1}^m a_i X_i, \sum_{j=1}^n b_j Y_j)= \sum_{i=1}^m \sum_{j=1}^n a_i b_j Cov(X_i,Y_j).\]

Example 1

Let \(X\) and \(Y\) be two random variables associated with the throws of two ordinary dice. In both cases the mean is \(E[X] = E[Y] = \frac{7}{2}\) and the variance is \(Var(X) = Var(Y) = \frac{35}{12}\).

We need to compute \(E[XY]\) first. Here, we are using the expectation definition.

\[\begin{align*} E[XY] = & \sum_{x=1}^6 \sum_{y=1}^6 xy P(X=x)P(Y=y) \\ = & \sum_{x=1}^6 \sum_{y=1}^6 xy \frac{1}{6} \frac{1}{6} \\ = & \frac{1}{36} \left(\sum_{x=1}^6 x\right) \left(\sum_{y=1}^6 y\right) \\ = & \frac{1}{36} (21)(21) \\ = & \frac{441}{36} \\ = & \frac{49}{4} \\ \end{align*}\]

Therefore,

\[\begin{align*} Cov(X,Y) = & E[XY]-E[X]E[Y] \\ = & \frac{49}{4} - \frac{7}{2} \frac{7}{2} \\ = & 0 \end{align*}\]

This makes sense since the throws are independent, which means \(E[XY] = E[X]E[Y]\).

Example 2

Suppose that \(X\) and \(Y\) be two random variables where both dice always show the same result. The mean and variance is still \(E[X] = E[Y] = \frac{7}{2}\) and \(Var(X) = Var(Y) = \frac{35}{12}\).

There are only 6 non-zero probabilities, all \(\frac{1}{6}\): So,

\[\begin{align*} E[XY] = & ((1)(1)+(2)(2)+(3)(3)+(4)(4)+(5)(5)+(6)(6)) \frac{1}{6} \\ = & \frac{1+4+9+16+25+36}{6} \\ = & \frac{91}{6} \\ \end{align*}\]

Therefore,

\[\begin{align*} Cov(X,Y) = & E[XY]-E[X]E[Y] \\ = & \frac{91}{6} - \frac{7}{2} \cdot \frac{7}{2} \\ = & \frac{35}{12} \end{align*}\]

Here, \(X\) and \(Y\) are dependent. So, a non-zero covariance makes sense.

Correlation (1/2)

The correlation coefficient, denoted by \(\rho_{XY}\) or \(\rho(X,Y)\), is obtained by normalizing the covariance. In particular, we define the correlation coefficient of two random variables \(X\) and \(Y\) as the covariance of the standardized versions of \(X\) and \(Y\), which can be written as:

\[\rho_{XY} = \rho(X,Y) = \frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}} = \frac{Cov(X,Y)}{\sigma_X \sigma_y}.\]

Correlation (2/2)

Consider two random variables X and Y: - If \(\rho(X,Y)=0\), we say that \(X\) and \(Y\) are uncorrelated. - If \(\rho(X,Y)>0\), we say that \(X\) and \(Y\) are positively correlated. - If \(\rho(X,Y)<0\), we say that \(X\) and \(Y\) are negatively correlated.

Properties of the correlation coefficient: 1. \(−1 \le \rho(X,Y) \le 1\); 2. if \(\rho(X,Y) = 1\), then \(Y = aX + b\), where \(a > 0\); 3. if \(\rho(X,Y) = −1\), then \(Y = aX + b\), where \(a < 0\); 4. \(\rho(aX+b,cY+d) = \rho(X,Y)\) for \(a\), \(c > 0\)

Example 3

Following from example 2.

We are given \(Var(X) = Var(Y) = \frac{35}{12}\) and we computed the covariance to be \(Cov(X,Y) = \frac{35}{12}\).

Therefore,

\[\begin{align*} \rho(X,Y) = & \frac{Cov(X,Y)}{\sqrt{Var(X)Var(Y)}} \\ = & \frac{\frac{35}{12}}{\sqrt{\frac{35}{12} \cdot \frac{35}{12}}} \\ = & 1 \end{align*}\]

This is an absolute positive correlation for rolling two dice where they come up always the same.

Mini-Activity

Mini-Assignment: Variance for DRVs

Back to Tentative Topics Schedule.