Random Variables, Expectations and Moments

STA 211 · Lecture 05

Introduction

A random variable assigns a numerical value to each outcome of a random experiment. This permits probability questions to be expressed in algebraic form and provides the basis for probability distributions, expectation and statistical modelling.

Learning Objectives

You should be able to:

• define discrete and continuous random variables;
• use probability mass, density and distribution functions;
• calculate expectations and variances;
• obtain moments and moment generating functions; and
• interpret skewness and kurtosis.

Random Variables and Distributions

A random variable is a measurable function

$$X:\Omega\mathbb{\rightarrow R.}$$

A discrete random variable has a countable support and probability mass function

$$p_{X}(x) = P(X = x),$$

where

$$p_{X}(x) \geq 0,\quad\quad\sum_{x}^{}p_{X}(x) = 1.$$

A continuous random variable has a probability density function $f_{X}(x)$ satisfying

$$f_{X}(x) \geq 0,\quad\quad\int_{- \infty}^{\infty}f_{X}(x)\, dx = 1.$$

For a continuous random variable,

$$P(a < X \leq b) = \int_{a}^{b}f_{X}(x)\, dx,$$

and $P(X = x) = 0$ for every single point $x$.

Cumulative Distribution Function

For any random variable,

$$F_{X}(x) = P(X \leq x).$$

The CDF is non-decreasing, right-continuous and satisfies

$$\lim_{x \rightarrow - \infty}F_{X}(x) = 0,\quad\quad\lim_{x \rightarrow \infty}F_{X}(x) = 1.$$

For a continuous random variable,

$$F_{X}(x) = \int_{- \infty}^{x}f_{X}(u)\, du,$$

and, where differentiable,

$$f_{X}(x) = \frac{d}{dx}F_{X}(x).$$

Expectation

For a discrete random variable,

$$E(X) = \sum_{x}^{}xp_{X}(x).$$

For a continuous random variable,

$$E(X) = \int_{- \infty}^{\infty}xf_{X}(x)\, dx.$$

More generally, for a function $g$,

$$E\left\lbrack g(X) \right\rbrack = \sum_{x}^{}g(x)p_{X}(x)$$

or

$$E\left\lbrack g(X) \right\rbrack = \int_{- \infty}^{\infty}g(x)f_{X}(x)\, dx.$$

This is often called the law of the unconscious statistician.

Variance and Moments

The variance is

$$Var(X) = E\left\lbrack (X - \mu)^{2} \right\rbrack,\quad\quad\mu = E(X).$$

An equivalent computational formula is

$$Var(X) = E\left( X^{2} \right) - \left\lbrack E(X) \right\rbrack^{2}.$$

The standard deviation is

$$\sigma = \sqrt{Var(X)}.$$

The $r$th raw moment is

$$\mu'_{r} = E\left( X^{r} \right),$$

and the $r$th central moment is

$$\mu_{r} = E\left\lbrack (X - \mu)^{r} \right\rbrack.$$

Linearity

For constants $a$ and $b$,

$$E(aX + b) = aE(X) + b,$$

$$Var(aX + b) = a^{2}Var(X).$$

Expectation is linear even when random variables are dependent:

$$E\left( \sum_{i = 1}^{n}X_{i} \right) = \sum_{i = 1}^{n}E\left( X_{i} \right).$$

Moment Generating Function

The moment generating function is

$$M_{X}(t) = E\left( e^{tX} \right),$$

for values of $t$ in an open interval containing zero. If it exists, moments are obtained by differentiation:

$$M_{X}^{(r)}(0) = E\left( X^{r} \right).$$

For independent random variables $X$ and $Y$,

$$M_{X + Y}(t) = M_{X}(t)M_{Y}(t).$$

Skewness and Kurtosis

The coefficient of skewness is

$$\gamma_{1} = \frac{E\left\lbrack (X - \mu)^{3} \right\rbrack}{\sigma^{3}}.$$

Positive skewness indicates a longer right tail; negative skewness indicates a longer left tail.

The kurtosis coefficient is

$$\beta_{2} = \frac{E\left\lbrack (X - \mu)^{4} \right\rbrack}{\sigma^{4}},$$

and excess kurtosis is $\gamma_{2} = \beta_{2} - 3$.

Example 5.1: Discrete distribution

Let $X$ have the distribution

$x$	0	1	2	3
$P(X = x)$	0.1	0.3	0.4	0.2

Then

$$E(X) = 0(0.1) + 1(0.3) + 2(0.4) + 3(0.2) = 1.7,$$

$$E\left( X^{2} \right) = 0 + 1(0.3) + 4(0.4) + 9(0.2) = 3.7,$$

$$Var(X) = 3.7 - (1.7)^{2} = 0.81.$$

Example 5.2: Finding a constant

Suppose $p_{X}(x) = kx$ for $x = 1,2,3,4$. Since probabilities sum to one,

$$k(1 + 2 + 3 + 4) = 1,$$

so $k = 0.1$.

Post-Test

1. State the defining properties of a PMF and a PDF.
2. For the distribution $P(X = 0) = 0.2$, $P(X = 1) = 0.5$, $P(X = 2) = 0.3$, find $E(X)$ and $Var(X)$.
3. If $Y = 4X - 3$, express $E(Y)$ and $Var(Y)$ in terms of $E(X)$ and $Var(X)$.
4. Show that $M_{X}'(0) = E(X)$.
5. Explain why a density value may exceed one without violating probability laws.

Selected answers: 2. $E(X) = 1.1$, $Var(X) = 0.49$; 3. $E(Y) = 4E(X) - 3$, $Var(Y) = 16Var(X)$.

Previous lecture

Course home

Next lecture