Distributions of Functions of Random Variables

STA 211 · Lecture 14

Introduction

Transformations convert known random variables into new ones. The main methods are direct use of the CDF, change of variables with a derivative or Jacobian, and convolution for sums. These methods also lead naturally to convergence concepts and the Central Limit Theorem.

Learning Objectives

You should be able to:

• find the distribution of a one-to-one transformation;
• use the CDF method for non-monotone transformations;
• apply a Jacobian for bivariate transformations;
• derive distributions of sums by convolution; and
• state the Law of Large Numbers and Central Limit Theorem.

Univariate Transformations

Let $Y = g(X)$, where $g$ is one-to-one and differentiable. If $x = g^{- 1}(y)$, then

$$f_{Y}(y) = f_{X}\left( g^{- 1}(y) \right)\left| \frac{d}{dy}g^{- 1}(y) \right|.$$

The absolute derivative adjusts for stretching or compression of the measurement scale.

Example 14.1: Linear transformation

If $Y = a + bX$, $b \neq 0$, then

$$f_{Y}(y) = \frac{1}{|b|}f_{X}\left( \frac{y - a}{b} \right).$$

CDF method

For any transformation,

$$F_{Y}(y) = P\left( g(X) \leq y \right).$$

After obtaining $F_{Y}$, differentiate if a density is required.

Example 14.2: Squaring a symmetric variable

Let $X \sim U( - 1,1)$ and $Y = X^{2}$. For $0 < y < 1$,

$$F_{Y}(y) = P\left( X^{2} \leq y \right) = P\left( - \sqrt{y} \leq X \leq \sqrt{y} \right) = \sqrt{y}.$$

Therefore,

$$f_{Y}(y) = \frac{1}{2\sqrt{y}},\quad\quad 0 < y < 1.$$

The transformation is not one-to-one, so both roots must be included.

Bivariate Transformations and Jacobians

Let

$$U = g_{1}(X,Y),\quad\quad V = g_{2}(X,Y),$$

with inverse

$$X = h_{1}(U,V),\quad\quad Y = h_{2}(U,V).$$

Then

$$f_{U,V}(u,v) = f_{X,Y}\left( h_{1}(u,v),h_{2}(u,v) \right)|J|,$$

where

$$J = \frac{\partial(x,y)}{\partial(u,v)} = \begin{vmatrix} \frac{\partial x}{\partial u} & \frac{\partial x}{\partial v} \\ \frac{\partial y}{\partial u} & \frac{\partial y}{\partial v} \end{vmatrix}.$$

The transformed support must be derived explicitly; a correct Jacobian with an incorrect support gives an incorrect density.

Convolution for Sums

If $X$ and $Y$ are independent continuous random variables and $S = X + Y$, then

$$f_{S}(s) = \int_{- \infty}^{\infty}f_{X}(x)f_{Y}(s - x)\, dx.$$

For discrete variables,

$$P(S = s) = \sum_{x}^{}P(X = x)P(Y = s - x).$$

Example 14.3: Sum of two uniform variables

If $X,Y \sim U(0,1)$ independently, then $S = X + Y$ has density

$$f_{S}(s) = \left\{ \begin{matrix} s, & 0 < s < 1, \\ 2 - s, & 1 \leq s < 2, \\ 0, & \text{otherwise}. \end{matrix} \right.\$$

Convergence Concepts

A sequence $X_{n}$ converges in probability to $X$ if, for every $\varepsilon > 0$,

$$P\left( \left| X_{n} - X \right| > \varepsilon \right) \rightarrow 0.$$

It converges in distribution if

$$F_{X_{n}}(x) \rightarrow F_{X}(x)$$

at every continuity point of $F_{X}$.

Convergence in probability implies convergence in distribution, but the converse is not generally true.

Weak Law of Large Numbers

If $X_{1},X_{2},\ldots$ are independent and identically distributed with mean $\mu$ and finite variance, then the sample mean

$${\bar{X}}_{n} = \frac{1}{n}\sum_{i = 1}^{n}X_{i}$$

satisfies

$${\bar{X}}_{n}\overset{P}{\rightarrow}\mu.$$

Thus, averages stabilise near the population mean as the sample size grows.

Central Limit Theorem

If $X_{1},X_{2},\ldots$ are iid with mean $\mu$ and finite positive variance $\sigma^{2}$, then

$$\frac{\sum_{i = 1}^{n}X_{i} - n\mu}{\sigma\sqrt{n}}\overset{d}{\rightarrow}N(0,1).$$

Equivalently, for large $n$,

$${\bar{X}}_{n} \approx N\left( \mu,\frac{\sigma^{2}}{n} \right).$$

The CLT concerns the distribution of a sum or average, not necessarily the distribution of the individual observations.

Example 14.4

A population has mean 80 and standard deviation 12. For samples of size 36,

$$\bar{X} \approx N\left( 80,\frac{12^{2}}{36} \right) = N(80,4).$$

Thus,

$$P\left( 78 < \bar{X} < 83 \right) \approx P( - 1 < Z < 1.5) \approx 0.7745.$$

Post-Test

1. State the one-to-one transformation formula.
2. Why is an absolute value used in the derivative or Jacobian?
3. State the convolution formula for a sum of independent continuous variables.
4. Distinguish convergence in probability from convergence in distribution.
5. State the Central Limit Theorem and explain what it approximates.

Previous lecture

Course home

Next lecture