Joint, Marginal and Conditional Distributions

STA 211 · Lecture 13

Introduction

A joint distribution describes two or more random variables simultaneously. It allows probabilities involving combinations of variables, and it provides marginal distributions, conditional distributions, covariance and correlation.

Learning Objectives

You should be able to:

• construct and interpret joint PMFs and PDFs;
• obtain marginal and conditional distributions;
• test independence by factorisation;
• calculate covariance and correlation; and
• find the mean and variance of sums.

Joint Discrete Distributions

For discrete $X$ and $Y$, the joint PMF is

$$p_{X,Y}(x,y) = P(X = x,Y = y),$$

with

$$p_{X,Y}(x,y) \geq 0,\quad\quad\sum_{x}^{}{\sum_{y}^{}p_{X,Y}}(x,y) = 1.$$

The marginal PMFs are

$$p_{X}(x) = \sum_{y}^{}p_{X,Y}(x,y),$$

$$p_{Y}(y) = \sum_{x}^{}p_{X,Y}(x,y).$$

The conditional PMF is

$$p_{X \mid Y}(x \mid y) = \frac{p_{X,Y}(x,y)}{p_{Y}(y)},\quad\quad p_{Y}(y) > 0.$$

Example 13.1: Joint table

Suppose the joint PMF is

$X \smallsetminus Y$	0	1	2	Row total
0	0.10	0.15	0.05	0.30
1	0.20	0.30	0.20	0.70
Column total	0.30	0.45	0.25	1.00

Then

$$P(X = 1 \mid Y = 2) = \frac{0.20}{0.25} = 0.8.$$

Joint Continuous Distributions

A joint PDF satisfies

$$f_{X,Y}(x,y) \geq 0,$$

$$\int_{- \infty}^{\infty}{\int_{- \infty}^{\infty}f_{X,Y}}(x,y)\, dx\, dy = 1.$$

For a region $R$,

$$P\left( (X,Y) \in R \right) = \iint_{R}^{}f_{X,Y}(x,y)\, dx\, dy.$$

The marginal densities are

$$f_{X}(x) = \int_{- \infty}^{\infty}f_{X,Y}(x,y)\, dy,$$

$$f_{Y}(y) = \int_{- \infty}^{\infty}f_{X,Y}(x,y)\, dx.$$

The conditional density is

$$f_{X \mid Y}(x \mid y) = \frac{f_{X,Y}(x,y)}{f_{Y}(y)}.$$

Independence

Discrete variables are independent if

$$p_{X,Y}(x,y) = p_{X}(x)p_{Y}(y)$$

for every support point. Continuous variables are independent if

$$f_{X,Y}(x,y) = f_{X}(x)f_{Y}(y)$$

throughout the support.

A rectangular support alone does not prove independence; the density must also factorise.

Joint Expectation

For a function $g$,

$$E\left\lbrack g(X,Y) \right\rbrack = \sum_{x}^{}{\sum_{y}^{}g}(x,y)p_{X,Y}(x,y)$$

or

$$E\left\lbrack g(X,Y) \right\rbrack = \int\int g(x,y)f_{X,Y}(x,y)\, dx\, dy.$$

Covariance and Correlation

The covariance is

$$Cov(X,Y) = E\left\lbrack \left( X - E(X) \right)\left( Y - E(Y) \right) \right\rbrack$$

or

$$Cov(X,Y) = E(XY) - E(X)E(Y).$$

The correlation coefficient is

$$\rho_{XY} = \frac{Cov(X,Y)}{\sigma_{X}\sigma_{Y}},$$

where $- 1 \leq \rho_{XY} \leq 1$.

Independence implies zero covariance when the required moments exist, but zero covariance does not generally imply independence.

Sums and Linear Combinations

For constants $a,b,c,d$,

$$Cov(aX + b,cY + d) = acCov(X,Y).$$

The variance of a sum is

$$Var(X + Y) = Var(X) + Var(Y) + 2Cov(X,Y).$$

If $X$ and $Y$ are independent,

$$Var(X + Y) = Var(X) + Var(Y).$$

Post-Test

1. Define a joint PMF and state its normalisation condition.
2. Explain how to obtain a marginal distribution.
3. If $E(X) = 2$, $E(Y) = 3$ and $E(XY) = 8$, find $Cov(X,Y)$.
4. If $Var(X) = 4$, $Var(Y) = 9$ and $Cov(X,Y) = 2$, find $Var(X + Y)$.
5. Does zero correlation always imply independence? Explain.

Selected answers: 3. $2$; 4. $17$; 5. No, except under additional conditions such as joint normality.

Previous lecture

Course home

Next lecture