Chapter 7: Joint Distributions

MATH/STAT 394: Probability I

Arman Jahangiri

University of Washington Department of Mathematics

Summer 2026

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026

1

Learning Goals

By the end of this lecture, students should be able to:

▶ deﬁne joint, marginal, and conditional distributions;

▶ understand independence of random variables;

▶ apply the two-dimensional LOTUS formula;

▶ deﬁne covariance and correlation;

▶ distinguish independence from zero correlation;

▶ deﬁne pairwise and mutual independence;

▶ deﬁne the Multinomial distribution;

▶ understand the Multivariate Normal distribution.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026

2

Joint Distributions

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026

3

Why Joint Distributions?

Previously, we studied one random variable at a time.

But many probabilistic questions involve several random variables simultaneously.

Examples:

▶ height and weight;

▶ temperature and rainfall;

▶ several coin tosses;

▶ sums of random variables;

▶ sample means.

Main idea

Joint distributions describe how multiple random variables behave together.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026

4

Joint PMF

Deﬁnition

For discrete random variables X, Y , the joint PMF is

p_X,Y(x, y) = P(X = x, Y = y).

Properties:

X X

p_X,Y(x, y) ≥ 0, and

p_X,Y(x, y) = 1.

x

y

Similarly, for n discrete random variables X₁, . . . , X_n, the joint PMF is

p_X_n(x₁, . . . , x_n) = P(X₁= x₁, . . . , X_n= x_n)

,...,X

1

Note {X ≤ x, Y ≤ y} = {X ≤ x} ∩ {Y ≤ y}

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026

5

Marginal PMFs

The marginal PMFs are obtained by summing out the other variable.

X

p_X(x) =

p_X,Y(x, y)

y

and

X

p_Y(y) =

p_X,Y(x, y).

x

Interpretation

Marginals describe each variable individually.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026

6

Example: Two Fair Dice and Marginals

Let X be the outcome of the ﬁrst die and Y the outcome of the second die.

1

p_X,Y(x, y) = P(X = x, Y = y) =

,

x, y ∈ {1, 2, 3, 4, 5, 6}.

36

p_X,Y(x, y) = P(X = x, Y = y) y = 1 y = 2 y = 3 y = 4 y = 5 y = 6 p_X(x) = P(X = x)

1

36

1

36

1

36

1

36

1

36

1

36

6

1

36

1

36

1

36

1

36

1

36

1

36

6

1

36

1

36

1

36

1

36

1

36

1

36

6

1

36

1

36

1

36

1

36

1

36

1

36

6

1

36

1

36

1

36

1

36

1

36

1

36

6

1

36

1

36

1

36

1

36

1

36

1

36

6

36

6

36

6

36

6

36

6

x = 1

x = 2

x = 3

x = 4

x = 5

36

6

36

x = 6

p_Y(y) = P(Y = y)

36

For example,

6

X

1

6

1

6

1

6

p_X(3) =

p_X,Y(3, y) =

+ · · · +

=

,

and p_Y(4) =

p_X,Y(x, 4) =

=

.

36

6

36

y=1

x=1

The marginal PMFs appear on the ”margins” of the joint PMF table.

Arman Jahangiri Chapter 7: Joint Distributions

Summer 2026

7

Proof of Marginal PMFs

We prove the formula for p_X(x).

Fix a value x in the support of X. Then

p_X(x) = P(X = x).

The event {X = x} can be written as the union of all events

{X = x, Y = y}

over all possible values y of Y :

[

{X = x} =

{X = x, Y = y}.

y

Reason:

[

ꢀ

ꢁ

{X = x} = {X = x} ∩ S = {X = x} ∩ ( {Y = y}) =

{X = x} ∩ {Y = y} =

{X = x, Y = y}

y

Thus, applying the probability function to the above equation yields the result. Indeed, these events are pairwise

disjoint, because Y cannot take two diﬀerent values at the same time. Therefore, by the third axiom of probability

(countable additivity),

X

Chapter 7: Joint Distributions

P(X = x) =

P(X = x, Y = y).

y

Hence,

Arman Jahangiri

Summer 2026

8

Conditional PMFs

Deﬁnition

If

P(X = x) > 0,

then

p_X,Y(x, y)

p_{Y |X}(y|x) = P(Y = y|X = x) =

.

p_X(x)

Conditional distributions describe one variable given information about another.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026

9

Joint PDFs

Deﬁnition

For continuous random variables X, Y , the joint PDF is a function

f_X,Y(x, y)

such that

ZZ

P((X, Y ) ∈ A) =

f_X,Y(x, y) dx dy.

A

Properties:

ZZ

f_X,Y(x, y) ≥ 0,

f_X,Y(x, y) dx dy = 1.

2

R

Similarly, for n continuous random variables

X₁, . . . , X_n,

the joint PDF is a function

f_X

_n(x₁, . . . , x_n)

,...,X

1

such that

Z

ꢀ

ꢁ

P (X₁, . . . , X_n) ∈ A =

· · ·

Chapter 7: Joint Distributions

f_X

_n(x₁, . . . , x_n) dx₁· · · dx_n.

,...,X

1

A

Arman Jahangiri

Summer 2026 10

Marginal PDFs

Marginal densities are obtained by integrating out the other variable.

Z

∞

f_X(x) = _−∞f_X,Y(x, y) dy

and

Z

∞

f_Y(y) = _−∞f_X,Y(x, y) dx.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 11

Conditional PDFs

Deﬁnition

If

f_X(x) > 0,

f_X,Y(x, y)

then

f_{Y |X}(y|x) =

.

f_X(x)

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 12

Example: Uniform on the Unit Square

Suppose

Then:

(X, Y ) ∼ Unif([0, 1]²).

f_X,Y(x, y) = 1,

0 < x < 1, 0 < y < 1.

Marginals:

Thus:

f_X(x) = 1,

f_Y(y) = 1.

f_X,Y(x, y) = f_X(x)f_Y(y).

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 13

Example: Uniform on the Unit Disk

Suppose (X, Y ) is uniformly distributed on the unit disk

D = {(x, y) : x²+ y²≤ 1}.

Since the area of the unit disk is π, the joint density is



1

π

x²+ y²≤ 1,



,

f_X,Y(x, y) =



0,

otherwise.

Indeed,

ZZ

1

f_X,Y(x, y) dx dy =

dx dy =

Area(D) = 1.

π

2

R

D

For a ﬁxed value of X = x, the possible values of Y satisfy

p

−

1 − x²≤ Y ≤ 1 − x².

Important

Knowing X restricts the possible values of Y , so X and Y are dependent.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 14

Independence

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 15

Independence

Deﬁnition

Random variables X, Y are independent if for all sets A, B,

P(X ∈ A, Y ∈ B) = P(X ∈ A)P(Y ∈ B).

Intuitively: knowing one variable gives no information about the other.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 16

Equivalent Deﬁnitions of Independence

Theorem

The following are equivalent for random variables X and Y :

1

X and Y are independent.

2

F_X,Y(x, y) = F_X(x)F_Y(y).

3

4

5

6

7

If discrete:

p_X,Y(x, y) = p_X(x)p_Y(y).

f_X,Y(x, y) = f_X(x)f_Y(y).

f_{Y |X}(y|x) = f_Y(y).

If continuous:

E[g(X)h(Y )] = E[g(X)]E[h(Y )].

If MGFs exist:

M_X,Y(s, t) = M_X(s)M_Y(t).

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 17

Which Criterion Should We Use?

Diﬀerent characterizations are useful in diﬀerent situations.

▶ PMF/PDF factorization: usually easiest computationally;

▶ conditional distributions: most conceptual;

▶ expectation factorization: useful theoretically;

▶ MGF factorization: useful for sums and CLT.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 18

Independence of Several Random Variables

Deﬁnition

Random variables

X₁, . . . , X_n

are mutually independent if for every subset

i₁, . . . , i_k,

k

Y

P(X_i∈ A₁, . . . , X_i∈ A_k) =

P(X_i∈ A_j).

1

j

k

j=1

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 19

Pairwise Independence

Deﬁnition

Random variables

X₁, . . . , X_n

are pairwise independent if every pair

is independent.

(X_i, X_j)

mutual independence =⇒ pairwise independence.

But the converse is false.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 20

Pairwise Independent But Not Mutually Independent

Let

X, Y

be independent Bernoulli(1/2) random variables.

Deﬁne

Z = X ⊕ Y ,

where ⊕ denotes XOR.

Then:

▶ every pair is independent;

▶ but

Z

is completely determined by

X, Y .

Hence:

X, Y , Z

are not mutually independent.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 21

2D LOTUS

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 22

2D LOTUS

Theorem

For a function g(X, Y ),

X X





g(x, y)p_X,Y(x, y),

if X, Y are discrete,





x

y

E[g(X, Y )] =

ZZ





g(x, y)f_X,Y(x, y) dx dy, if X, Y are continuous.

Meaning

To ﬁnd the expected value of a function of two random variables, average g(x, y) using the joint distribution

of (X, Y ).

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 23

Linearity of Expectation Revisited

Using 2D LOTUS:

More generally:

E(X + Y ) = E(X) + E(Y ).

E(aX + bY + c) = aE(X) + bE(Y ) + c.

Importantly: independence is NOT required.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 24

Covariance and Correlation

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 25

Covariance

Deﬁnition

The covariance of X, Y is

Cov(X, Y ) = E[(X − E(X))(Y − E(Y ))].

Equivalent formula:

Cov(X, Y ) = E(XY ) − E(X)E(Y ).

Proposition: Cov(X, X) = Var(X)

Interpretation of Covariance

▶ Positive covariance: variables tend to move together;

▶ Negative covariance: one tends to increase while the other decreases;

▶ Zero covariance: no linear relationship (but there may be other forms of nonlinear relationships).

Important

Covariance measures linear dependence only.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 26

Correlation

Deﬁnition

The correlation coeﬃcient is

Cov(X, Y )

Corr(X, Y ) =

.

σ_Xσ_Y

Properties:

−1 ≤ Corr(X, Y ) ≤ 1.

Proof: Chapter 10a (inequalities) - needs Cauchy-Schwarz inequality.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 27

Variance of Sums

Theorem

For random variables X, Y ,

Var(X + Y ) = Var(X) + Var(Y ) + 2 Cov(X, Y ).

If X, Y are independent:

Var(X + Y ) = Var(X) + Var(Y ).

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 28

Independence Implies Zero Correlation

Theorem

If X, Y are independent, then

Cov(X, Y ) = 0.

Hence:

Corr(X, Y ) = 0.

This follows because:

E(XY ) = E(X)E(Y ).

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 29

Zero Correlation Does Not Imply Independence

Let

X ∼ N(0, 1),

and deﬁne

Y = X².

E(X) = 0,

E(XY ) = E(X³) = 0.

Cov(X, Y ) = 0.

Y

Then:

and

Hence:

But:

is completely determined by

X.

Thus X, Y are dependent.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 30

Heatmaps and Correlation Visualization

Now, suppose that instead of 2 variables, we have n variables X₁, . . . , X_n. If we consider all the correlations, i.e.,

ꢀ ꢁ

n

2

{Corr(X_i, X_j) : 1 ≤ i

̸

distinct correlations, so we can deﬁne a matrix whose ij entry is

Corr(X_i, X_j). This matrix is symmetric since Corr(X_i, X_j) = Corr(X_j, X_i). There is an interesting way to visualize

this matrix, because all entries are between -1 and 1, we can use a heatmap.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 31

Correlation Heatmap

▶ A heatmap provides a visual summary of

pairwise correlations between variables.

▶ Each cell shows the correlation coeﬃcient

r(X, Y )

between two variables.

▶ Interpretation:

• Blue: positive correlation.

• Red: negative correlation.

• White/light colors: weak or near-zero

correlation.

▶ The closer | Corr(X, Y )| is to 1, the stronger

the linear relationship.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 32

Multinomial Distribution

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 33

Multinomial Distribution (generalization of Binomial Distribution)

In the Binomial setting, each trial has two possible outcomes:

o₁: Success

or

o₂: Failure,

with probabilities

p₁= p, p₂= 1 − p,

where

p₁+ p₂= 1

After n independent trials,

X₁:= X

and

X₂:= n − X

count the number of trials resulting in category o₁: Success and o₂: Failure, respectively. Note that

X₁+ X₂= X + n − X = n

In the multinomial setting, each trial can result in one of k (instead of 2) possible categories

o₁, . . . , o_k,

with probabilities

p₁, . . . , p_k,

where

p₁+ · · · + p_k= 1

After n independent trials,

X₁, . . . X_k

count the number of trials resulting in category o₁, . . . , o_k. Note that X₁+ · · · + X_k= n, since each trial results in

exactly one category.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 34

Multinomial PMF

Deﬁnition

If (X₁, . . . , X_k) counts the numbers of occurrences of the k categories in n independent trials, then

(X₁, . . . , X_k) ∼ Mult(n, p₁, . . . , p_k).

n!

x

p₁^x· · · p_k

,

1

k

P(X₁= x₁, . . . , X_k= x_k) =

x₁! · · · x_k!

subject to:

x₁+ · · · + x_k= n.

Note. The Multinomial distribution is a multivariate distribution k − dimensional, while the Binomial is a

univariable distribution 1 − dimensional. A multivariate random variable is also called a random vector. Note. It

can be shown using the Multinomial Theorem (an extension of the Binomial Theorem) that the Multinomial PMF

sums up to 1. Click here to see the Multinomial Theorem in Wikipedia.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 35

Moments of the Multinomial

For

(X₁, . . . , X_k) ∼ Mult(n, p₁, . . . , p_k),

E(X_i) = np_i,

Var(X_i) = np_i(1 − p_i),

and for

i

̸= j,

Cov(X_i, X_j) = −np_ip_j.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 36

Multivariate Normal Distribution

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 37

Bivariate Normal Distribution

Big Picture

The bivariate normal distribution is the two-dimensional version of the normal distribution.

It describes two continuous random variables X and Y whose joint density has a bell-shaped surface over

the xy-plane.

A bivariate normal distribution is determined by ﬁve parameters:

µ_X, µ_Y, σ_X², σ_Y², ρ.

Here,

ρ = Corr(X, Y )

controls the strength and direction of the linear relationship between X and Y .

Click this link for interactive bivariate normal visualization.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 38

Bivariate Normal Distribution: Density

Deﬁnition

We say

ꢂꢂ ꢃ ꢂ

ꢃꢃ

µ_X

µ_Y

σ_X²

ρσ_Xσ_Y

σ_Y²

(X, Y ) ∼ N₂

,

ρσ_Xσ_Y

if the joint density of (X, Y ) is

1

p

f_X,Y(x, y) =

2πσ_Xσ_Y1 − ρ²

(

"

#)

ꢂ

ꢃ

ꢂ

ꢃ ꢂ

ꢃ

ꢂ

ꢃ

2

²− 2ρ

+

.

1

x − µ_X

σ_X

x − µ_X

σ_X

y − µ_Y

× exp

−

2(1 − ρ²)

σ_Y

where σ_X> 0, σ_Y> 0, and −1 < ρ < 1.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 39

Bivariate Normal Distribution: Main Properties

If

ꢂꢂ ꢃ ꢂ

ꢃꢃ

µ_X

µ_Y

σ_X²

ρσ_Xσ_Y

σ_Y²

(X, Y ) ∼ N₂

,

ρσ_Xσ_Y

then:

▶ The means are

E(X) = µ_X,

E(Y ) = µ_Y.

Var(Y ) = σ_Y².

▶ The variances are

▶ The covariance is

▶ Therefore,

Var(X) = σ_X²,

Cov(X, Y ) = ρσ_Xσ_Y.

Corr(X, Y ) = ρ.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 40

Marginal Distributions

A beautiful property of the bivariate normal distribution is that the marginal distributions are normal.

Marginals

If

ꢂꢂ ꢃ ꢂ

ꢃꢃ

µ_X

µ_Y

σ_X²

ρσ_Xσ_Y

σ_Y²

(X, Y ) ∼ N₂

,

ρσ_Xσ_Y

then

and

X ∼ N(µ_X, σ_X²)

Y ∼ N(µ_Y, σ_Y²).

So the joint distribution is bivariate normal, but each marginal variable individually is ordinary normal.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 41

Conditional Distributions

Another beautiful property is that the conditional distributions are also normal.

Conditionals

If

(X, Y )

is bivariate normal, then

ꢂ

ꢃ

σ_X

X | Y = y ∼ N µ_X+ ρ

(y − µ_Y), σ_X²(1 − ρ²)

σ_Y

and

ꢂ

ꢃ

σ_Y

σ_X

Y | X = x ∼ N µ_Y+ ρ

(x − µ_X), σ_Y²(1 − ρ²)

.

The conditional mean is linear in the observed value.

Arman Jahangiri Chapter 7: Joint Distributions

Summer 2026 42

Independence and Zero Correlation

For general random variables,

Corr(X, Y ) = 0

does not necessarily imply independence.

But for the bivariate normal distribution, it does.

Important Gaussian Property

If (X, Y ) is bivariate normal, then

X and Y independent

⇐⇒ ρ = 0.

When ρ = 0, the joint density factors:

f_X,Y(x, y) = f_X(x)f_Y(y).

So in the bivariate normal case,

uncorrelated ⇐⇒ independent.

Arman Jahangiri

Chapter 7: Joint Distributions

Summer 2026 43