Chapter 4: Expectation

MATH/STAT 394: Probability I

Arman Jahangiri

University of Washington Department of Mathematics

Summer 2026

Arman Jahangiri

Chapter 4: Expectation

Summer 2026

1

Learning Goals

By the end of this lecture, students should be able to:

▶ deﬁne expectation for discrete random variables;

▶ compute expectations from PMFs;

▶ use linearity of expectation;

▶ use indicator random variables and the fundamental bridge;

▶ deﬁne and compute variance and standard deviation;

▶ understand the Poisson approximation to the Binomial;

▶ know the expectations of the main discrete distributions.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026

2

Deﬁnition of Expectation

Arman Jahangiri

Chapter 4: Expectation

Summer 2026

3

Why Expectation?

A distribution gives detailed information about a random variable:

P(X = x),

P(X ≤ x),

P(a ≤ X ≤ b).

But sometimes we want a single number summarizing the “average” value of X.

Main idea

The expected value of a random variable is its probability-weighted average.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026

4

Idea: (Weighted) Averages

For ordinary numbers

the usual average is

x₁, x₂, . . . , x_n,

n

X

1

x¯ =

x_j

(1)

n

i=1

More generally, if

n

X

p₁, . . . , p_n≥ 0,

p_j= 1,

j=1

then the weighted average is

n

X

x_jp_j.

j=1

Note that equation (1) uses equal weights p_j= _n¹, ∀j = 1, . . . , n, so it is a special case.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026

5

Deﬁnition: Expected Value

Deﬁnition (Expected value of a discrete r.v.)

Let X be a discrete random variable with possible values

x₁, x₂, . . . .

The expected value of X is

X

E(X) =

x_jP(X = x_j).

j

Equivalently,

where

X

E(X) =

x p_X(x),

x

p_X(x) = P(X = x).

Interpretation

Expectation is the center of mass of the PMF (next slide).

Arman Jahangiri Chapter 4: Expectation

Summer 2026

6

Expectation as a Center of Mass (Optional)

Suppose a discrete random variable X takes values

x₁, x₂, . . .

with probabilities

p₁, p₂, . . .

Think of the PMF as placing masses p_iat locations x_ion the number line.

The center of mass (balance point) is a point c such that

X

(x_i− c)p_i= 0.

i

This means:

▶ points to the right of c pull one way,

▶ points to the left of c pull the other way,

▶ and the system balances perfectly at c.

Expanding the equation:

X

x_ip_i− c

p_i= 0.

i

Since

X

p_i= 1,

i

we obtain

X

Chapter 4: Expectation

Arman Jahangiri

Summer 2026

7

E

Example: One Fair Die

Let

X = result of one fair die roll.

Then

1

P(X = k) =

,

k = 1, 2, . . . , 6.

6

Thus

6

X

1 + 2 + 3 + 4 + 5 + 6

6

E(X) =

kP(X = k) =

.

k=1

So

E(X) = 3.5.

Important

The expectation does not have to be a possible value of X.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026

8

Example: Bernoulli

Let

X ∼ Bern(p).

P(X = 1) = p, P(X = 0) = 1 − p.

Then

Therefore

So

E(X) = 1 · p + 0 · (1 − p) = p.

E(X) = p.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026

9

Example: Discrete Uniform

Let

X ∼ DUnif({a, a + 1, . . . , b}).

All values are equally likely.

There are

b − a + 1

possible values.

Thus

a + (a + 1) + · · · + b

E(X) =

.

b − a + 1

The average of equally spaced endpoints is the midpoint:

a + b

E(X) =

.

2

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 10

Example: Geometric Distribution

Let

X ∼ Geom(p).

Recall:

P(X = k) = (1 − p)^k−1p,

k = 1, 2, . . .

Then

∞

X

E(X) =

k(1 − p)^k−1p.

k=1

Using the geometric-series identity,

∞

X

1

kr^k−1

=

,

|r| < 1,

(1 − r)²

k=1

with r = 1 − p, we obtain

1

E(X) =

.

p

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 11

Example: Negative Binomial Distribution

Let

X ∼ NegBin(r, p),

where X counts the number of trials needed to obtain r successes.

Write

X = X₁+ · · · + X_r,

where each

X_i∼ Geom(p).

Therefore,

1

E(X) = r ·

.

p

Hence

r

E(X) =

.

p

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 12

Example: Poisson Distribution

Let

X ∼ Pois(λ).

Recall:

Then

_−λλ^k

P(X = k) = e

,

k = 0, 1, 2, . . .

k!

∞

_−λλ^k

X

E(X) =

ke

.

k!

k=0

Rearranging,

Since

∞

λ^k−1

X

= λe^−λ

.

(k − 1)!

k=1

∞

j

X

λ

= e^λ,

j!

j=0

we obtain

E(X) = λ.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 13

Expectation Depends Only on the Distribution

Proposition

If X and Y have the same distribution, then

E(X) = E(Y ),

provided the expectations exist.

Reason: the formula

X

E(X) =

xP(X = x)

x

depends only on the PMF of X.

Warning

The converse is false: two random variables can have the same expectation but diﬀerent distributions.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 14

Linearity of Expectation

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 15

Linearity of Expectation

Theorem

For random variables X, Y and constant c,

E(X + Y ) = E(X) + E(Y ).

Also,

E(cX) = cE(X).

More generally,

!

n

X

E

c_jX_j

=

c_jE(X_j).

j=1

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 16

Application of Linearity in ﬁnding the Expectation of Bin(n,p)

Let

X ∼ Bin(n, p).

Think of X as the number of successes in n Bernoulli trials.

Write

X = I₁+ · · · + I_n,

where

(

1, success on trial j,

I_j=

0, otherwise.

Each

I_j∼ Bern(p),

so

E(I_j) = p.

Therefore, by linearity of expectation,

E(X) = E[I₁+ · · · + I_n] = E[I₁] + · · · + E[I_n] = p + · · · + p = np.

|

{z

}

n times

Note that we did not even use the PMF of Ber(n, p) to ﬁnd its expectation using the main formula, which made it

much easier computationally.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 17

Example: Hypergeometric Expectation

Let

X ∼ HGeom(w, b, n),

where X is the number of white balls in a sample of size n drawn without replacement.

Write

X = I₁+ · · · + I_n,

where I_j= 1 if the jth draw is white.

For each j,

w

P(I_j= 1) =

.

w + b

So

w

E(I_j) =

.

w + b

By linearity,

w

E(X) = n

.

w + b

Key point

The draws are dependent, but linearity still works.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 18

Monotonicity of Expectation

Theorem

If

X ≥ Y

with probability 1, then

E(X) ≥ E(Y ).

Sketch of proof:

Therefore,

X − Y ≥ 0.

E(X − Y ) ≥ 0.

E(X) − E(Y ) ≥ 0.

By linearity,

For a concise proof, we need more technical tools that are not covered in this course.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 19

Indicator Random Variables and the Fundamental Bridge

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 20

Indicator Random Variables

For an event A, deﬁne

(

1, if A occurs,

I_A

=

0, if A does not occur.

Then

I_A∼ Bern(P(A)).

Interpretation

Indicators convert events into random variables.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 21

Basic Indicator Properties

For events A and B:

c

I_A= 1 − I_A.

I_A∩B= I_AI_B.

I_A∪B= I_A+ I_B− I_AI_B.

Reason

Indicators behave like yes/no algebra:

1 = yes,

0 = no.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 22

The Fundamental Bridge

Fundamental Bridge

For any event A ⊆ S,

E(I_A) = P(A).

Proof:

I_A∼ Bern(P(A)).

Since the expectation of a Bernoulli random variable is its success probability,

E(I_A) = P(A).

Meaning

Probability questions can often be turned into expectation questions, and vice versa.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 23

Example: Matching Problem

A deck has cards labeled

1, 2, . . . , n.

After shuﬄing, a match occurs if card j is in position j.

Let

X = number of matches.

Deﬁne

(

1, card j is in position j,

I_j=

0, otherwise.

Then

Since

X = I₁+ · · · + I_n.

1

P(I_j= 1) =

,

n

we get

1

E(X) = n · = 1.

n

E(X) = 1.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 24

Example: Birthday Matches

Suppose there are n people.

Let

X = number of pairs of people with the same birthday.

There are

!

n

2

pairs.

For each pair (i, j), deﬁne

(

1, people i, j have the same birthday,

I_ij=

0, otherwise.

Then

X

X =

I_ij.

i<j

Since

1

E(I_ij) = P(I_ij= 1) =

,

365

we have

!

Chapter 4: Expectation

n

1

E(X) =

.

2

365

Arman Jahangiri

Summer 2026 25

Variance

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 26

Why Variance?

Expectation measures the center of a distribution.

But two distributions can have the same expectation and very diﬀerent spread.

We need a numerical measure of variability.

Variance measures the average squared distance from the mean.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 27

Deﬁnition: Variance and Standard Deviation

Deﬁnition

The variance of X is

h

i

Var(X) = E (X − EX)²

.

Variance is a measure of dispersion (from the mean). It is a measure of how far a set of numbers are spread

out from their mean, on average.

The standard deviation is

p

SD(X) = Var(X).

Variance has squared units; standard deviation returns to the original units.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 28

Motivation behind the formula E[X − E[X]]²?

Consider the function

ꢀ

ꢁ

g(c) = E (X − c)²

.

The quantity g(c) measures the average squared distance between X and the constant c.

We claim that g(c) is minimized when

c = E[X].

Expand the square:

h

i

g(c) = E (X − E[X] + E[X] − c)²

h

i

= E (X − E[X])²+ 2(E[X] − c) E[X − E[X]]

+ (E[X] − c)².

Since E[X − E[X]] = 0,, we obtain g(c) = Var(X) + (E[X] − c)².

Because

(E[X] − c)²≥ 0,

the minimum occurs when

c = E[X].

Thus, the expectation is the best constant predictor of X in mean squared error.

Arman Jahangiri Chapter 4: Expectation

Summer 2026 29

Alternative Formula for Variance

Variance Formula

Var(X) = E(X²) − (EX)².

Proof : Let

Then

µ = EX.

h

i

Var(X) = E (X − µ)²

.

Expanding:

E(X²− 2µX + µ²) = E(X²) − 2µE(X) + µ².

Since µ = EX,

Var(X) = E(X²) − µ².

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 30

Properties of Variance

Shifting does not change spread. Scaling by b scales squared distances by b².

Var(X + a) = Var(X) and Var(bX) = b²Var(X).

Warning

Unlike Expectation, Variance is not linear, i.e., in general,

Var(X + Y ) ̸= Var(X) + Var(Y ).

In fact,

Var(X + Y ) = Var(X) + Var(Y ) + 2Cov(X, Y )

(2)

(3)

and for n > 1 random variables,

n

X

Var(

X_i) =

Var(X_i) +

Cov(X_i, X_j)

i=1

i

̸=j

We will study this in further depth in Chapter 7 (Joint Distributions).

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 31

Variance of a Binomial Random Variable

Let

X ∼ Bin(n, p).

Write

X = I₁+ · · · + I_n,

where the I_jare independent Bernoulli(p) random variables.

Each has variance

Var(I_j) = p(1 − p).

Since the indicators are independent (*),

n

X

Var(X) =

Var(I_j).

j=1

Therefore

Var(X) = np(1 − p).

(*): Note that we previously proved that the expected value of a sum of random variables is equal to the sum of

their expected values. However, this property does not generally hold for variance. In general, the variance of a

sum is not equal to the sum of the variances. Variances add only in the special case where the random variables

have zero covariance, that is, when they are uncorrelated. In particular, independence implies zero covariance, so

variances of independent random variables do add. We will study covariance, correlation, and independence in

more detail later. For a direct derivation of the variance of the Binomial distribution using its PMF, click here.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 32

A Useful Variance Formula (exercise)

Show that for any random variable X,

Var(X) = E[X(X − 1)] + E[X] − (E[X])².

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 33

Discrete Uniform Variance

Let

X ∼ DUnif{1, 2, . . . , n}.

Then

n + 1

E[X] =

.

2

Also,

n

X

1

n(n + 1)(2n + 1)

(n + 1)(2n + 1)

E[X²] =

k²=

·

=

.

n

6

k=1

Therefore,

Var(X) = E[X²] − (E[X])².

n²− 1

Var(X) =

.

12

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 34

Poisson Variance

Let

X ∼ Pois(λ).

Then

_−λλ^k

P(X = k) = e

,

k = 0, 1, 2, . . .

k!

We use

∞

_−λλ^k

X

E[X(X − 1)] =

k(k − 1)e

.

k!

k=0

For k ≥ 2,

λ^k

k(k − 1)

=

.

k!

(k − 2)!

Thus,

∞

j

X

λ

E[X(X − 1)] = λ²e^−λ

= λ².

j!

j=0

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 35

Poisson Variance Continued

Since

E[X] = λ,

we get

Var(X) = E[X(X − 1)] + E[X] − (E[X])².

Therefore,

Var(X) = λ²+ λ − λ².

Var(X) = λ.

So for a Poisson random variable,

E[X] = Var(X) = λ.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 36

Geometric Variance

Let

X ∼ Geom(p),

where

Thus,

X = number of trials until the ﬁrst success.

P(X = k) = (1 − p)^k−1p,

k = 1, 2, 3, . . .

Let q = 1 − p. Then

1

E[X] =

.

p

A standard power series identity gives

∞

X

1 + q

(1 − q)³

k²q^k−1

=

.

k=1

Therefore,

∞

X

1 + q

(1 − q)³

E[X²] = p

k²q^k−1= p ·

.

k=1

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 37

Geometric Variance Continued

Since

1 − q = p,

we have

1 + q

p³

1 + q

p²

E[X²] = p ·

=

.

Thus,

So

Var(X) = E[X²] − (E[X])².

1 + q

p²

1

p²

q

p²

Var(X) =

−

=

.

Since q = 1 − p,

1 − p

Var(X) =

.

p²

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 38

Negative Binomial Variance

Let

X ∼ NegBin(r, p),

where

X = number of trials needed to obtain r successes.

A Negative Binomial random variable can be written as a sum of r independent Geometric random variables:

X = G₁+ G₂+ · · · + G_r,

where each

G_i∼ Geom(p).

Each G_iis the waiting time for one additional success.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 39

Negative Binomial Variance Continued

Since

X = G₁+ · · · + G_r

and the G_iare independent,

Var(X) = Var(G₁) + · · · + Var(G_r).

From the Geometric variance formula,

Therefore,

1 − p

Var(G_i) =

.

p²

1 − p

Var(X) = r ·

.

p²

r(1 − p)

Var(X) =

p²

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 40

Hypergeometric Variance

Let

X ∼ HGeom(w, b, n),

where an urn has w white balls and b black balls.

Let

N = w + b.

We draw n balls without replacement, and

X = number of white balls selected.

Write

where

X = I₁+ I₂+ · · · + I_n,

(

1, if draw j is white,

I_j=

0, otherwise.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 41

Hypergeometric Variance Continued

Each indicator satisﬁes

w

N

E[I_j] =

.

So

w

E[X] = n

.

N

Also,

ꢂ

ꢃ

w

N

Var(I_j) =

1 −

.

N

Because sampling is without replacement, the indicators are not independent.

For i ̸= j,

w b

1

Cov(I_i, I_j) = −

.

N N N − 1

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 42

Hypergeometric Variance Formula

Using

n

X

Var(X) =

Var(I_j) + 2

Cov(I_i, I_j),

j=1

i<j

we get

Factor:

w b

1

Var(X) = n

− n(n − 1)

.

N N

N N N − 1

ꢄ

ꢅ

w b

N N

n − 1

Var(X) = n

1 −

.

N − 1

Therefore,

w b N − n

Var(X) = n

.

N N N − 1

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 43

Nonnegativity of Variance

Proposition

For any r.v. X,

Var(X) ≥ 0.

Reason:

(X − EX)²≥ 0.

Thus its expectation is nonnegative.

Later, we will learn the Jensen’s Inequality, which implies this Proposition as a special case.

Also,

Var(X) = 0

if and only if X is constant with probability 1.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 44

Poisson Approximation to Binomial

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 45

Poisson Approximation to Binomial

When

X ∼ Bin(n, p).

AND

▶ n is large;

▶ p is small;

▶ λ = np is moderate,

Then,

X ≈ Pois(λ).

That is,

!

n

_−λλ^k

P(X = k) =

p^k(1 − p)^n−k≈ e

.

k

k!

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 46

Why the Poisson Approximation Works

Assume X ∼ Bin(n, p) with p = λ/n, where λ > 0 is ﬁxed. For each ﬁxed k = 0, 1, 2, . . . , we have

!

ꢄ

ꢅ ꢄ

ꢅ

k

n−k

n

λ

n!

λ^k

λ

−k

P_Binom(n,p)(X = k) =

1 −

=

(1 − )ⁿ(1 −

)

k

n

k!(n − k)! n^k

n

ꢆ

|

ꢄ

ꢅ ꢄ

ꢅ

ꢄ

ꢅꢇ

ꢄ

ꢅ

ꢄ

ꢅ_−k

}

n

1

2

k − 1

λ^k

λ

=

(1) 1 −

1 −

· · · 1 −

1 −

n

k!

stays ﬁxed

n

|{z}

{z

}

|

{z

} |

{z

−λ

−→ 1

−→ e

λ^k

_−λλ^k

−→ 1 ·

· e^−λ· 1 = e

= P_{X∼Poisson(λ)}(X = k)

k!

Thus,

_−λλ^k

n→∞

P

_X∼Bin(n,p)(X = k) −−−→ e

.

k!

This phenomenon is formally known as convergence in distribution (or weak convergence) in Probability theory.

We will see more with Central Limit Theorem (CLT) and Law of Large Numbers (LLN) in Chapter 10B (Limit

Theorems).

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 47

Visualization of Poisson Approximation to the Binomial

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 48

Example: Website Visitors

Suppose 1,000,000 people independently decide whether to visit a website.

Each visits with probability

p = 2 × 10⁻⁶

.

Let

X = number of visitors.

X ∼ Bin(1,000,000, 2 × 10⁻⁶).

λ = np = 2.

Then

Here

So

X ≈ Pois(2).

For example,

P(X = 0) ≈ e⁻²

.

Note that it is computationally hard to compute directly from the Binomial PMF:

1000000!

k!(1000000 − k)! 10⁶

2

10⁶

1000000−k

P(X = k) =

(

)^k(1 −

)

(4)

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 49

Summary

▶ Expectation is a probability-weighted average:

X

E(X) =

xP(X = x).

x

▶ Linearity:

E(X + Y ) = E(X) + E(Y ),

with no independence required.

▶ Fundamental bridge:

E(I_A) = P(A).

▶ Variance measures spread:

Var(X) = E(X²) − (EX)².

▶ Poisson approximates Binomial when n is large, p is small, and λ = np is moderate.

Next lecture

Continuous random variables.

Arman Jahangiri

Chapter 4: Expectation

Summer 2026 50