Chapter 10A: Inequalities

MATH/STAT 394: Probability I

Arman Jahangiri

University of Washington Department of Mathematics

Summer 2026

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026

1

Learning Goals

By the end of this lecture, students should be able to:

▶ understand why inequalities are important in probability;

▶ apply the Cauchy–Schwarz inequality;

▶ use Jensen’s inequality with convex and concave functions;

▶ apply Markov’s inequality;

▶ apply Chebyshev’s inequality;

▶ apply Chernoﬀ bounds;

▶ interpret inequalities geometrically and probabilistically.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026

2

Why Inequalities Matter

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026

3

What If Exact Calculation Is Impossible?

In many real problems:

▶ exact probabilities are diﬃcult;

▶ integrals may be impossible;

▶ distributions may be unknown;

▶ computations may be too expensive.

Three major strategies:

1

simulation;

2

inequalities (bounds);

3

asymptotic approximations.

Today’s focus

Probability inequalities

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026

4

What Is a Bound?

Suppose we cannot compute

P(A)

exactly.

An inequality may tell us:

0.01 ≤ P(A) ≤ 0.03.

Even without the exact answer, we gain useful information.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026

5

Cauchy–Schwarz Inequality

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026

6

Cauchy–Schwarz Inequality

Theorem

For random variables X, Y with ﬁnite second moments,

p

|E(XY )| ≤ E(X²)E(Y ²).

This is one of the most important inequalities in mathematics.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026

7

Idea Behind the Proof

For every real number t,

Taking expectations:

Expanding:

(Y − tX)²≥ 0.

E(Y − tX)²≥ 0.

E(Y ²) − 2tE(XY ) + t²E(X²) ≥ 0.

This quadratic must always be nonnegative.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026

8

Proof of Cauchy–Schwarz

Since

E(Y ²) − 2tE(XY ) + t²E(X²) ≥ 0

for all t, its discriminant must satisfy:

(−2E(XY ))²− 4E(X²)E(Y ²) ≤ 0.

Thus,

E(XY )²≤ E(X²)E(Y ²).

Taking square roots gives:

p

|E(XY )| ≤ E(X²)E(Y ²).

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026

9

Application of Cauchy-Schwarz: −1 ≤ Corr(X, Y ) ≤ 1

Apply Cauchy–Schwarz to

X − E(X)

and

Y − E(Y ).

Then:

p

| Cov(X, Y )| ≤ Var(X) Var(Y ).

Dividing both sides:

| Corr(X, Y )| ≤ 1.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 10

Application of Cauchy-Schwarz: Var(X) ≥ 0

Apply Cauchy–Schwarz with

Then:

Y = 1.

2

|E(X)| ≤ E(X²).

Therefore,

Var(X) = E(X²) − (E(X))²≥ 0.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 11

Jensen’s Inequality

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 12

Convex Functions

Deﬁnition

A function g : I → R is called convex if

g(λx + (1 − λ)y) ≤ λg(x) + (1 − λ)g(y)

for all x, y ∈ I and all

0 ≤ λ ≤ 1.

Interpretation: λx + (1 − λ)y is a weighted average of x and y, and convexity says g(average) ≤ average of g.

Geometrically: the graph of a convex function always lies below its secant lines.

Second Derivative Test

If g is twice diﬀerentiable, then:

g^′′(x) ≥ 0 for all x

implies that g is convex.

Examples of convex functions:

g(x) = x²,

g(x) = e^x.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 13

Jensen’s Inequality

Theorem

If g is convex, then

E(g(X)) ≥ g(E(X)).

If g is concave, the inequality reverses:

E(g(X)) ≤ g(E(X)).

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 14

Geometric Interpretation of Jensen

For a convex function:

▶ tangent lines lie below the graph;

▶ averaging inputs produces smaller values than averaging outputs.

Mnemonic

g(average) ≤ average of g.

for convex functions.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 15

Example (Jensen’s Inequality): Var(X) ≥ 0 (again)

Take

g(x) = x².

Since

g^′′(x) = 2 > 0,

the function is convex.

Jensen gives:

E(X²) ≥ (E(X))².

Thus:

Var(X) ≥ 0.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 16

Example (Jensen’s Inequality): Logarithms

Take

g(x) = log x.

Since

1

x²

g^′′(x) = −

< 0,

the logarithm is concave.

Thus:

E(log X) ≤ log(E(X)).

This inequality is fundamental in statistics and information theory.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 17

Example (Jensen’s Inequality): Sample Standard Deviation Is Biased

Recall:

S_n²

is unbiased for

σ².

But:

p

S_n= S_n².

Since

√

g(x) =

x

is concave, Jensen gives:

q

p

E(S_n) = E( S_n²) ≤ E(S_n²) = σ.

Conclusion

Sample standard deviation tends to underestimate the true standard deviation.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 18

Tail Bounds

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 19

Tail Probabilities

We often want bounds on:

or

P(X ≥ a),

P(|X − µ| ≥ a).

These are called tail probabilities.

Applications

▶ rare events;

▶ risk analysis;

▶ statistics;

▶ machine learning;

▶ concentration inequalities.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 20

Markov’s Inequality

Theorem (Markov’s Inequality)

If X ≥ 0 and a > 0, then

E(X)

a

P(X ≥ a) ≤

.

Proof

First, note that X ≥ a I(X ≥ a).

Taking expectations from both sides,

E(X) ≥ aE(I(X ≥ a)).

Using the fundamental bridge:

Therefore:

E(I(X ≥ a)) = P(X ≥ a).

E(X)

P(X ≥ a) ≤

.

a

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 21

Chebyshev’s Inequality

Theorem (Chebyshev’s Inequality)

If X has mean µ and variance σ², then for every a > 0,

σ²

a²

P(|X − µ| ≥ a) ≤

.

Proof

Apply Markov to (X − µ)²≥ 0. Then,

E(X − µ)²

σ²

a²

P((X − µ)²≥ a²) ≤

=

.

a²

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 22

Standardized Form of Chebyshev

Let

a = cσ.

Then:

1

c²

P(|X − µ| ≥ cσ) ≤

.

Examples:

1

P(|X − µ| ≥ 2σ) ≤

,

4

1

9

P(|X − µ| ≥ 3σ) ≤

.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 23

Chernoﬀ Bounds

Markov becomes stronger if we apply it to:

e^tX

,

t > 0.

Chernoﬀ Bound

For every

t > 0,

E(e^tX

)

P(X ≥ a) ≤

.

e^ta

The numerator is the MGF of X.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 24

Why Chernoﬀ Is Powerful

Advantages:

▶ often much sharper than Chebyshev;

▶ uses MGFs;

▶ can optimize over

t.

Chernoﬀ bounds are central in:

▶ machine learning;

▶ randomized algorithms;

▶ statistical learning theory;

▶ concentration inequalities.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 25

Comparing Bounds for Normal Tails

Suppose

Z ∼ N(0, 1).

P(|Z| > 3) ≈ 0.003.

Markov: 0.27,

True probability:

Bounds:

Chebyshev: 0.11,

Chernoﬀ: 0.022.

Observation

Chernoﬀ bounds are often dramatically better.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 26

Summary

▶ Cauchy–Schwarz bounds expectations.

▶ Jensen compares

E(g(X))

and

g(E(X)).

▶ Markov bounds nonnegative tails.

▶ Chebyshev controls deviations from the mean.

▶ Chernoﬀ bounds are often much sharper.

▶ Inequalities are fundamental tools throughout probability and statistics.

Next lecture

Law of Large Numbers and Central Limit Theorem.

Arman Jahangiri

Chapter 10A: Inequalities

Summer 2026 27