University of Washington
Department of Mathematics Homework 8 Solutions

MATH 394: Probability I

Instructor: Arman Jahangiri

Submission: Single PDF on Gradescope

Deadline: 10:00 PM Pacific Time on the listed due date

This homework is worth 50 points.

Across the quarter, there are eight homework assignments, worth 400 points total. Homework assignments together account for 40% of the final course grade. Further course policies can be seen in the MATH 394 syllabus.

Homework Policy

Submission and Deadline

Please:

Submit homework as a single PDF on Gradescope by 10:00 PM (Pacific Time) on the listed due date.
Begin each problem on a new page, and ensure that pages are correctly tagged in Gradescope.
Write solutions clearly and legibly.
Typesetting solutions in LaTeX is encouraged, but not required.

Late Days

Each student is allotted six late days for the quarter. A late day extends a homework deadline by up to 24 hours without penalty. For example, submitting an assignment anytime between 10:01 PM on the due date and 10:00 PM the following day counts as one late day.

The following rules apply:

No explanation is required to use late days.
At most two late days may be used on any single homework assignment.
Late days may not be used on the final homework assignment or on any assignment for which solutions have already been released.
It is the student’s responsibility to keep track of how many late days have been used during the quarter.

Once all late days have been exhausted, additional late submissions will incur a penalty of 10% per day, up to a maximum deduction of 50%. Assignments submitted more than five days late, or after solutions have been released, will not be accepted.

Submission Issues and Technical Difficulties

If a serious technical issue prevents a timely Gradescope submission, students may temporarily submit their assignment by email to the instructor at armanjg@uw.edu. In such cases:

The submission must consist of a single PDF file. The email subject line must follow the format: \[ \texttt {HW\# Submission - FULL NAME - STUDENT ID}. \]
Once the Gradescope issue has been resolved, the student must upload the exact same file to Gradescope as soon as possible.
The student should clearly indicate in the Gradescope submission that the assignment was previously submitted by email before the deadline.

Email submissions are intended only for genuine technical emergencies and should not be used as a substitute for timely Gradescope submission.

Problem 1.

Source: Blitzstein and Hwang, Introduction to Probability, Chapter 8, Problem 1–5.

Find the following PDFs.

(a): Find the PDF of \(e^{-X}\) for \(X\sim \operatorname {Exp}(1)\).
(b): Find the PDF of \(X^7\) for \(X\sim \operatorname {Exp}(\lambda )\).
(c): Find the PDF of \(Z^3\) for \(Z\sim N(0,1)\).
(d): Find the PDF of \(Z^4\) for \(Z\sim N(0,1)\).
(e): Find the PDF of \(|Z|\) for \(Z\sim N(0,1)\).

Solution

Let \(\phi (z)=(2\pi )^{-1/2}e^{-z^2/2}\) denote the standard normal density.

(a) Put \(Y=e^{-X}\). Since \(X>0\), we have \(0<Y<1\). The inverse transformation is \(x=-\log y\), and \[ \left |\frac {dx}{dy}\right |=\frac 1y. \] Thus \[ f_Y(y)=e^{-(-\log y)}\frac 1y=1, \qquad 0<y<1. \]

(b) Put \(Y=X^7\). The inverse is \(x=y^{1/7}\) for \(y>0\), and \[ \left |\frac {dx}{dy}\right |=\frac {1}{7}y^{-6/7}. \] Therefore \[ f_Y(y)=\frac {\lambda }{7}y^{-6/7}e^{-\lambda y^{1/7}}, \qquad y>0. \]

(c) Put \(Y=Z^3\). This transformation is one-to-one. For \(y\ne 0\), the inverse is \(z=y^{1/3}\) and \[ \left |\frac {dz}{dy}\right |=\frac {1}{3|y|^{2/3}}. \] Hence \[ f_Y(y)=\frac {\phi (y^{1/3})}{3|y|^{2/3}}, \qquad y\ne 0. \] The value assigned to the density at the single point \(y=0\) is immaterial.

(d) Put \(Y=Z^4\). The support is \(y>0\). For each \(y>0\), the two preimages are \(y^{1/4}\) and \(-y^{1/4}\). Adding the two contributions gives \[ f_Y(y) = \phi (y^{1/4})\frac {1}{4y^{3/4}} + \phi (-y^{1/4})\frac {1}{4y^{3/4}} = \frac {\phi (y^{1/4})}{2y^{3/4}}, \qquad y>0. \]

(e) Put \(Y=|Z|\). For \(y>0\), the two preimages are \(y\) and \(-y\), and therefore \[ f_Y(y)=\phi (y)+\phi (-y)=2\phi (y), \qquad y>0. \]

Problem 2.

Source: Blitzstein and Hwang, Introduction to Probability, Chapter 8, Problem 13.

Let \(X\) and \(Y\) be i.i.d. \(\operatorname {Exp}(\lambda )\), and let \[ T=\log (X/Y). \] Find the CDF and PDF of \(T\).

Solution

For real \(t\), \[ F_T(t)=P\!\left (\log (X/Y)\le t\right )=P(X\le e^tY). \] Conditioning on \(Y=y\) gives \[ F_T(t)=\int _0^\infty P(X\le e^t y)\lambda e^{-\lambda y}\dd y. \] Since \(X\sim \operatorname {Exp}(\lambda )\), \[ P(X\le e^t y)=1-e^{-\lambda e^t y}. \] Thus \[ F_T(t)=\int _0^\infty \left (1-e^{-\lambda e^t y}\right )\lambda e^{-\lambda y}\dd y =1-\frac {1}{1+e^t} =\frac {e^t}{1+e^t}. \] Differentiating the CDF gives \[ f_T(t)=\frac {e^t}{(1+e^t)^2}, \qquad -\infty <t<\infty . \] The parameter \(\lambda \) disappears because only the ratio \(X/Y\) matters.

Problem 3.

Source: Blitzstein and Hwang, Introduction to Probability, Chapter 8, Problem 14.

Let \(X\) and \(Y\) have joint PDF \(f_{X,Y}(x,y)\), and transform \((X,Y)\mapsto (T,W)\) linearly by letting \[ T=aX+bY, \qquad W=cX+dY, \] where \(a,b,c,d\) are constants such that \[ ad-bc\ne 0. \]

(a): Find the joint PDF \(f_{T,W}(t,w)\) in terms of \(f_{X,Y}\), though your answer should be written as a function of \(t\) and \(w\).
(b): For the case where \[ T=X+Y, \qquad W=X-Y, \] show that \[ f_{T,W}(t,w) = \frac 12 f_{X,Y}\left (\frac {t+w}{2},\frac {t-w}{2}\right ). \]

Solution

(a) Let \[ D=ad-bc. \] Solving the linear system gives \[ x=\frac {dt-bw}{D}, \qquad y=\frac {-ct+aw}{D}. \] The Jacobian determinant of the inverse transformation is \[ \left |\frac {\partial (x,y)}{\partial (t,w)}\right |=\frac {1}{|D|}. \] Therefore \[ f_{T,W}(t,w) = \frac {1}{|ad-bc|} f_{X,Y}\left (\frac {dt-bw}{ad-bc},\frac {-ct+aw}{ad-bc}\right ), \] for all \((t,w)\) whose inverse point lies in the support of \((X,Y)\).

(b) Here \(a=1\), \(b=1\), \(c=1\), and \(d=-1\), so \[ ad-bc=-2. \] Also \[ x=\frac {t+w}{2}, \qquad y=\frac {t-w}{2}. \] Thus \[ f_{T,W}(t,w) = \frac 12 f_{X,Y}\left (\frac {t+w}{2},\frac {t-w}{2}\right ). \]

Problem 4.

Source: Blitzstein and Hwang, Introduction to Probability, Chapter 10, Problem 2.

For i.i.d. random variables \(X_1,\dots ,X_n\) with mean \(\mu \) and variance \(\sigma ^2\), give a value of \(n\), as a specific number, that will ensure that there is at least a \(99\%\) chance that the sample mean will be within \(2\) standard deviations of the true mean \(\mu \).

Solution

The sample mean has \[ E(\bar X_n)=\mu , \qquad \Var (\bar X_n)=\frac {\sigma ^2}{n}. \] Chebyshev’s inequality gives \[ P(|\bar X_n-\mu |\ge 2\sigma ) \le \frac {\Var (\bar X_n)}{(2\sigma )^2} = \frac {1}{4n}. \] To make the probability of being outside the interval at most \(0.01\), it is enough to require \[ \frac {1}{4n}\le 0.01. \] Thus \(n\ge 25\) is sufficient. With \(n=25\), Chebyshev gives \[ P(|\bar X_n-\mu |<2\sigma )\ge 0.99. \]

Problem 5.

Source: Blitzstein and Hwang, Introduction to Probability, Chapter 10, Problem 4.

The famous arithmetic mean–geometric mean inequality says that for any positive numbers \[ a_1,a_2,\dots ,a_n, \] we have \[ \frac {a_1+a_2+\cdots +a_n}{n} \ge (a_1a_2\cdots a_n)^{1/n}. \]

Show that this inequality follows from Jensen’s inequality, by considering \(E(\log X)\) for a random variable \(X\) whose possible values are \(a_1,\dots ,a_n\). You should specify the PMF of \(X\).

Solution

Let \(X\) take the values \(a_1,a_2,\dots ,a_n\) with equal probabilities: \[ P(X=a_j)=\frac 1n, \qquad j=1,\dots ,n. \] Then \[ E(X)=\frac {a_1+\cdots +a_n}{n} \] and \[ E(\log X)=\frac 1n\sum _{j=1}^n \log a_j = \log \left ((a_1a_2\cdots a_n)^{1/n}\right ). \] Since \(\log x\) is concave, Jensen’s inequality gives \[ E(\log X)\le \log (E X). \] Therefore \[ \log \left ((a_1a_2\cdots a_n)^{1/n}\right ) \le \log \left (\frac {a_1+\cdots +a_n}{n}\right ). \] Because the logarithm is increasing, Expnentiating both sides gives \[ (a_1a_2\cdots a_n)^{1/n} \le \frac {a_1+a_2+\cdots +a_n}{n}. \]

Problem 6.

Source: Blitzstein and Hwang, Introduction to Probability, Chapter 10, Problem 7.

Let \(X\) and \(Y\) be i.i.d. positive random variables, and let \(c>0\). For each part below, fill in the appropriate equality or inequality symbol.

Write \(=\) if the two sides are always equal, \(\le \) if the left-hand side is less than or equal to the right-hand side but not necessarily equal, and similarly for \(\ge \). If no relation holds in general, write \(?\).

(a): \(E(\log X)\ \underline {\hspace {1cm}}\ \log (E X)\)
(b): \(E(X)\ \underline {\hspace {1cm}}\ \sqrt {E(X^2)}\)
(c): \(E(\sin ^2 X)+E(\cos ^2 X)\ \underline {\hspace {1cm}}\ 1\)
(d): \(E(|X|)\ \underline {\hspace {1cm}}\ \sqrt {E(X^2)}\)
(e): \(P(X>c)\ \underline {\hspace {1cm}}\ \dfrac {E(X^3)}{c^3}\)
(f): \(P(X\le Y)\ \underline {\hspace {1cm}}\ P(X\ge Y)\)
(g): \(E(XY)\ \underline {\hspace {1cm}}\ \sqrt {E(X^2)E(Y^2)}\)
(h): \(P(X+Y>10)\ \underline {\hspace {1cm}}\ P(X>5\text { or }Y>5)\)
(i): \(E(\min (X,Y))\ \underline {\hspace {1cm}}\ \min (EX,EY)\)
(j): \(E(X/Y)\ \underline {\hspace {1cm}}\ \dfrac {EX}{EY}\)
(k): \(E\!\left (X^2(X^2+1)\right )\ \underline {\hspace {1cm}}\ E\!\left (X^2(Y^2+1)\right )\)
(l): \(E\!\left (\dfrac {X^3}{X^3+Y^3}\right )\ \underline {\hspace {1cm}}\ E\!\left (\dfrac {Y^3}{X^3+Y^3}\right )\)

Solution

The answers are as follows. \[ \begin {array}{c|c|l} \text {Part} & \text {Symbol} & \text {Reason}\\ \hline (a) & \le & \log x\text { is concave, by Jensen.}\\ (b) & \le & (E X)^2\le E(X^2).\\ (c) & = & \sin ^2 X+\cos ^2 X=1\text { pointwise.}\\ (d) & \le & (E|X|)^2\le E(X^2).\\ (e) & \le & \text {Markov applied to }X^3.\\ (f) & = & \text {Symmetry of i.i.d. }X,Y.\\ (g) & \le & \text {Cauchy--Schwarz.}\\ (h) & \le & X+Y>10\text { implies }X>5\text { or }Y>5.\\ (i) & \le & \min (X,Y)\le X\text { and }\min (X,Y)\le Y.\\ (j) & ? & \text {No fixed relation holds in general.}\\ (k) & \ge & E(X^4)\ge (E X^2)^2.\\ (l) & = & \text {Symmetry, and the two fractions exchange when }X,Y\text { are swapped.} \end {array} \]

For part (k), independence gives \[ E\!\left (X^2(Y^2+1)\right )=(E X^2)^2+E(X^2), \] while the left-hand side is \(E(X^4)+E(X^2)\), so the comparison reduces to \(E(X^4)\ge (E X^2)^2\).

Problem 7.

Source: Blitzstein and Hwang, Introduction to Probability, Chapter 10, Problem 22.

Let \[ U_1,U_2,\dots ,U_{60} \] be i.i.d. \(\operatorname {Unif}(0,1)\) and let \[ X=U_1+U_2+\cdots +U_{60}. \]

(a): Which important distribution is the distribution of \(X\) very close to? Specify what the parameters are, and state which theorem justifies your choice.
(b): Give a simple but accurate approximation for \[ P(X>17). \] Justify briefly.

Solution

(a) For one \(\operatorname {Unif}(0,1)\) random variable, \[ E(U_j)=\frac 12, \qquad \Var (U_j)=\frac 1{12}. \] Therefore \[ E(X)=60\cdot \frac 12=30, \qquad \Var (X)=60\cdot \frac 1{12}=5. \] By the Central Limit Theorem, the distribution of \(X\) is very close to \[ N(30,5). \]

(b) Using the normal approximation, \[ P(X>17) \approx P\left (N(30,5)>17\right ) = P\left (Z>\frac {17-30}{\sqrt 5}\right ). \] The standardized value is \[ \frac {17-30}{\sqrt 5}\approx -5.81. \] Thus \[ P(X>17)\approx P(Z>-5.81)=\Phi (5.81), \] which is essentially \(1\).

Problem 8.

Source: Blitzstein and Hwang, Introduction to Probability, Chapter 10, Problem 25.

(a): Let \[ Y=e^X, \qquad X\sim \operatorname {Exp}(3). \] Find the mean and variance of \(Y\).
(b): For \(Y_1,\dots ,Y_n\) i.i.d. with the same distribution as \(Y\) from part (a), what is the approximate distribution of the sample mean \[ \bar Y_n=\frac 1n\sum _{j=1}^{n}Y_j \] when \(n\) is large?

Solution

(a) Since \(X\sim \operatorname {Exp}(3)\), its MGF is \[ M_X(t)=\frac {3}{3-t}, \qquad t<3. \] Because \(Y=e^X\), \[ E(Y)=E(e^X)=M_X(1)=\frac 32. \] Also \[ E(Y^2)=E(e^{2X})=M_X(2)=3. \] Therefore \[ \Var (Y)=E(Y^2)-[E(Y)]^2 =3-\frac 94 =\frac 34. \]

(b) By the Central Limit Theorem, when \(n\) is large, \[ \bar Y_n\approx N\left (E(Y),\frac {\Var (Y)}{n}\right ) = N\left (\frac 32,\frac {3}{4n}\right ). \]

Problem 9.

Source: Blitzstein and Hwang, Introduction to Probability, Chapter 10, Problem 27.

Consider i.i.d. \(\operatorname {Pois}(\lambda )\) random variables \[ X_1,X_2,\dots . \] The MGF of \(X_j\) is \[ M(t)=e^{\lambda (e^t-1)}. \]

(a): Find the MGF \(M_n(t)\) of the sample mean \[ \bar X_n=\frac 1n\sum _{j=1}^{n}X_j. \]
(b): Find the limit of \(M_n(t)\) as \(n\to \infty \). You can do this with almost no calculation using a relevant theorem, or you can use part (a) and the fact that \(e^x\approx 1+x\) if \(x\) is very small.

Solution

(a) We compute \[ M_n(t)=E\left (e^{t\bar X_n}\right ) =E\left (e^{(t/n)(X_1+\cdots +X_n)}\right ). \] By independence, \[ M_n(t)=\prod _{j=1}^n E\left (e^{(t/n)X_j}\right ) =\left [M\left (\frac tn\right )\right ]^n. \] Using the Poisson MGF, \[ M_n(t)=\left [e^{\lambda (e^{t/n}-1)}\right ]^n = \exp \left (n\lambda (e^{t/n}-1)\right ). \]

(b) As \(n\to \infty \), \[ e^{t/n}-1\sim \frac tn. \] Therefore \[ n\lambda (e^{t/n}-1)\to \lambda t. \] It follows that \[ M_n(t)\to e^{\lambda t}. \] This is the MGF of the constant random variable equal to \(\lambda \), which agrees with the Law of Large Numbers.