Let \(T\) denote the total number of tosses required until both Heads and Tails have appeared at least once.

After the first toss, exactly one of the two outcomes has appeared. Define \[ X=T-1, \] so that \(X\) is the number of additional tosses needed to observe the outcome opposite to the first toss.

Conditional on the outcome of the first toss, each subsequent toss produces the opposite outcome with probability \(1/2\), independently of all previous tosses. Therefore, \[ X\sim \operatorname {Geom}\left (\frac 12\right ), \] where \(X\) counts the number of trials up to and including the first success. Hence \[ E(X)=\frac {1}{1/2}=2. \]

Since \[ T=1+X, \] linearity of expectation gives \[ E(T)=1+E(X)=1+2=3. \]

(a) By LOTUS and the Poisson PMF, \[ E\bigl (Xg(X)\bigr ) = \sum _{x=0}^{\infty }xg(x)e^{-\lambda }\frac {\lambda ^x}{x!}. \] The term for \(x=0\) is zero, so \[ E\bigl (Xg(X)\bigr ) = \sum _{x=1}^{\infty }g(x)e^{-\lambda }\frac {\lambda ^x}{(x-1)!}. \] Letting \(y=x-1\), we get \[ E\bigl (Xg(X)\bigr ) = \lambda \sum _{y=0}^{\infty }g(y+1)e^{-\lambda }\frac {\lambda ^y}{y!} = \lambda E\bigl (g(X+1)\bigr ). \]

(b) To find \(E(X^3)\), take \(g(x)=x^2\). Then the identity gives \[ E(X^3)=\lambda E\bigl ((X+1)^2\bigr ). \] Expanding the square, \[ E(X^3)=\lambda \{E(X^2)+2E(X)+1\}. \] For a Poisson random variable, \(E(X)=\lambda \) and \(\operatorname {Var}(X)=\lambda \), so \[ E(X^2)=\operatorname {Var}(X)+\{E(X)\}^2=\lambda +\lambda ^2. \] Therefore \[ E(X^3)=\lambda (\lambda ^2+\lambda +2\lambda +1) = \lambda ^3+3\lambda ^2+\lambda . \]

(a) The population values \(y_1,\dots ,y_N\) are fixed characteristics of the population. Once the population is specified, these values do not depend on which sample is selected. Therefore, under the sampling procedure, they are constants rather than random variables.

By contrast, \(W_r\) depends on which population member appears in sample position \(r\), and \(I_j\) depends on whether person \(j\) is selected. Therefore, their values can vary across possible random samples. Since \(\bar W\) is computed from the random observations \(W_1,\dots ,W_n\), it is also a random variable.

(b) By symmetry, every population member is equally likely to appear in sample position \(r\). Therefore, \[ P(W_r=y_j)=\frac {1}{N}, \qquad j=1,\dots ,N. \] By LOTUS, \[ E(W_r) = \sum _{j=1}^N y_jP(W_r=y_j) = \frac {1}{N}\sum _{j=1}^N y_j = \bar y. \] Using linearity of expectation, \begin {align*} E(\bar W) &= E\left (\frac {1}{n}\sum _{r=1}^n W_r\right )\\ &= \frac {1}{n}\sum _{r=1}^n E(W_r)\\ &= \frac {1}{n}\sum _{r=1}^n \bar y\\ &= \bar y. \end {align*}

Thus, the sample mean is an unbiased estimator of the population mean.

(c) For each \(j\), the product \(I_jy_j\) equals \(y_j\) if person \(j\) is selected and equals \(0\) otherwise. Therefore, \[ \sum _{j=1}^N I_jy_j \] is exactly the sum of the values belonging to the sampled people. Hence \[ \sum _{r=1}^n W_r = \sum _{j=1}^N I_jy_j, \] and therefore \[ \bar W = \frac {1}{n}\sum _{j=1}^N I_jy_j. \]

Every population member has the same probability of being included in a simple random sample of size \(n\). Thus, \[ P(I_j=1) = \frac {\binom {N-1}{n-1}}{\binom {N}{n}} = \frac {n}{N}. \] Since \(I_j\) is an indicator, \[ E(I_j)=P(I_j=1)=\frac {n}{N}. \] Consequently, \begin {align*} E(\bar W) &= E\left (\frac {1}{n}\sum _{j=1}^N I_jy_j\right )\\ &= \frac {1}{n}\sum _{j=1}^N y_jE(I_j)\\ &= \frac {1}{n}\sum _{j=1}^N y_j\frac {n}{N}\\ &= \frac {1}{N}\sum _{j=1}^N y_j\\ &= \bar y. \end {align*}

(d) Every possible sample contains exactly \(n\) people. Since \(I_j=1\) precisely when person \(j\) is selected, the number of selected people is \[ \sum _{j=1}^N I_j. \] Therefore, \[ \sum _{j=1}^N I_j=n \] for every possible sample, and hence with probability \(1\).

This does not imply that the individual indicators are constants. For example, one sample may have \(I_1=1\), while another may have \(I_1=0\). The indicators vary across samples, but they vary in a coordinated way so that their sum is always \(n\).

(e) Since \(I_j\) is Bernoulli with success probability \(n/N\), \[ \operatorname {Var}(I_j) = \frac {n}{N}\left (1-\frac {n}{N}\right ) = \frac {n(N-n)}{N^2}. \]

To find the variance of the sample mean, begin with \[ \bar W-\bar y = \frac {1}{n}\sum _{j=1}^N I_jy_j-\bar y. \] Since \[ \sum _{j=1}^N I_j=n \] for every possible sample, we may write \[ \bar y = \frac {1}{n}\sum _{j=1}^N I_j\bar y. \] Therefore, \[ \bar W-\bar y = \frac {1}{n}\sum _{j=1}^N I_j(y_j-\bar y). \]

Because \(E(\bar W)=\bar y\), \[ \operatorname {Var}(\bar W) = E\bigl [(\bar W-\bar y)^2\bigr ]. \] Thus, \begin {align*} \operatorname {Var}(\bar W) &= \frac {1}{n^2} E\left [ \left ( \sum _{j=1}^N I_j(y_j-\bar y) \right )^2 \right ]\\ &= \frac {1}{n^2} E\left [ \sum _{j=1}^N I_j^2(y_j-\bar y)^2 + 2\sum _{1\leq i<j\leq N} I_iI_j(y_i-\bar y)(y_j-\bar y) \right ]. \end {align*}

Since \(I_j^2=I_j\), \[ E(I_j^2)=E(I_j)=\frac {n}{N}. \] Also, for \(i\neq j\), \[ E(I_iI_j) = P(I_i=1 \text { and } I_j=1) = \frac {\binom {N-2}{n-2}}{\binom {N}{n}} = \frac {n(n-1)}{N(N-1)}. \] Hence \begin {align*} \operatorname {Var}(\bar W) &= \frac {1}{n^2} \left [ \frac {n}{N} \sum _{j=1}^N (y_j-\bar y)^2 + \frac {2n(n-1)}{N(N-1)} \sum _{i<j}(y_i-\bar y)(y_j-\bar y) \right ]. \end {align*}

Now, \[ \sum _{j=1}^N (y_j-\bar y)=0. \] Squaring this identity gives \[ 0 = \sum _{j=1}^N (y_j-\bar y)^2 + 2\sum _{i<j}(y_i-\bar y)(y_j-\bar y). \] Therefore, \[ 2\sum _{i<j}(y_i-\bar y)(y_j-\bar y) = -\sum _{j=1}^N (y_j-\bar y)^2. \]

Substituting, \begin {align*} \operatorname {Var}(\bar W) &= \frac {1}{n^2} \left [ \frac {n}{N} - \frac {n(n-1)}{N(N-1)} \right ] \sum _{j=1}^N (y_j-\bar y)^2\\ &= \frac {1}{n^2} \frac {n(N-n)}{N(N-1)} \sum _{j=1}^N (y_j-\bar y)^2. \end {align*}

Since \[ S_y^2 = \frac {1}{N-1} \sum _{j=1}^N (y_j-\bar y)^2, \] we conclude that \[ \operatorname {Var}(\bar W) = \frac {N-n}{Nn}S_y^2. \] Equivalently, \[ \operatorname {Var}(\bar W) = \left (1-\frac {n}{N}\right )\frac {S_y^2}{n}. \]

(f)

(i)

If \(n=1\), then \[ \operatorname {Var}(\bar W) = \frac {N-1}{N}S_y^2. \] In this case, the sample mean consists of only one observation and generally has substantial variability.

(ii)

If \(n=N\), then \[ \operatorname {Var}(\bar W)=0. \] The entire population is observed, so \[ \bar W=\bar y \] for every possible sample.

(iii)

If all population values are equal, then \[ S_y^2=0, \] and hence \[ \operatorname {Var}(\bar W)=0. \] Every possible sample has the same sample mean.

(iv)

As \(n\) increases, both \(1/n\) and the finite-population correction \[ 1-\frac {n}{N} \] decrease. Therefore, the variance of the sample mean decreases as a larger fraction of the population is observed.

(g) Under sampling with replacement, the observations \[ W_1,\dots ,W_n \] are independent and identically distributed, with \[ P(W_r=y_j)=\frac {1}{N}, \qquad j=1,\dots ,N. \] Therefore, \[ E(W_r)=\bar y. \] Also, \begin {align*} \operatorname {Var}(W_r) &= E\bigl [(W_r-\bar y)^2\bigr ]\\ &= \frac {1}{N}\sum _{j=1}^N (y_j-\bar y)^2\\ &= \frac {N-1}{N}S_y^2. \end {align*}

Since the observations are independent, \begin {align*} \operatorname {Var}(\bar W_{\mathrm {rep}}) &= \operatorname {Var}\left ( \frac {1}{n}\sum _{r=1}^n W_r \right )\\ &= \frac {1}{n^2} \sum _{r=1}^n\operatorname {Var}(W_r)\\ &= \frac {1}{n}\frac {N-1}{N}S_y^2. \end {align*}

Under sampling without replacement, \[ \operatorname {Var}(\bar W) = \frac {N-n}{Nn}S_y^2. \] Since \[ N-n\leq N-1, \] we have \[ \operatorname {Var}(\bar W) \leq \operatorname {Var}(\bar W_{\mathrm {rep}}). \] In fact, when \(n>1\), the inequality is strict unless \(S_y^2=0\).

Sampling without replacement is more precise because it avoids duplicate observations. For a fixed sample size, it therefore gathers information from more distinct population members.

(h) Let \(\theta \) be the parameter being estimated. Write \[ T-\theta = \bigl (T-E(T)\bigr ) + \bigl (E(T)-\theta \bigr ). \] Squaring both sides gives \begin {align*} (T-\theta )^2 &= \bigl (T-E(T)\bigr )^2\\ &\quad + 2\bigl (T-E(T)\bigr )\bigl (E(T)-\theta \bigr )\\ &\quad + \bigl (E(T)-\theta \bigr )^2. \end {align*}

Taking expectations, \begin {align*} E\bigl [(T-\theta )^2\bigr ] &= E\bigl [(T-E(T))^2\bigr ]\\ &\quad + 2\bigl (E(T)-\theta \bigr ) E\bigl [T-E(T)\bigr ]\\ &\quad + \bigl (E(T)-\theta \bigr )^2. \end {align*}

Since \[ E\bigl [T-E(T)\bigr ]=0, \] the middle term vanishes. Hence \[ \operatorname {MSE}(T) = \operatorname {Var}(T) + \operatorname {Bias}(T)^2. \]

For the sample mean, \[ \operatorname {Bias}(\bar W) = E(\bar W)-\bar y = 0. \] Therefore, \[ \operatorname {MSE}(\bar W) = \operatorname {Var}(\bar W) = \frac {N-n}{Nn}S_y^2. \]

(i) The statement is false. Unbiasedness is a statement about the average value of an estimator over all possible random samples, not about the value obtained from every individual sample.

A particular sample may contain an unusually large number of people with high values, in which case \[ \bar W>\bar y. \] Another sample may contain an unusually large number of people with low values, in which case \[ \bar W<\bar y. \]

The equality \[ E(\bar W)=\bar y \] means that if the sampling procedure were repeated many times, the average of the resulting sample means would equal the population mean. It does not imply that \[ \bar W=\bar y \] for every possible sample.