Final #1 - Questions and Solutions

Let X_{1}, X_{1}, ..., X_{n} be a random sample from a population with the following probability function:

P (X = k) = (\frac{1}{2})^{k−θ+1} ,  k = θ, θ + 1, θ + 2, ...  (1)

where θ is an unknown non-negative integer. (Hint: Notice the relation with geometric distribution.)

(a) Estimate θ by the method of moments.

Here and later it will be convenient to observe that, if X is distributed according to (1), then the random variable Y defined by

Y = X − (θ − 1)  (2)

is distributed G(\frac{1}{2}). Similarly, we will denote Y_{i} = X_{i} − (θ − 1) for 1 ≤ i ≤ n. Since E(Y) = 2, we have E(X) = 2 + θ − 1 = θ + 1. Thus, the required estimator is Θ̂_{1} = X̄_{n} − 1.

(b) Find a sufficient statistic for the parameter θ, of dimension as small as possible.

The joint probability function of the sample X_{1}, X_{2}, ..., X_{n} is

P (x_{1}, x_{2}, ..., x_{n}; θ) = (\frac{1}{2})^{∑_{i=1}^{n} x_{i}+n} · 2^{nθ}1_{\{θ,θ+1,...\}}( min_{1≤i≤n} x_{i}) . (3)

By the factorization theorem

S = min_{1≤i≤n} X_{i}  (4)

is a sufficient statistic for θ.

(c) Find the maximum likelihood estimator for θ.

The joint probability function (3) increases with θ, but only as long as θ ≤ min_{1≤i≤n} x_{i}. Hence S is the maximum likelihood estimator for θ.

(d) Find an unbiased estimator for θ.

According to (a), we may simply take Θ̂_{2} = X_{1} − 1. (Of course, Θ̂_{1} = X̄_{n} − 1 is also unbiased and, intuitively at least, should be much better than Θ̂_{2}.)

(e) Find an unbiased estimator for θ^{2}.

We have

E(X^{2}) = E ((Y + (θ − 1))^{2})
= E (((Y − 2) + (θ + 1))^{2})
= V (Y) + (θ + 1)^{2}
= 2 + (θ + 1)^{2}
= θ^{2} + 2θ + 3

and hence

E(X^{2} − 2X − 1) = θ^{2}.

It follows that T_{1} = X_{1}^{2} − 2X_{1} − 1 is an unbiased estimator for θ^{2}.

(f) Improve the estimator for θ^{2}, found in the preceding part, using part (b). (The improvement should be reflected by a smaller variance, while preserving the unbiasedness.)

It will be convenient to use the following lemma.

Lemma: Let V_{i} ∼ G(p), i = 1, 2, ..., n, be independent random variables. Then:

min_{1≤i≤n} V_{i} ∼ G(1 − q^{n}).  (5)

Proof: Since

P (V_{i} ≥ k) = p ∑_{j=k}^{∞} q^{j−1} = q^{k−1}, k = 1, 2, ...

we have

P ( min_{1≤i≤n} V_{i} ≥ k) = (P (V ≥ k))^{n} = (q^{n})^{k−1}, k = 1, 2, ...

which implies (5).

□

Denote S′ = min_{1≤i≤n} Y_{i}. By the lemma S′ ∼ G (1 − \frac{1}{2^{n}} ), and therefore

E(S′) = \frac{1}{1 − \frac{1}{2^{n}}} = \frac{2^{n}}{2^{n} − 1}

and

V (S′) = \frac{\frac{1}{2^{n}}}{(1 − \frac{1}{2^{n}})^{2}} = \frac{2^{n}}{(2^{n} − 1)^{2}} .

Consider the sufficient statistic S defined by (4). Obviously S = S′ + θ − 1, and therefore

E(S) = \frac{2^{n}}{2^{n} − 1} + θ − 1 = θ + \frac{1}{2^{n} − 1}

and

E(S^{2}) = V (S) + E^{2}(S) = V (S′) + E^{2}(S)

= \frac{2^{n}}{(2^{n} − 1)^{2}} + θ^{2} + \frac{2}{2^{n} − 1}θ + \frac{1}{(2^{n} − 1)^{2}}

= θ^{2} + \frac{2}{2^{n} − 1}θ + \frac{2^{n} + 1}{(2^{n} − 1)^{2}} .

Consequently:

E (S^{2} − \frac{2}{2^{n} − 1}S) = θ^{2} + \frac{2^{n} + 1}{(2^{n} − 1)^{2}} − \frac{2}{2^{n} − 1} · \frac{1}{2^{n} − 1} = θ^{2} + \frac{1}{2^{n} − 1}.

Thus the required estimator for θ^{2} is:

T_{2} = S^{2} − \frac{2}{2^{n} − 1}S − \frac{1}{2^{n} − 1}.

(g) Without performing any explicit calculations, opine about the relevance of the Cramér-Rao lower bound to our case in general and to the estimator found in the preceding part in particular.

Notice that the regularity conditions do not make sense in this case, as the parameter θ is discrete, so that we cannot differentiate with respect to it. Even if this were remedied (say, by letting θ vary over all reals), the family would not satisfy the reqularity conditions, since the support of the distribution changes with θ. In fact, the perfomance of the estimator we found is much superior to what we may have expected according to Cramér-Rao’s bound. Whereas the bound is inversely proportional to n, the actual variance of T_{2} decreases exponentially as a function of n.

(h) Find a confidence interval of confidence level 1 − α (approximately) for θ based on X̄_{n} for large n.

Since E(X̄_{n}) = 1 + θ and V (X̄_{n}) = \frac{2}{n}, we obtain

P (−z_{α/2} ≤ \frac{(X̄_{n} − 1 − θ)\sqrt{n}}{\sqrt{2}} ≤ z_{α/2}) = 1 − α.

Thus the confidence interval is:

I = [X̄_{n} − 1 − z_{α/2}\sqrt{\frac{2}{n}}, X̄_{n} − 1 + z_{α/2}\sqrt{\frac{2}{n}}] .

(i) Find the confidence level of the following confidence interval for θ:

I′ = [ min_{1≤i≤n} X_{i} − \frac{1}{2}, min_{1≤i≤n} X_{i} + \frac{1}{2}] .

The significance level of I′ is:

\begin{aligned}P (θ ∈ I′) & = P (θ − \frac{1}{2} ≤ S ≤ θ + \frac{1}{2}) \\ & = P (S = θ) = P (S′ = 1) = 1 − \frac{1}{2^{n}} . \end{aligned}

(j) Suggest a test with significance level \frac{1}{16} based on S_{4} = min_{1≤i≤4} X_{i} (n=4) to

test the hypothesis

\begin{matrix}H_{0} : & θ = 5, \\ H_{1} : & θ > 5, \end{matrix}

whose power is maximal. Calculate the power function.

It is natural to consider a critical region of the form R = {S_{4} > a}, where a > 0. We may assume a to be an integer, a ≥ 4. To find the value of a, write:

\frac{1}{16} = P_{θ=5}(S_{4} > a) = P_{θ=5} (S′_{4} > a − 5 + 1) = (\frac{1}{2^{4}} )^{a−4} .

This yields a = 5.

To see that the power is the maximum possible, calculate:

1 − β = P_{θ>5}(S_{4} > 5) = 1.

(k) To test the above hypothesis the following test was used: R = {X̄_{4} ≥ 5.5} (again n = 4). Find its significance level and power. Can you suggest a test based on X̄_{4} which will have a smaller significance level (and the same power) or a bigger power (and the same significance level)?

We start with by calculating the significance level:

α = P_{θ=5}(X̄_{4} ≥ 5.5) = P_{θ=5} (∑_{i=1}^{4} X_{i} ≥ 22)

= 1 − P_{θ=5}(∑_{i=1}^{4} X_{i} < 22)

= 1 − P_{θ=5}(X_{1} = X_{2} = X_{3} = X_{4} = 5)

− 4P_{θ=5}(X_{1} = X_{2} = X_{3} = 5, X_{4} = 6)

= 1 − \frac{1}{2^{4}} − 4 · \frac{1}{2^{3}} · \frac{1}{4} = 1 − \frac{3}{16} = \frac{13}{16}.

Clearly, as for θ > 0 the X′_{i}s assume values from 6 and above, the power of the test is identically 1. By the same reasoning, the test R′ = {X̄_{4} ≥ 6} still has power 1, but its significance level may be

shown to decrease to \frac{17}{32}.

(l) A factory which produces motors claims that the life length of motors measured in years (rounded to the nearest integer) is distributed as in (1) with θ = 2. To test the claim, 12 motors were selected. Out of this sample, 4 lasted 2 years, 6 lasted 3 years and 2 lasted 4 years. Test the factory’s claim against the alternative that the distribution is different (not necessarily from the same family). Use a 0.1 level of significance.

To test the claim we use the chi-square goodness-of-fit test. Denote by X the life length of a random unit. Consider 3 regions:

X = 2 X = 3 X ≥ 4
e_{j} 12 · \frac{1}{2} 12 · \frac{1}{4} 12 · \frac{1}{4}
o_{j}  4  6  2

Calculating the value of X^{2} for our sample

x^{2} = \frac{(6 − 4)^{2}}{6} + \frac{(3 − 6)^{2}}{3} + \frac{(3 − 2)^{2}}{3} = 4

and comparing it with χ_{0.1}^{2}(2) = 9.21, we conclude that we can accept the claim.

Note that for more percise calculation we should divide the support of X into two regions, such that for each one e_{j} ≥ 5:

X = 2 X > 2
e_{j} 12 · \frac{1}{2} 12 · \frac{1}{2}
o_{j}  4  8

Again, calculating the value of X^{2}

x^{2} = \frac{(6 − 4)^{2}}{6} + \frac{(6 − 8)^{2}}{6} = \frac{4}{3}

and comparing it with χ_{0.1}^{2}(1) = 6.635, we conclude that we can accept the claim.