The Jackknife Estimation Method Avery I. McIntosh
1 Introduction
Statistical resampling methods have become feasible for parametric estimation, hypothesis testing,
and model validation now that the computer is a ubiquitous tool for statisticians. This essay focuses
on the resampling technique for parametric estimation known as the Jackknife procedure. To outline
the usefulness of the method and its place in the general class of statistical resampling techniques,
I will quickly delineate two similar resampling methods: the bootstrap and the permutation test.
1.1 Other Sampling Methods: The Bootstrap
The bootstrap is a broad class of usually non-parametric resampling methods for estimating the
sampling distribution of an estimator. The method was described in 1979 by Bradley Efron, and
was inspired by the previous success of the Jackknife procedure.
1
Imagine that a sample of n independent, identically distributed observations from an unknown
distribution have been gathered, and a mean of the sample,
¯
Y , has been calculated. To make
inferences about the population mean we need to know the variability of the sample mean, which
we know from basic statistical theory is V[
¯
Y ] = V[Y ]/n. Here, since the distribution is unknown,
we do not know the value of V[Y ] = σ
2
. The central limit theorem (CLT) states that the stan-
dardized sample mean converges in distribution to a standard normal Z as the sample size grows
large—and we can invoke Slutsky’s theorem to demonstrate that the sample standard deviation is
an adequate estimator for standard deviation σ when the distribution is unknown. However, for
other statistics of interest that do not admit the CLT, and for small sample sizes, the bootstrap is
a viable alternative.
Briefly, the bootstrap method specifies that B samples be generated from the data by sampling
with replacement from the original sample, with each sample set being of identical size as the
original sample (here, n). The larger B is, the closer the set of samples will be to the ideal exact
bootstrap sample, which is of the order of an n-dimensional simplex: |C
n
| = (2n − 1)C(n). The
computation of this number, never mind the actual sample, is generally unfeasible for all but the
smallest sample sizes (for example a sample size of 12 has about 1.3 million with-replacement
subsamples). Furthermore, the bootstrap follows a multinomial distribution, and the most likely
sample is in fact the original sample, hence it is almost certain that there will be random bootstrap
samples that are replicates of the original sample. This means that the computation of the exact
bootstrap is all but impossible in practice. However, Efron and Tibshirani have argued that in
some instances, as few as 25 bootstrap samples can be large enough to form a reliable estimate.
2
The next step in the process is to perform the action that derived the initial statistic—here the
mean: so we sum each bootstrap sample and divide the total by n, and use those quantities to
generate an estimate of the variance of
¯
Y as follows:
SE(
¯
Y )
B
=
n
1
B
B
X
b=1
(
¯
Y
b
−
¯
Y )
2
o
1/2
1
剩余10页未读，继续阅读
资源评论
Lee_gaeul
- 粉丝: 26
- 资源: 2
