Statistical Models F 29
It is often sufficient to describe the average behavior of a process, rather than each particular
realization. For example, a regression model might be developed to relate plant growth to
nutrient availability. The explicit aim of the model might be to describe how the average
growth changes with nutrient availability, not to predict the growth of an individual plant.
The support for the notion of averaging in a model lies in the nature of expected values,
describing typical behavior in the presence of randomness. This, in turn, requires that the
model contain stochastic components.
The defining characteristic of statistical models is their dependence on parameters and the incorpo-
ration of stochastic terms. The properties of the model and the properties of quantities derived from
it must be studied in a long-run, average sense through expectations, variances, and covariances.
The fact that the parameters of the model must be estimated from the data introduces a stochastic
element in applying a statistical model: because the model is not deterministic but includes random-
ness, parameters and related quantities derived from the model are likewise random. The properties
of parameter estimators can often be described only in an asymptotic sense, imagining that some
aspect of the data increases without bound (for example, the number of observations or the number
of groups).
The process of estimating the parameters in a statistical model based on your data is called fitting
the model. For many classes of statistical models there are a number of procedures in SAS/STAT
software that can perform the fitting. In many cases, different procedures solve identical estima-
tion problems—that is, their parameter estimates are identical. In some cases, the same model
parameters are estimated by different statistical principles, such as least squares versus maximum
likelihood estimation. Parameter estimates obtained by different methods typically have differ-
ent statistical properties—distribution, variance, bias, and so on. The choice between competing
estimation principles is often made on the basis of properties of the estimators. Distinguishing
properties might include (but are not necessarily limited to) computational ease, interprative ease,
bias, variance, mean squared error, and consistency.
Model-Based and Design-Based Randomness
A statistical model is a description of the data-generating mechanism, not a description of the spe-
cific data to which it is applied. The aim of a model is to capture those aspects of a phenomenon
that are relevant to inquiry and to explain how the data could have come about as a realization of
a random experiment. These relevant aspects might include the genesis of the randomness and the
stochastic effects in the phenomenon under study. Different schools of thought can lead to different
model formulations, different analytic strategies, and different results. Coarsely, you can distinguish
between a viewpoint of innate randomness and one of induced randomness. This distinction leads
to model-based and design-based inference approaches.
In a design-based inference framework, the random variation in the observed data is induced by
random selection or random assignment. Consider the case of a survey sample from a finite popula-
tion of size N ; suppose that F
N
D fy
i
W i 2 U
N
g denotes the finite set of possible values and U
N
is the index set U
N
D f1; 2; : : : ; N g. Then a sample S, a subset of U
N
, is selected by probability
rules. The realization of the random experiment is the selection of a particular set S; the associated
values selected from F
N
are considered fixed. If properties of a design-based sampling estimator
