Sampling Error and Confidence Intervals

In the 1997 National Household Survey on Drug Abuse (NHSDA), sampling error occurs due to the random process of sampling the total population of inferential interest (i.e., the civilian, noninstitutionalized population aged 12 or older of the United States). Other NHSDA reports have used 95% confidence intervals to quantify sampling error. Because the estimates in the NHSDA are frequently small percentages, the confidence intervals are based on logit transformations. Logit transformations yield asymmetric interval boundaries that provide a more suitable measure of sampling error for small percentages.

To illustrate, let the proportion P_d represent the true prevalence rate for a particular analysis domain "d." Then the logit transformation of P_d, commonly referred to as the "log odds," is defined as

where "ln" denotes the natural logarithm.

Letting p_d be the estimate of the proportion, the log odds estimate becomes

Then the lower and upper limits of L are calculated as
where var(p_d) is the variance estimate of p_d, and K is the constant chosen to yield the proper level of confidence (e.g., K = 1.96 for 95% confidence limits).

Applying the inverse logit transformation to A and B above yields a confidence interval for p_d as follows:

where "exp" denotes the inverse log transformation. The upper and lower limits of the percentage estimate are obtained by simply multiplying the upper and lower limits of p by 100.

Corresponding to the percentage estimates, the number of drug users, Y_d, can be estimated as

where

= estimated population total for domain d and
= estimated proportion for domain d.

The confidence interval foris obtained by multiplying the lower and upper limits of the proportion confidence interval by. In addition, the variance ofcan be estimated as

For the 1997 NHSDA, the design-based variance was estimated using a Taylor series linearization. For a given variance estimate, the associated design effect is the ratio of the design-based variance estimate over the variance that would have been obtained from a simple random sample of the same size. Because the combined design features of stratification, clustering, and unequal weighting are expected to increase the variance estimates, the design effect should virtually always be greater than one. For prevalence rates near zero, however, the variance inflating effects of unequal weighting and clustering were sometimes underestimated, resulting in design effects of less than one. Because the corresponding variance estimates were considered anomalously small, two other variance estimates were computed as quality control measures. The first was based only on the stratification and unequal weighting effects, and the second was based on no effects or simple random sampling. The reported variance estimate was then the maximum of these three estimates.

As in other publications, estimates with low precision were not reported. The criterion used for suppressing estimates was based on the relative standard error (RSE). The RSE is defined as the ratio of the standard error (SE) of the estimate over the estimate. For the 1997 NHSDA reports, the log transformation of the proportion estimate was used to calculate the RSE. Specifically, percentages and corresponding population estimates were suppressed if


or

For computational purposes, this is equivalent to


or

where SE(p) equals the standard error estimate of p. The log transformation of p is used to provide a more balanced treatment of measuring the quality of small, large, and intermediate p values. The switch to (1-p) for p greater than 0.5 provides a symmetric suppression rule across the range of possible p values.

32 This approach treats the estimated domain size, N hat sub d, as fixed. This is a good approximation in most cases because ratio adjustments tend to control the variability of estimates of domain sizes.

This page was last updated on December 30, 2008.