Skip To Content

Table Of Contents

Click for DHHS Home Page
Click for the SAMHSA Home Page
Click for the OAS Drug Abuse Statistics Home Page
Click for What's New
Click for Recent Reports and HighlightsClick for Information by Topic Click for OAS Data Systems and more Pubs Click for Data on Specific Drugs of Use Click for Short Reports and Facts Click for Frequently Asked Questions Click for Publications Click to send OAS Comments, Questions and Requests Click for OAS Home Page Click for Substance Abuse and Mental Health Services Administration Home Page Click to Search Our Site

Computer Assisted Interviewing for SAMHSA's National Household Survey on Drug Abuse

6. Comparisons of CAPI/ACASI to PAPI for Selected Outcomes: 1997 Field Experiment

In this chapter, we compare, for selected outcomes, the computer-assisted interviewing (CAI) procedures with the paper-and-pencil interview (PAPI) procedures. We examine the effects for the different modes of interview on the following topics:

  1. the reported prevalence of alcohol, marijuana, cocaine, and any illicit drug use;

  2. the reported prevalence of nonmedical use of psychotherapeutic drugs;

  3. answers to questions on mental health, which were interviewer administered in the 1997 NHSDA and self-administered via audio computer-assisted self-interviewing (ACASI) in the 1997 field experiment;

  4. answers to questions on methods of use, risk and availability of drugs, and drug dependence; and

  5. the reporting of 12-month frequency of use.

We also discuss the respondent and interviewer assessments of the interviewing environment.

6.1 Alcohol, Marijuana, Cocaine/Crack, and Any Illicit Drug Use in Lifetime, Past Year, and Past Month

Exhibit 6.1.1 contains the weighted prevalence of tobacco, alcohol, marijuana, cocaine, and any illicit drug use averaged across the ACASI treatment groups and for the 1997 NHSDA Quarter 4 comparison group. 12  Exhibit 6.1.2 presents the prevalence estimates by treatment level of the three experimental factors.

To compare ACASI treatments to PAPI, we fit a model with the main effects of the treatment group and the covariates. Levels 1-8 represented the eight ACASI experimental combinations used in the study, and Level 9 represented PAPI (the covariates in the model were gender, age, race/ethnicity, and education):

The following single degree-of-freedom contrasts were constructed from the nine-level treatment effects and evaluated as follows:

Because there are no interaction effects in this model, all contrasts were automatically adjusted for other covariates present in the model. Wald chi-square tests were used to evaluate each of these effects.

The logistic regression was used because response variable Y takes on the values 0 (nonuser) or 1 (drug user). The set of treatments and covariates is denoted by X and includes the treatment groups and the covariates. In the logistic regression model, p = prob (Y = 1 | x) is the response probability that a respondent has used the drug, given his or her set of covariates X.

The logit form of the model is formally stated as follows:






The expression represents the linear combination of covariate effects for each respondent, where β is the set of regression coefficients to be estimated and q is the number of covariates in the model. The expression represents the odds that a particular respondent has used the drug. The model response, log , is referred to as the log odds or the logit of p. Exponentiating the regression coefficients provides estimates of the odds ratio for different values of X, such as for respondents in one of the ACASI treatment cells versus the control (PAPI) group. These statistics quantify the strength of the treatment effect.

Exhibit 6.1.3 summarizes the modeling results for the overall comparison of ACASI to PAPI for the total sample and 12 to 17 year olds. In conducting these analyses, we classified the 12 to 17 year olds as less than high school graduates in order for all respondents to be included in the overall analysis. A total of 5,087 respondents were included in the analysis of the total sample, and 2,096 respondents were included in the analysis of 12 to 17 year olds. There were no missing values in this analysis because the privacy covariates were not included in the model (privacy information was unavailable for the PAPI group). Covariates in the model for the total sample analyses included gender, age (12 to 17 vs. 18+), race (Hispanic; Non-Hispanic, All Other Races; Non-Hisp., Black), and education (less than high school, high school, more than high school). Covariates in the model for the 12 to 17 year olds included gender and race only.

For the overall sample, there were only two significant differences: lifetime cocaine use and past year use of any illicit drug. In each case, ACASI yielded higher prevalence estimates, and the odds ratio was greater than 1.5. Among the youths (12 to 17 year olds), there were seven significant differences observed. Significantly higher reports of prevalence under ACASI were observed for lifetime cigarette use (OR=1.43), lifetime alcohol use (OR=1.35), lifetime marijuana use (OR=1.36), and past year marijuana use (OR=1.35). Also, for the summary variable, any illicit drug, ACASI yielded significantly higher reports for all three reporting periods: lifetime (OR=1.76), past year (OR=1.76), and past month (OR=1.57) use.

In addition to these analyses, for the total sample we also compared multiple gate versus PAPI, single gate versus PAPI, multiple chance present versus PAPI, multiple chance absent versus PAPI, consistency checks present versus PAPI, and consistency checks absent versus PAPI. Exhibit 6.1.4 presents the results, and Exhibit 6.1.5 summarizes the comparisons that were either significant or marginally significant. Significantly higher reports under PAPI were observed in only one case. The reported prevalence of past month cigarette use by the total sample was significantly reduced in the absence of multiple chances to report use compared to PAPI (29.8% PAPI vs. 21.2% multiple chance absent).

The reported prevalence of past year alcohol use by the total sample was marginally increased (0.05 < p < 0.10) in the presence of multiple chances to report use and in the presence of consistency checks compared to PAPI. Prevalence rates ranged from 66% for PAPI to 72.8% and 73.3% for the presence of multiple chance and consistency checks, respectively.

Total sample analyses of cocaine use showed that reported prevalence of lifetime use was significantly increased for overall ACASI versus PAPI (13.6% vs. 9.0% for ACASI vs. PAPI, respectively). Reported prevalence of lifetime use was also significantly increased in the single gate version, presence of multiple chances, and presence of consistency checks versus PAPI. Prevalence rates ranged from 9.0% for PAPI versus 17.8% for the single gate version, 16.4% for the presence of multiple chances, and 17.2% for the presence of consistency checks. Reported prevalence of past year use was significantly increased in the single gate questions versus PAPI group, with prevalences of 3.5% for single gate and 1.5% for PAPI.

Total sample analyses of any illicit drug use revealed that reported prevalence of lifetime use was marginally increased (0.05 < p < 0.10) for overall ACASI compared to PAPI (43.5% for ACASI vs. 37.2% for PAPI). Also for lifetime use, prevalence was marginally increased in the single gate treatment compared to PAPI and in the presence of multiple chances compared to PAPI (prevalence rates were just over 46% for the two ACASI classifications and 37.2% for PAPI). Reported prevalence of past year use was significantly increased for overall ACASI compared to PAPI (14.8% for ACASI vs. 10.0% for PAPI). Also for past year use, prevalence was significantly increased in the single gate versus PAPI in the absence of multiple chances versus PAPI and in the presence of consistency checks versus PAPI. Prevalence was marginally increased in the absence of consistency checks versus PAPI (16.6% for single gate, 14.3% absence of multiple chances, 14.1% for absence and presence of consistency checks, 10.0% PAPI).

Exhibit 6.1.1 Prevalence by Mode of Interview and Age of Respondent--Weighted and Edited Estimates

Reporting Period

Overall 12 to 17 Year Olds 18+ Year Olds
ACASI PAPI ACASI PAPI ACASI PAPI
Cigarettes
Lifetime Use 72.0 71.0 44.5 36.1 75.2 74.9
Past Year Use 31.2 33.5 22.9 25.2 32.2 34.5
Past Month Use 26.2 30.5 16.1 18.6 27.4 31.8

Alcohol

Lifetime Use 83.2 82.2 44.9 38.1 87.6 87.3
Past Year Use 68.3 65.0 36.5 32.7 72.0 68.7
Past Month Use 48.7 52.2 17.0 18.8 52.4 56.1

Marijuana

Lifetime Use 35.2 35.0 20.6 16.1 36.9 37.2
Past Year Use 10.2 9.3 16.7 13.0 9.5 8.9
Past Month Use 4.7 5.3 8.4 7.3 4.3 5.0

Cocaine

Lifetime Use 14.9 10.0 2.9 3.1 16.3 10.8
Past Year Use 2.6 1.6 1.8 2.5 2.7 1.5
Past Month Use 0.6 0.6 0.7 1.1 0.6 0.5

Any Illicit Drug1

Lifetime Use 43.7 18.8 31.2 20.5 45.2 39.5
Past Year Use 15.2 10.3 23.0 14.5 14.2 9.8
Past Month Use 5.9 5.4 12.0 8.0 5.2 5.1

1Any illicit drug includes marijuana, cocaine, crack, heroin, inhalants, hallucinogens, and nonmedical use of analgesics, sedatives, stimulants, and tranquilizers.

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

Exhibit 6.1.2 Prevalence Estimates for ACASI Experimental Factors: Overall and by Age
  Overall 18+ Years Old 12 to 17 Years Old
  Lifetime 12 Month 30 Day Lifetime 12 Month 30 Day Lifetime 12 Month 30 Day

Cigarettes

Single gate questions 74.2 29.2 26.6 77.9 30.0 27.8 43.2 22.7 16.6
Multiple gate questions 69.4 33.5 25.7 72.0 34.6 26.9 46.0 23.0 15.5
Multiple use questions absent 73.9 29.2 21.7 76.8 29.4 22.2 47.4 27.4 17.3
Multiple use questions present 70.0 33.3 31.0 73.4 35.2 33.0 41.7 18.5 15.0
Consistency checks absent 69.7 28.6 22.9 72.6 29.1 23.8 45.5 23.8 15.5
Consistency checks/present 74.5 34.1 29.9 78.0 35.5 31.3 43.3 21.8 16.8

Alcohol

Single gate questions 82.0 65.1 47.3 86.4 68.5 50.9 44.9 36.4 17.4
Multiple gate questions 84.5 72.0 50.3 89.0 76.0 54.1 44.9 36.5 16.6
Multiple use questions absent 83.5 66.2 50.2 87.5 69.3 53.7 46.9 37.7 18.0
Multiple use questions present 82.7 70.6 47.0 87.7 75.0 50.9 43.0 35.3 16.1
Consistency checks absent 82.1 65.1 44.8 86.5 68.5 48.3 45.6 37.0 16.2
Consistency checks/present 84.3 71.8 53.0 88.8 75.8 56.9 44.1 35.8 18.1

Marijuana

Single gate questions 35.4 12.5 5.5 36.9 11.9 5.1 22.1 17.6 9.4
Multiple gate questions 35.0 7.6 3.8 36.8 6.7 3.4 18.7 15.6 7.1
Multiple use questions absent 33.4 9.1 5.3 34.6 8.3 4.9 21.8 17.0 9.8
Multiple use questions present 37.1 11.4 4.0 39.3 10.8 3.7 19.4 16.4 7.0
Consistency checks absent 35.0 10.0 3.7 36.8 9.2 3.2 20.7 16.2 7.7
Consistency checks/present 35.3 10.6 5.8 37.0 9.8 5.5 20.4 17.3 9.1

Cocaine

Single gate questions 19.2 3.7 0.6 21.0 3.9 0.6 3.6 2.2 1.0
Multiple gate questions 9.9 1.4 0.5 10.8 1.4 0.6 1.9 1.2 0.3
Multiple use questions absent 12.1 3.1 0.6 13.1 3.3 0.6 2.7 1.7 0.2
Multiple use questions present 17.9 2.1 0.6 19.8 2.2 0.5 3.0 1.8 1.1
Consistency checks absent 11.6 2.3 0.6 12.7 2.3 0.5 2.5 1.7 1.0
Consistency checks/present 18.5 3.0 0.5 20.3 3.2 0.6 3.3 1.8 0.2

Any Illicit Drug1

Single gate questions 46.3 16.9 6.9 47.8 16.1 6.2 33.5 23.7 13.0
Multiple gate questions 40.7 13.1 4.7 42.2 12.1 4.1 28.3 22.2 10.6
Multiple use questions absent 41.6 14.5 6.6 42.6 13.5 5.9 32.8 24.3 13.1
Multiple use questions present 46.0 15.8 5.1 48.0 15.0 4.4 29.6 21.9 10.8
Consistency checks absent 43.4 14.6 4.7 45.0 13.8 4.0 30.8 21.2 10.4
Consistency checks/present 44.0 15.8 7.3 45.4 14.8 6.6 31.5 25.2 13.7

1Any illicit drug includes marijuana, cocaine, crack, heroin, inhalants, hallucinogens, and nonmedical use of analgesics, sedatives, stimulants, and tranquilizers.

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

Exhibit 6.1.3 Results of Logistic Regression Modeling of ACASI Versus PAPI
  Adjusted Prevalence
  ACASI PAPI p value Odds Ratio
Overall: Adjusted for gender, age group, education, and race/ethnicity
Cigarettes: Lifetime Use 73.0 71.8 0.72 1.06
Past Year Use
30.8 33.1 0.51 0.90
Past Month Use
25.5 29.8 0.16 0.81
Alcohol: Lifetime Use 86.5 85.5 0.66 1.09
Past Year Use
69.5 66.0 0.32 1.17
Past Month Use
47.8 51.9 0.32 0.85
Marijuana: Lifetime Use 34.6 34.4 0.95 1.01
Past Year Use
9.5 8.6 0.63 1.11
Past Month Use
4.1 4.6 0.72 0.90
Cocaine: Lifetime Use 13.6 9.0 0.05 1.59
Past Year Use
2.5 1.5 0.15 1.67
Past Month Use
0.4 0.4 0.97 0.98
Any Illicit Drug:1 Lifetime Use 46.5 37.2 0.08 1.30
Past Year Use
14.8 10.0 0.00 1.57
Past Month Use
5.2 4.8 0.73 1.10
12 to 17 Year Olds: Adjusted for gender and race/ethnicity
Cigarettes: Lifetime Use 44.5 35.7 0.002 1.43
Past Year Use
22.7 24.9 0.37 0.89
Past Month Use
15.9 18.3 0.22 0.84
Alcohol: Lifetime Use 45.1 37.9 0.007 1.35
Past Year Use
36.6 32.6 0.12 1.19
Past Month Use
16.7 18.5 0.45 0.88
Marijuana: Lifetime Use 20.6 16.1 0.013 1.36
Past Year Use
16.7 12.9 0.041 1.35
Past Month Use
8.3 7.3 0.47 1.16
Cocaine: Lifetime Use 2.6 2.9 0.83 0.91
Past Year Use
1.5 2.1 0.51 0.69
Past Month Use
0.6 0.9 0.47 0.60
Any Illicit Drug:1 Lifetime Use 31.1 20.4 0.000 1.76
Past Year Use
23.0 14.5 0.000 1.76
Past Month Use
11.9 7.9 0.023 1.57

1Any illicit drug includes marijuana, cocaine, crack, heroin, inhalants, and hallucinogens, as well as nonmedical use of analgesics, sedatives, stimulants, and tranquilizers.

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

Exhibit 6.1.4 Logistic Regression Modeling Results: ACASI Treatment Groups Versus PAPI
Effect Effect Categories p value Odds
Ratio
Adjusted Prevalence
Lifetime Cigarette Use
ACASI vs. PAPI ACASI 0.72 1.06 73.0%
Multiple Gate vs. PAPI Multiple Gate 0.81 0.94 70.5%
Single Gate vs. PAPI Single Gate 0.32 1.18 75.1%
Multiple Chance: Present vs. PAPI Present 0.85 0.96 71.5%
Multiple Chance: Absent vs. PAPI Absent 0.45 1.16 74.8%
Consistency Checks: Present vs. PAPI Present 0.30 1.24 75.9%
Consistency Checks: Absent vs. PAPI Absent 0.68 0.93 70.2%
Past Year Cigarette Use
ACASI vs. PAPI ACASI 0.51 0.90 30.8%
Multiple Gate vs. PAPI Multiple Gate 0.93 1.02 33.6%
Single Gate vs. PAPI Single Gate 0.29 0.80 28.5%
Multiple Chance: Present vs. PAPI Present 0.96 0.99 32.9%
Multiple Chance: Absent vs. PAPI Absent 0.38 0.82 28.9%
Consistency Checks: Present vs. PAPI Present 0.74 1.05 34.2%
Consistency Checks: Absent vs. PAPI Absent 0.28 0.78 27.8%
Past Month Cigarette Use
ACASI vs. PAPI ACASI 0.16 0.81 25.5%
Multiple Gate vs. PAPI Multiple Gate 0.34 0.81 25.6%
Single Gate vs. PAPI Single Gate 0.34 0.81 25.5%
Multiple Chance: Present vs. PAPI Present 0.92 1.02 30.2%
Multiple Chance: Absent vs. PAPI Absent 0.04 0.64 21.2%
Consistency Checks: Present vs. PAPI Present 0.99 0.99 29.7%
Consistency Checks: Absent vs. PAPI Absent 0.05 0.66 21.8%
Lifetime Alcohol Use
ACASI vs. PAPI ACASI 0.66 1.09 86.5%
Multiple Gate vs. PAPI Multiple Gate 0.42 1.21 87.7%
Single Gate vs. PAPI Single Gate 0.98 1.00 85.4%
Multiple Chance: Present vs. PAPI Present 0.62 1.10 86.6%
Multiple Chance: Absent vs. PAPI Absent 0.78 1.07 86.3%
Consistency Checks: Present vs. PAPI Present 0.34 1.20 87.6%
Consistency Checks: Absent vs. PAPI Absent 0.97 0.99 85.4%
Past Year Alcohol Use
ACASI vs. PAPI ACASI 0.32 1.17 69.5%
Multiple Gate vs. PAPI Multiple Gate 0.13 1.40 73.1%
Single Gate vs. PAPI Single Gate 0.92 1.02 66.5%
Multiple Chance: Present vs. PAPI Present 0.06 1.38 72.8%
Multiple Chance: Absent vs. PAPI Absent 0.95 1.01 66.3%
Consistency Checks: Present vs. PAPI Present 0.08 1.42 73.3%
Consistency Checks: Absent vs. PAPI Absent 0.99 0.99 66.0%
Past Month Alcohol Use
ACASI vs. PAPI ACASI 0.33 0.85 47.8%
Multiple Gate vs. PAPI Multiple Gate 0.61 0.89 48.9%
Single Gate vs. PAPI Single Gate 0.37 0.82 46.9%
Multiple Chance: Present vs. PAPI Present 0.38 0.82 47.0%
Multiple Chance: Absent vs. PAPI Absent 0.49 0.88 48.6%
Consistency Checks: Present vs. PAPI Present 0.91 1.03 52.6%
Consistency Checks: Absent vs. PAPI Absent 0.11 0.72 43.5%
Lifetime Marijuana Use
ACASI vs. PAPI ACASI 0.95 1.01 34.6%
Multiple Gate vs. PAPI Multiple Gate 0.94 0.98 34.1%
Single Gate vs. PAPI Single Gate 0.87 1.03 35.1%
Multiple Chance: Present vs. PAPI Present 0.53 1.12 37.1%
Multiple Chance: Absent vs. PAPI Absent 0.61 0.91 32.4%
Consistency Checks: Present vs. PAPI Present 0.92 1.02 34.9%
Consistency Checks: Absent vs. PAPI Absent 1.00 1.00 34.4%
Past Year Marijuana Use
ACASI vs. PAPI ACASI 0.60 1.11 9.5%
Multiple Gate vs. PAPI Multiple Gate 0.32 0.80 7.0%
Single Gate vs. PAPI Single Gate 0.15 1.40 11.6%
Multiple Chance: Present vs. PAPI Present 0.39 1.25 10.6%
Multiple Chance: Absent vs. PAPI Absent 0.94 0.98 8.5%
Consistency Checks: Present vs. PAPI Present 0.58 1.18 10.0%
Consistency Checks: Absent vs. PAPI Absent 0.80 1.05 9.0%
Past Month Marijuana Use
ACASI vs. PAPI ACASI 0.72 0.90 4.1%
Multiple Gate vs. PAPI Multiple Gate 0.31 0.72 3.4%
Single Gate vs. PAPI Single Gate 0.91 1.04 4.8%
Multiple Chance: Present vs. PAPI Present 0.43 0.75 3.5%
Multiple Chance: Absent vs. PAPI Absent 0.92 1.04 4.7%
Consistency Checks: Present vs. PAPI Present 0.73 1.15 5.2%
Consistency Checks: Absent vs. PAPI Absent 0.16 0.68 3.2%
Lifetime Cocaine Use
ACASI vs. PAPI ACASI 0.05 1.59 13.6%
Multiple Gate vs. PAPI Multiple Gate 0.93 0.98 8.8%
Single Gate vs. PAPI Single Gate 0.01 2.19 17.8%
Multiple Chance: Present vs. PAPI Present 0.01 1.99 16.4%
Multiple Chance: Absent vs. PAPI Absent 0.50 1.24 11.0%
Consistency Checks: Present vs. PAPI Present 0.04 2.11 17.2%
Consistency Checks: Absent vs. PAPI Absent 0.57 1.17 10.3%
Past Year Cocaine Use
ACASI vs. PAPI ACASI 0.15 1.67 2.5%
Multiple Gate vs. PAPI Multiple Gate 0.68 0.87 1.3%
Single Gate vs. PAPI Single Gate 0.04 2.40 3.5%
Multiple Chance: Present vs. PAPI Present 0.47 1.32 1.9%
Multiple Chance: Absent vs. PAPI Absent 0.14 2.02 2.9%
Consistency Checks: Present vs. PAPI Present 0.19 1.94 2.8%
Consistency Checks: Absent vs. PAPI Absent 0.27 1.44 2.1%
Past Month Cocaine Use
ACASI vs. PAPI ACASI 0.97 0.98 0.36%
Multiple Gate vs. PAPI Multiple Gate 0.87 0.93 0.34%
Single Gate vs. PAPI Single Gate 0.95 1.03 0.38%
Multiple Chance: Present vs. PAPI Present 0.91 0.96 0.35%
Multiple Chance: Absent vs. PAPI Absent 0.98 1.01 0.37%
Consistency Checks: Present vs. PAPI Present 0.89 0.94 0.35%
Consistency Checks: Absent vs. PAPI Absent 0.96 1.03 0.38%
Any Lifetime Illicit Drug Use1
ACASI vs. PAPI ACASI 0.08 1.30 43.5%
Multiple Gate vs. PAPI Multiple Gate 0.50 1.14 40.4%
Single Gate vs. PAPI Single Gate 0.06 1.45 46.2%
Multiple Chance: Present vs. PAPI Present 0.06 1.44 46.0%
Multiple Chance: Absent vs. PAPI Absent 0.41 1.18 41.2%
Consistency Checks: Present vs. PAPI Present 0.22 1.33 44.0%
Consistency Checks: Absent vs. PAPI Absent 0.22 1.28 43.1%
Any Past Year Illicit Drug Use1
ACASI vs. PAPI ACASI 0.004 1.57 14.8%
Multiple Gate vs. PAPI Multiple Gate 0.21 1.32 12.8%
Single Gate vs. PAPI Single Gate 0.01 1.79 16.6%
Multiple Chance: Present vs. PAPI Present 0.02 1.64 15.4%
Multiple Chance: Absent vs. PAPI Absent 0.03 1.50 14.3%
Consistency Checks: Present vs. PAPI Present 0.04 1.67 15.7%
Consistency Checks: Absent vs. PAPI Absent 0.05 1.48 14.1%
Any Past Month Illicit Drug Use1
ACASI vs. PAPI ACASI 0.73 1.10 5.2%
Multiple Gate vs. PAPI Multiple Gate 0.63 0.88 4.2%
Single Gate vs. PAPI Single Gate 0.44 1.29 6.1%
Multiple Chance: Present vs. PAPI Present 0.81 0.93 4.4%
Multiple Chance: Absent vs. PAPI Absent 0.45 1.27 6.0%
Consistency Checks: Present vs. PAPI Present 0.31 1.41 6.6%
Consistency Checks: Absent vs. PAPI Absent 0.50 0.83 4.0%

1Any illicit drug includes marijuana, cocaine, crack, heroin, inhalants, hallucinogens, and nonmedical use of analgesics, sedatives, stimulants, and tranquilizers.

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

Exhibit 6.1.5 Statistically Significant Modeling Results: Prevalence of Drug Use in the ACASI Treatment Groups Versus PAPI (Weighted Results)
Outcome Effects p value Odds Ratio Adjusted Prevalence
Total Sample
Cigarettes:
Past Month
Multiple Chance Absent vs. PAPI 0.04 0.64 21.2%
Alcohol:
Past Year
Multiple Chance Present vs. PAPI 0.06 1.38 72.8%
  Consistency Checks Present vs. PAPI 0.08 1.42 73.3%
Cocaine:
Lifetime
ACASI vs. PAPI 0.05 1.59 13.6%
  Single Gate vs. PAPI 0.01 2.19 17.8%
  Multiple Chance Present vs. PAPI 0.01 1.99 16.4%
  Consistency Checks Present vs. PAPI 0.04 2.11 17.2%
Cocaine:
Past Year
Single Gate vs. PAPI 0.04 2.40 3.5%
Any Illicit Drug Use:1
Lifetime
ACASI vs. PAPI 0.08 1.3 43.5%
  Single Gate vs. PAPI 0.06 1.45 46.2%
  Multiple Chance Present vs. PAPI 0.06 1.44 46.0%
Any Illicit Drug Use:1
Past Year
ACASI vs. PAPI 0.004 1.57 14.8%
  Single Gate vs. PAPI 0.01 1.79 16.6%
  Multiple Chance Absent vs. PAPI 0.03 1.50 14.3%
  Consistency Checks Absent vs. PAPI 0.05 1.48 14.1%
  Consistency Checks Present vs. PAPI 0.04 1.67 14.1%

1Any illicit drug includes marijuana, cocaine, crack, heroin, inhalants, hallucinogens, and nonmedical use of analgesics, sedatives, stimulants, and tranquilizers.

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

6.2 Reporting of Nonmedical Use of Psychotherapeutic Drugs

There was a single version of the ACASI program for questions on the nonmedical use of psychotherapeutic drugs, which include analgesics, tranquilizers, stimulants, and sedatives. This version differed from the PAPI answer sheets in several respects. As for the other sections of the core interview, the ACASI provided a more private mode of data collection. The "answer all questions" privacy-enhancing procedure was not used in these answer sheets, and the protocol allowed respondents to be routed past the detailed questions if they had never used one of the specific drugs. Thus, the ACASI privacy enhancement was even greater for these drugs than for other sections of the core. In the PAPI answer sheet, respondents received the list of drugs in a matrix format listing the drug on the left and two columns of check boxes on the right. One column was labeled "yes" and the other "no," and the respondent was asked to check a box for each drug. In contrast, in the ACASI version, respondents were presented with an individual question on each of the drugs of interest and asked to indicate whether he or she had ever used it. This served to focus their attention on each individual drug.

Taken together, these differences made for both a much more private interviewing environment and one that required increased attention to the response task. One would expect that this would have some impact on reporting, which it did. Exhibit 6.2.1 presents the results.

Overall, lifetime reporting of the nonmedical use of analgesics increased by 300%, from an estimated 4.9% of the population to 14.8%. Both youths and adults showed dramatic increases, with the youths' lifetime prevalence rate being 3.7 times higher under ACASI and the adults' 3 times greater. Use of analgesics during the past 12 months showed a similar dramatic increase. For the other psychotherapeutic drugs, similar but less dramatic results were obtained. For example, overall the reported lifetime prevalence of nonmedical use of tranquilizers was 2.7 times higher under ACASI, 1.8 times higher for stimulants, and 2.5 times higher for sedatives.



6.3 Effect of ACASI on Mental Health Questions

Questions about adult mental health syndromes have been included in the NHSDA instrument since 1994. These questions were interviewer-administered. Consequently, respondents who had experienced symptoms suggestive of mental health disorders in the past 12 months were required to report these problems aloud to the interviewers. Because of stigma or shame associated with mental health problems, some respondents who experienced such problems may have been reluctant to report them to an interviewer, particularly if another household member was within hearing distance. We included the adult mental health questions in the ACASI section in the 1997 field experiment because we felt that increased privacy might facilitate the reporting of potentially sensitive mental health problems.

The mental health questions also included numerous routing patterns, some of which required checking of multiple items to determine if a condition had been met (e.g., existence of at least one affirmative answer to a series of gate agoraphobia questions). Other contingent questioning structures in the mental health section involved follow-up questions to route respondents back into a "branch" if they initially answered "no" to a gate question but then answered affirmatively to the follow-up question. The field interviewers sometimes made routing errors. Errors were particularly problematic if interviewers incorrectly bypassed mental health questions that respondents should have answered, because some of these respondents might have reported significant mental health problems had they been asked the questions. In contrast, routing errors are virtually nonexistent in a well-tested CAI instrument because the computer program routinely checks the routing conditions and then executes the instructions according to the contingent questioning structure that was set up by the researchers.

In this section, we first compare ACASI mental health data across the eight experimental treatments. We then compare estimates of different mental health syndromes based on the PAPI or ACASI modes of administration. Finally, we examine selected data quality issues.

6.3.1 Comparisons of Rates Within ACASI Treatments

In the 1997 field experiment, adult respondents were asked the same set of mental health questions regardless of the experimental treatment to which they were assigned. All other things being equal, respondents in the different treatments should not vary significantly in how they answered the mental health questions. We did not know, however, whether the number or types of questions posed in a given treatment might affect respondents' subsequent answers in nonexperimental sections. Respondents in a given treatment might be more likely than respondents in other treatments to answer gate mental health questions negatively and to thus bypass the branch questions (i.e., if a given treatment induces more "nay-saying").

To test the assumption that the experimental treatment did not appreciably affect respondents' answers to the mental health questions, we examined the unweighted percentages of adults in each treatment who answered affirmatively to lead questions for the different mental health syndromes covered in the ACASI instrument: major depressive episode, generalized anxiety disorder, agoraphobia, and panic attack. As expected, no clear patterns emerged in terms of the tendency of respondents to answer affirmatively to the gate mental health questions according to their experimental treatment. For example, about 23% to 30% of respondents who were asked the gate depression question answered affirmatively. Similarly, 8% to 14% of respondents in the different treatments who were asked the agoraphobia questions gave an affirmative answer, and 7% to 16% of respondents who were asked the panic attack questions gave an affirmative answer.

One exception was that 28% of respondents in Treatment 4 (single gate question, consistency check present, multiple use questions present) who were asked the gate anxiety question answered affirmatively compared to about 13% to 20% of the respondents in the other treatments who were asked this question. However, these respondents were not consistently more likely than respondents in other treatments to answer affirmatively to the gate questions. For example, respondents in four other treatments were somewhat more likely than respondents in Treatment 4 to answer affirmatively to the gate depression question.

Despite the higher rate of respondents in Treatment 4 who answered affirmatively to the lead anxiety question, these findings generally suggest that the experimental treatment did not affect adults' willingness to answer affirmatively to the lead questions in the mental health section. Therefore, in the remainder of this section, we present ACASI mental health data pooled across all eight treatments.

6.3.2 Comparison of Rates of Mental Health Syndromes

We compared mental health short-form scores for the adult field experiment respondents (n=865) and the comparison group of adults, that is, the Quarter 4 adult PAPI respondents subsetted to English-language interviews in the primary sampling units (PSUs) that corresponded to the field experiment (n=2,126). The PAPI scoring routines that have been used since the 1994 NHSDA create two sets of short-form score variables: one that does not take into account logical editing of the data and one that does take edits into account. For the ACASI/PAPI comparisons, we used the PAPI short-form score variables that do not take editing into account because we did no editing of the ACASI mental health data. To create short-form scores for the ACASI data, we adapted the PAPI scoring routines.

In both the ACASI and PAPI data, a short-form score of "3 or more" for major depressive episode, generalized anxiety disorder, or panic attack indicates that a person is a "probable case" for that mental syndrome. For agoraphobia, a short-form score of "1 or more" indicates a probable case.

Exhibit 6.3.1 shows the results of the mental health syndrome short-form score comparisons between the ACASI and the PAPI comparison group. Exhibit 6.3.2 compares results for ACASI and PAPI by gender and educational level. Exhibit 6.3.3 compares results for the two modes of interviewing by race/ethnicity. All estimates are weighted. We did not test for the statistical significance of any differences that we observed; therefore, we discuss differences that we observed as trends.

We observed a definite trend for ACASI to yield higher estimates of these mental syndromes compared with PAPI. In particular, the estimate of "probable caseness" for major depressive episode based on ACASI (14.6%) was nearly double the rate for PAPI (7.4%). In addition, the ACASI estimate for generalized anxiety disorder (5.8%) was nearly four times the PAPI rate (1.6%). Rates for panic attack and agoraphobia also tended to be higher for the ACASI instrument, although the rates of these two syndromes were generally low. Overall, the PAPI rates for this subset of the 1997 NHSDA were comparable to rates that have been observed in prior NHSDAs (e.g., SAMHSA, 1997).

Although rates of these different mental health syndromes tended to be higher in ACASI for both males and females compared with corresponding PAPI rates, the effects were particularly pronounced for males (see Exhibit 6.3.2). For example, the ACASI estimate of probable caseness for generalized anxiety disorder among males (5.1%) was more than 4.5 times the PAPI rate for males (1.1%). Similarly, ACASI estimated that 12.7% of adult males were probable cases for major depressive episode, or more than twice the PAPI rate of 5.5% for males.

In both ACASI and PAPI, women were more likely than men to score as probable cases for major depressive episode. This is consistent with prior NHSDA findings and with the mental health literature in general (Kessler et al., 1994; SAMHSA, 1996b, 1997). However, the gender differential in the rates for major depressive episode was not as pronounced for ACASI as it was for PAPI. The PAPI estimate of the prevalence of major depressive episode among women was almost twice that for men (9.1% vs. 5.5%, respectively). In comparison, the ACASI estimate for women was only 30% higher than the rate for men (16.4% vs. 12.7%). These findings suggest that the privacy of ACASI may provide respondents with more freedom to report these problems than might be the case if they had to report these problems to an interviewer.

Among the educational groups shown in Exhibit 6.3.2, there was a clear trend for ACASI to yield higher estimates for the different mental health syndromes for adults with some education beyond high school compared with PAPI data for this group. For the other educational groups, there was not a consistent pattern of ACASI yielding higher estimates compared with PAPI. These findings suggest that increased reporting of mental health problems by adults with higher levels of education may be driving the higher estimates in ACASI. Interestingly, PAPI yielded somewhat higher estimates for generalized anxiety disorder among adults who had not finished high school (4.4%) compared with ACASI estimates for this group (2.4%). Similarly, PAPI estimates for agoraphobia were somewhat higher for adults with a high school education or less compared with the corresponding ACASI estimates.

Among the different racial/ethnic groups shown in Exhibit 6.3.3, ACASI tended to yield higher estimates of the different mental health syndromes for whites and blacks relative to the corresponding PAPI estimates for these groups. With the smaller number of adult Hispanic respondents in the field experiment (n=145), estimates for Hispanics showed more variability across the different modes of administration. One notable finding was that blacks (0.8%) had a lower PAPI estimate for generalized anxiety disorder compared with whites (1.7%), but the ACASI rates for both of these groups were considerably higher and were comparable (6.7% vs. 5.9%, respectively). Similarly, the PAPI estimate for panic attack among blacks was lower than the estimate for whites (1.3% vs. 1.9% respectively), but the ACASI rate was higher for blacks than it was for whites (4.4% vs. 3.8%, respectively).

6.3.3 Comparison of Data Quality in the ACASI and PAPI Mental Health Items

We investigated the extent to which the ACASI administration of the mental health items improved data quality relative to PAPI, and whether improvements in data quality for the ACASI instrument might explain the higher rates of mental health problems based on ACASI. In particular, if interviewers administering the PAPI instrument made errors that caused them to skip over key questions used in scoring the mental health variables, respondents who might otherwise have reported significant problems would have missing data for these items. In contrast, we would expect careful testing of the ACASI program to lead to the elimination of these types of errors in ACASI. In addition, the programming of contingent routing instructions in ACASI can reduce or eliminate the occurrence of inconsistencies, such as respondents who report zero occurrences of a problem in the past 12 months (e.g., panic attacks) after having previously reported that they had the problem in that same period.

Exhibit 6.3.4 presents data on missing data patterns in the ACASI and Quarter 4 PAPI comparison group where (a) all items pertaining to a given mental health problem were blank (e.g., due to a breakoff prior to that point in the interview), or (b) the gate question was refused and data within the branch were missing, indicating that these questions within the branch were bypassed. In both the ACASI and PAPI groups, these rates of missing data were low. In particular, these missing percentages based on the PAPI comparison group were lower than the ACASI percentages. These findings suggest that the higher estimates of mental health problems based on ACASI cannot be explained by a tendency for entire sets of PAPI questions to have missing data.

Exhibit 6.3.5 compares selected patterns of inconsistent or ambiguous data in the ACASI or Quarter 4 PAPI comparison samples. These patterns include the following:

  1. the reported year when the respondent last had a 2-week period of depression being inconsistent with prior data indicating that the respondent had a 2-week period of depression in the past 12 months;
  2. the respondent not knowing or refusing to answer questions about how long the period of anxiety lasted or has been going on;
  3. the reported number of months that the respondent had agoraphobia-related fears being inconsistent with the reported length of these fears;
  4. zero panic attacks being reported in the past 12 months in question after the respondent had answered affirmatively to one of the gate panic attack questions; and
  5. the reported year when the respondent last had a panic attack being inconsistent with prior data indicating that the respondent had a panic attack in the past 12 months.

If respondents reported a 2-week period of feeling depressed or a panic attack in the past 12 months, the ACASI programming did not allow them to report a year other than 1996 or 1997 for the most recent occurrence of these problems. Furthermore, the ACASI program asked the question about the year when these problems last occurred only if the reported month when these problems last occurred matched the interview month (e.g., if a respondent reported last having a panic attack in November and the interview was conducted in November 1997).

Consequently, out-of-range years for the most recent symptoms of depression or panic attack did not occur in the ACASI data. Although these out-of-range years sometimes occurred in the PAPI data, they occurred relatively rarely in the Quarter 4 comparison sample. However, the occurrence of out-of-range years in the PAPI data raises the question of whether the ACASI program should "force" this consistency. Given the ability of CAI to resolve inconsistencies in the course of an interview, an alternative might be to permit a wider range of allowable years and then to prompt ACASI respondents to resolve any inconsistencies. That could involve resolving inconsistencies in the direction of less recent occurrence of depression symptoms or panic attacks.

Information on responses of "don't know" or "refused" to the questions about the duration of anxiety symptoms are presented in Exhibit 6.3.5 because ACASI respondents who answer "don't know" or "refused" to the duration questions are routed out of subsequent questions about symptoms associated with anxiety. All four of the ACASI respondents who had this particular pattern did not know how long they had experienced periods of anxiety or worry. If ACASI respondents are unable to recall the length of time that they experienced periods of anxiety, we would recommend adding a follow-up probe to determine if the period lasted fewer than 6 months, or 6 months or more, which is the key piece of information sought. Adding this follow-up probe would allow more information to be collected on symptoms associated with anxiety, as opposed to letting these respondents be routed out.

In addition, the ACASI programming specifications did not completely eliminate the potential for inconsistent reporting in the mental health items. Specifically, in the agoraphobia questions, it was possible for ACASI respondents to report a number of months that they had these fears that was inconsistent with the prior question on the length of time that they had these fears. Six ACASI respondents showed this pattern of inconsistent reporting. Therefore, this indicated that ACASI programming needs to be revised either by restricting the allowable range or by prompting respondents to resolve inconsistent answers.

Exhibit 6.3.6 shows selected patterns of skip errors for the Quarter 4 PAPI data only. These errors concerned the following:

  1. respondents being incorrectly asked entire sets of related questions twice, when they should have been asked only one set or the other (i.e., both sets of depression questions or both sets of anxiety questions); and
  2. respondents not being asked questions that are used in computing mental health scores, when other data suggest that they should have been asked (e.g., if a lead question was answered as "yes" but questions within the branch were blank).

The first type of contingent questioning error is problematic from a burden perspective in that an entire set of questions is essentially repeated. This unnecessarily increases the interview time, can add to respondent fatigue, and can result in breakoffs if respondents become angry about being re-asked questions that they already answered. The second type of routing error is problematic from a measurement perspective in that some respondents who might otherwise have reported significant problems would not be asked the relevant questions.

As shown in Exhibit 6.3.6, about 1% of the PAPI respondents (i.e., unweighted) in the Quarter 4 comparison group were incorrectly asked both sets of depression questions. Instead of being routed to the start of the anxiety section, these respondents were asked a second set of depression questions. Although relatively few respondents in the Quarter 4 PAPI comparison group were asked both sets of depression questions (n=26), a rate of 1.2% translates to more than 900 respondents in a sample of more than 75,000 interviews, which was the number planned for the 1999 NHSDA.

No respondents in the Quarter 4 PAPI comparison group had an affirmative answer to a lead depression question, with the questions being blank within the branch pattern. Similarly, rates of contingent questioning errors that affected the short-form score variables were low for the anxiety and agoraphobia sections of the interview. The rate of contingent questioning errors increased for the panic attack section, approaching 1% of respondents in the Quarter 4 PAPI comparison group (n=17). Compared with the other mental health sections, the panic attack section had more routing patterns that would provide opportunities for interviewers to exit the section prematurely.

However, these findings in Exhibit 6.3.6 suggest that contingent questioning errors in the PAPI instrument that could affect scoring of the mental health syndromes were relatively uncommon, at least in the fourth quarter among respondents completing an English-language interview. However, one caution is that we might expect error rates in the PAPI mental health data to be low by the fourth quarter due to interviewer experience. Most of the productive, high-quality interviewers working on the NHSDA by the start of the fourth quarter had three or more quarters of experience with the NHSDA interview by that time.

Nevertheless, these findings suggest that contingent questioning errors on the part of the PAPI interviewers are not likely to explain the considerably lower rates of some mental health syndromes in the PAPI data. Rather, the findings from these analyses of data quality issues suggest that the higher estimates of mental health syndromes based on ACASI reflect a greater tendency of ACASI respondents to report mental health problems, particularly among males and adults with higher levels of education.

Exhibit 6.3.1 Adult Mental Health Short-Form Scores
  1997 Field Experiment Comparison Group
Score CAPI/ACASI 1997 Q4 PAPI
Total 18+ (n=865) (n=2,126)
Major Depressive Episode Short-Form Score
0
85.2 91.9
1 or 2
0.1 0.7
3 or more
14.6 7.4
Generalized Anxiety Disorder Short-Form Score
0
94.2 98.3
1 or 2
0.0 0.1
3 or more
5.8 1.6
Agoraphobia Short-Form Score
0
98.1 98.4
1 or more
1.9 1.6
Panic Attack Short-Form Score
0
92.5 96.4
1 or 2
3.6 1.5
3 or more
3.9 2.0

Note: Percentages are weighted. PAPI short-form scores do not take into account logically imputed data. A score of "3 or more" indicates a probable case for major depressive episode, generalized anxiety disorder, or panic attack. A score of "1 or more" indicates a "probable case" for agoraphobia. Some table entries may not sum to 100% because of rounding.

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

 



 

Exhibit 6.3.3 Mental Health Syndromes, by Race/Ethnicity
    1997 Field Experiment Comparison Group
Syndrome Race/Ethnicity CAPI/ACASI 1997 Q4 PAPI
Unweighted Numbers of Respondents Total 18+ (n=865) (n=2,126)
  Hispanic (n=145) (n=347)
  Non-Hisp., Black (n=283) (n=748)
  Non-Hisp., All Other Races (n=437) (n=1,031)
Probable Case for Major Depressive Episode Total 18+ 14.6 7.4
  Hispanic 12.7 3.2
  Non-Hisp., Black 7.5 3.9
  Non-Hisp., All Other Races 16.0 8.3
Probable Case for Generalized Anxiety Disorder Total 18+ 5.8 1.6
  Hispanic 2.7 2.3
  Non-Hisp., Black 6.7 0.8
  Non-Hisp., All Other Races 5.9 1.7
Probable Case for Agoraphobia Total 18+ 1.9 1.6
  Hispanic 4.0 1.3
  Non-Hisp., Black 2.4 1.9
  Non-Hisp., All Other Races 1.6 1.6
Probable Case for Panic Attack Total 18+ 3.9 2.0
  Hispanic 4.1 4.5
  Non-Hisp., Black 4.4 1.3
  Non-Hisp., All Other Races 3.8 1.9

Note: PAPI short-form scores do not take into account logically imputed data.

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.  


 


 

6.4 Effect of Audio Computer-Assisted Self-Interviewing on Responses to Questions on Risk and Availability

The perceived risk of using drugs was assessed by asking how much people risk harming themselves when they engage in each of 11 different drug-using behaviors. The response choices offered were "no risk," "slight risk," "moderate risk," and "great risk." Respondents were also asked about how difficult or easy it would be to get each of five drugs. Response options were "probably impossible," "very difficult," "fairly difficult," "fairly easy," and "very easy."

6.4.1 Perceived Risk of Different Drug-Using Behaviors

Questions about levels of risk perceived to be associated with using different drugs were included in every NHSDA since 1985 except for the 1995 NHSDA. The major purpose for asking these questions has been to quantify the national population's knowledge about risks of harm, physical and otherwise, that might result from using drugs. Such information has been useful for guiding substance abuse prevention and education efforts. The questions used in the 1997 NHSDA constitute a set that have been consistently asked, although there have been some improvements in the frequency of use wordings. The risk questions were presented on their own answer sheet using the NHSDA self-administered questionnaire (SAQ).

Exhibit 6.4.1 shows the percentages (weighted) of the ACASI and PAPI/SAQ total populations and the populations within specific age groups, as well as current cigarette smokers and those who used any illicit drug in the past year who reported perceptions of great risk or other harm associated with different drug-using behaviors. For the total population and the two age groups, the level of risk was perceived as great by more respondents to the PAPI/SAQ instrument than by ACASI respondents. The only noteworthy exceptions were youths aged 12 to 17 who answered with ACASI. Fewer of them than PAPI/SAQ youths perceived that smoking one or more packs of cigarettes and smoking marijuana once a month entailed great risk. Differences between the methodologies in percentages of the total population and the two age groups who reported great risk ranged from 0% to about 6% for most of the drug-using behaviors. For the two drinking behaviors, however, differences between the two methodologies were greater, with at least 7% to 10% more PAPI/SAQ respondents reporting perceptions of great risk.

Perceptions that great risk is associated with different drug-using behaviors varied with drug use. In Exhibit 6.4.1, perceptions about risk associated with the different drug-using behaviors are reported for current cigarette smokers and for persons who used any illicit drug in the past year. Almost without exception, fewer smokers and fewer illicit drug users reported that the different drug-using behaviors entailed great risk than did the total populations of the two age groups and of all ages. In a few instances, drug users rated an activity as more risky: (a) current cigarette smokers of all ages when they answered for trying heroin once or twice and (b) smokers aged 12 to 17 when they answered questions on using heroin and cocaine. In addition, more 12- to 17-year-old past year illicit drug users reported great risk for heroin and cocaine use. This pattern of more (usually younger) smokers and past year illicit drug users who reported great risk of using heroin and cocaine appears in both the ACASI and the PAPI/SAQ data and is more pronounced under ACASI.

In general, past year illicit drug users were less likely than the overall population to perceive that smoking marijuana entailed great risk. The ACASI respondents who were classified as past year drug users were somewhat more likely than the PAPI/SAQ respondents to rate monthly marijuana use as harmful.

Differences between the ACASI and PAPI/SAQ instruments may be contributing to the general pattern of ACASI percentages reporting great risk being smaller than the PAPI/SAQ percentages. On the PAPI/SAQ answer sheet, the boxes for indicating one's response are all printed on the same line as the text of the drug-using behavior, with the actual text of the response options printed only once on the page as headings above columns of boxes and ordered left to right from "no risk" to "great risk." In ACASI, each response option appears, along with its numbers to be keyed to indicate one's choice, in a list below the text of the general question's introductory phrase and the phrase specifying the drug-using behavior. The response options are identical to those on the PAPI/SAQ answer sheet and are ordered top to bottom from "no risk" to "great risk." Respondents to the PAPI/SAQ risk questions, most of whom were likely to have been right-handed, may have been somewhat more likely to mark the boxes physically located on the right; if so, reports about the risk associated with the drug-using behaviors will be biased toward greater risk. On the other hand, respondents to the ACASI risk questions may have been somewhat more likely to key in the number for a response option printed higher in the list and therefore encountered more quickly than they were to read the entire list or even further down the list before keying an answer. There is considerable evidence that, for response options similar in content to those in the risk questions (e.g., rankings of satisfaction, importance), those printed closer to the top of a list of options will be chosen more often than items printed closer to the end of the list, even when the ordering of the listing is reversed. If this tendency is occurring, ACASI reports about risk of drug-using behaviors will be biased toward lower levels of risk.

Exhibit 6.4.2 presents the (unweighted) percentages of the ACASI and PAPI/SAQ total populations and the populations of specific age groups who failed to answer each question about how much risk they thought each drug-using behavior exposed one to. The missing data types "refused" and "no answer (blank)" were combined in Exhibit 6.4.2 because PAPI/SAQ respondents were not required to indicate their refusal to answer any particular question or even the entire set. So some of the "no answer (blank)" missing data in the PAPI/SAQ data may actually reflect any such refusals.

The amount of missing data was small for both data collection methodologies, never exceeding 4.5% for any risk question and occurring much less often for most of the risk questions. Most ACASI missing data resulted from "don't know" responses; most of the PAPI/SAQ missing data resulted from "no answer (blank)."

More data are "missing" in the ACASI mostly because more ACASI respondents answered "don't know," an option not explicitly offered in the PAPI/SAQ. PAPI/SAQ respondents apparently either tried harder to respond with a risk answer or simply left the item blank. In PAPI/SAQ, the generally higher rates of "no answer (blank)" missing data were still very small (never exceeding 1.7% and usually no greater than 0.7%), suggesting that most uncertain respondents tried to give a risk answer. In ACASI, however, the explicit availability of "don't know" and "refused" response options may have made it easier for respondents not to provide risk answers, resulting in higher rates of missing data of both kinds, especially the "don't know" type. If the goal was to reduce the absolute amount of missing data, PAPI/SAQ did a better job; however, enabling respondents who were very uncertain about the risk associated with a drug-using behavior to register that uncertainty may actually result in more valid data.

6.4.2 Availability of Drugs

Exhibit 6.4.3 presents percentages (weighted) of the ACASI and PAPI/SAQ total populations, current cigarette smokers, and past year illicit drug users who reported that five different drugs were fairly or very easy to get. The overall pattern of responses was for more PAPI/SAQ than ACASI respondents to report that getting each drug would be easy, although the differences between the two modes were usually smaller than in the risk data. Indeed, for all ages and the two age-group total populations, differences between the ACASI and PAPI/SAQ percentages reporting that getting the drugs would be fairly or very easy were unremarkable, except that about 15% more PAPI/SAQ than ACASI respondents reported that getting marijuana was easy. For both modes, subgroups who used cigarettes or drugs were more likely to report that getting each drug would be fairly or very easy. This last finding is consistent with our expectation that persons who use drugs, even if only cigarettes, are more likely to feel as though they can get drugs without too much difficulty, while persons who use no drugs are more likely to feel that getting drugs would be difficult.

This constancy of the pattern of differences between the two modes points toward a systematic mode effect. Being able to see all the items and availability response options on the PAPI/SAQ answer sheet may encourage more variation in responses with a net effect of shifting the average response toward the "fairly easy" or "very easy" options. This factor does not seem sufficient by itself, however, to cause the regular and consistent pattern in the differences between the ACASI and the PAPI/SAQ availability responses.

Exhibit 6.4.4 presents (unweighted) percentages of the ACASI and PAPI/SAQ total populations and the populations in specific age groups who did not provide a response to availability questions. The missing data types "refused" and "no answer (blank)"were combined because of the impossibility of distinguishing between the two types of missing data for PAPI/SAQ respondents.

The amounts of missing data in both the ACASI and PAPI/SAQ data for the total samples and for the subgroups defined by age ranged from 2.1% to 6.8% of the ACASI subsamples, with most of the rates exceeding 4% and from 0.5% to 5.8% of the PAPI/SAQ subsamples with most of the rates falling below 2.2%. As in the risk data, most ACASI missing data consisted of "don't know" responses, while most of the PAPI/SAQ missing data resulted from "no answer (blank)."

Again, rates of missing data in ACASI were higher than in PAPI/SAQ because of the "don't know" responses. The ready availability of this response for those who used ACASI may mean that they are truly uncertain of the answer.  


 

 

 

Exhibit 6.4.4 Unweighted Missing Data in Questions About How Difficult or Easy Getting Different Drugs Is for the Total Population, by Age Group
Respondent Characteristics 1997 Field Experiment Comparison Group 1997 Field Experiment Comparison Group
Age Group Type of Missing Data CAPI/ACASI 1997 Q4 NHSDA:
PAPI/SAQ
CAPI/ACASI 1997 Q4 NHSDA:
PAPI/SAQ
    a. Availability of Marijuana d. Availability of "Crack"
All Ages Total 3.0% 1.4% 4.5% 1.8%
  Don't Know 2.4% 0.6% 3.8% 0.8%
  Multiple Response 0.0% 0.0% 0.0% 0.0%
  Refusal, No Answer (Blank) 0.7% 0.8% 0.7% 1.0%
12-17 Total 2.1% 0.6% 3.8% 1.0%
  Don't Know 1.6% 0.1% 3.1% 0.2%
  Multiple Response 0.0% 0.0% 0.0% 0.0%
  Refusal, No Answer (Blank) 0.5% 0.5% 0.6% 0.8%
18+ Total 4.2% 1.8% 5.5% 2.2%
  Don't Know 3.4% 0.8% 4.7% 1.1%
  Multiple Response 0.0% 0.0% 0.0% 0.0%
  Refusal, No Answer (Blank) 0.8% 0.9% 0.8% 1.1%
    b. Availability of LSD e. Availability of Heroin
All Ages Total 5.6% 2.2% 4.9% 2.8%
  Don't Know 4.9% 0.9% 4.2% 1.3%
  Multiple Response 0.0% 0.1% 0.0% 0.0%
  Refusal, No Answer (Blank) 0.7% 1.2% 0.8% 1.5%
12-17 Total 4.7% 1.4% 3.6% 0.5%
  Don't Know 3.9% 0.3% 2.9% 0.2%
  Multiple Response 0.0% 0.2% 0.0% 0.0%
  Refusal, No Answer (Blank) 0.7% 0.9% 0.7% 0.4%
18+ Total 6.8% 2.5% 6.7% 5.8%
  Don't Know 6.1% 1.2% 5.9% 2.8%
  Multiple Response 0.0% 0.1% 0.0% 0.0%
  Refusal, No Answer (Blank) 0.7% 1.3% 0.8% 3.0%
    c. Availability of Cocaine Unweighted n's (Denominators) Unweighted n's (Denominators)
All Ages Total 4.4% 1.8% 1,982 3,105
  Don't Know 3.6% 0.8% 1,982 3,105
  Multiple Response 0.0% 0.1% 1,982 3,105
  Refusal, No Answer (Blank) 0.8% 0.9% 1,982 3,105
12-17 Total 3.5% 1.0% 1,117 979
  Don't Know 2.7% 0.2% 1,117 979
  Multiple Response 0.0% 0.2% 1,117 979
  Refusal, No Answer (Blank) 0.8% 0.6% 1,117 979
18+ Total 5.5% 2.2% 865 2,126
  Don't Know 4.7% 1.1% 865 2,126
  Multiple Response 0.0% 0.0% 865 2,126
  Refusal, No Answer (Blank) 0.8% 1.1% 865 2,126

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures, 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

6.5 Effect of New ACASI Strategy on Frequency of Use

Each of the treatments employed the new method of asking about the frequency of use in the past 12 months.13 Our primary interest in analyzing the data from the revised 12-month frequency-of-use items was to determine whether the new format affected the quality of the data collected. To investigate this, we conducted two types of analyses. First, we investigated the rates of missing data for the gate question into each 12-month frequency-of-use series and compared the rates to the Quarter 4 PAPI data collection version. Second, we calculated the 12-month frequency of use for each respondent by multiplying the reporting units by the frequency of use to create a continuous variable. We then created categories similar to those used in the PAPI questionnaire and compared the distribution of responses to those collected in the Quarter 4 PAPI interview. Each of these analyses is presented below.

6.5.1 Missing Data Rates

By design, CAI respondents cannot mistakenly bypass an item or leave an item blank. The CAI program requires an answer to be entered for every question. However, respondents are still free to provide a "don't know" or "refused" response in the CAI instrument. In fact, they may be more likely to use these responses in the CAI interview because they have been specifically trained in the use of the "don't know" key and both the "don't know" and "refused" keys are explicitly labeled on the keyboard. This differs from the PAPI environment where these responses do not appear on the answer sheet and the respondent is not explicitly told how to record one of these answers. A respondent who wished to provide one of these responses in the self-administered portion of the PAPI interview had to ask the interviewer how to record the answer,14 come up with a method on his or her own, or simply leave the item blank.

Because the process for providing the 12-month frequency-of-use data differs between CAI and PAPI, it is somewhat difficult to compare the missing data rates. For our purposes here, we chose to examine the data in two ways. We first compared the "don't know" and "refused" from the CAI interview to the explicit "don't know" and "refused" responses from the PAPI data. We found that there were no explicit "don't know" or "refused" responses for any of the 12-month frequency-of-use items in the PAPI data. In Exhibit 6.5.1, we present data that adds all the "bad data" into the PAPI data. These include multiple responses, inappropriately answered questions, and blanks because it seems likely that questions left blank could reflect an inability or unwillingness to respond on the part of the respondent. In an effort to maintain comparability between the two modes, we include only the data provided during the initial pass through the 12-month frequency-of-use item for respondents assigned to the consistency check treatment.

The data in Exhibit 6.5.1 show that for the alcohol and marijuana sections, the CAI data were more complete than the PAPI data. In fact, for the alcohol section, the bad data rate was more than twice that for CAI (7.9% vs. 3.5%). However, for all other drugs types except hallucinogens, the "don't know" and "refused" rates for CAI were still higher than for all the bad data associated with the PAPI item. For hallucinogens, the rates were comparable. Comparing the CAI and PAPI data in this manner, however, is somewhat artificial. For the CAI data, the "don't know" and "refused" rates were calculated based on only those respondents who were routed to the question (users of the drug). By contrast, all respondents were included in the calculation for the PAPI data because every respondent answered every question regardless of use status. The result is that the CAI rates may appear artificially high when compared to the PAPI data. Nevertheless, the comparison may provide some insight into the quality of the data collected for the 12-month frequency item.

6.5.2 Distribution of the Responses

To compare the distribution of responses collected in each mode, we collapsed the data into two categories: (a) use for 50 days or less and (b) use for 51 days or more. Then we compared the distribution of responses. These analyses excluded those respondents who indicated no use during the 12-month period.

Exhibit 6.5.2 shows the weighted estimates for youths and adults separately. We see that the younger respondents consistently reported higher frequencies of use in CAI than in PAPI. This trend was not as strong for adult respondents, however, suggesting that youths were more influenced by question wording.

These findings, although intriguing, are somewhat difficult to explain. It is possible that the increased frequency of use reported in CAI is due to the revised question wording. Because respondents were generating their own responses without being able to see the other response choices (as is the case on the answer sheets used in PAPI), they may have been less influenced by the desire not to appear a "serious user" or trying to appear "like everyone else." The ability to influence frequency reporting by altering the endpoints of the scale has been well documented in the literature (see, e.g., Schwarz & Hippler, 1991). A similar phenomenon may be taking place here, in that respondents' answers were more accurate because they were less influenced by outside factors. It is also possible that these results were just another indication of the benefits of audio computer-assisted self-interviewing (ACASI) for collecting sensitive data. Having improved the privacy of the interview, we may now be improving respondents' honesty in reporting. Because the higher reporting is most obvious for younger respondents, this second hypothesis seems especially likely as youths are more likely to have the privacy of their interview compromised by the presence of other household members. Younger respondents, however, could be more likely to be influenced by the presence of scale categories, so the first hypothesis should not be ruled out. Most likely, both hypotheses are contributing to the results seen here.

Further research should be conducted to fully disentangle the effect of ACASI from the effect of the revised question text. As a first step in this direction, we compared the original 12- month frequency distribution to the 12-month distribution that was obtained from multiplying respondents' 30-day frequency by 12 to see whether the CAI or the PAPI method was more similar. In an attempt to understand the differences, we examined the consistency of the 30-day and 12-month frequency-of-use responses by multiplying the reports of use in the past 30 days by 12. As noted by the respondents, these will not necessarily agree because the 12-month frequency is likely to reflect the respondent's assessment of his or her average rate of use over a 12-month period.

Exhibit 6.5.3 presents the results of this comparison. We show the estimated number of days used and the ratio of 12 times the 30-day reports to the reported frequency of use in the past 12 months. It appears that the ACASI gave more consistent results in that all of the ratios were greater than one. This is what we would expect because respondents are likely to forget instances of use over a 12-month recall period. In addition, we note that, in general, the ratios tended to be similar; however, when they were different, in most cases the ACASI ratios were closer to one.

Exhibit 6.5.3 also shows the average number of days used in the past 30 days that were reported by respondents under each interview mode. We note that for the more sensitive drugs, ACASI resulted in higher reports of the number of days used.

Finally, we examined the patterns of reporting to determine if there is evidence that respondents were completing the response task as we intended. Exhibit 6.5.4 presents data on the number and percentage of respondents who chose to provide their answers using the three available reporting frequencies (days per week, month, or year) and the distribution of the responses among those who chose to report using a particular reporting period. Data for alcohol, marijuana, and cocaine are shown.

The data indicate that the majority of respondents appeared to be responding as intended. For example, we would expect that respondents who chose to report monthly would tend to use less frequently than several days a week and those who selected yearly reporting would tend to use less than once a month. Thus, we see that most of the respondents who selected the monthly or yearly reporting periods were infrequent users. For example, if we examine the reported frequency of alcohol use among the youths, 91.1% of those who chose to report their use on a monthly basis reported using 10 or less days per month. For those who chose to report days per year, 80.8% reported using less than 10 or less days per year. Corresponding percentages for adults were 85.8% and 72.9%. Similar results were observed for marijuana and cocaine.

Based upon these findings, this revised procedure for asking about frequency of use in the past 12 months was adopted for the 1999 NHSDA.

Exhibit 6.5.1 Comparison of CAI to PAPI for "Don't Know" and "Refused" Responses for the 12-Month Frequency-of-Use Item (Including All Bad Data from the PAPI Data)
  DK and REF Responses
  CAI PAPI
Drug Type % n % n
Alcohol 3.5 36 7.9 246
Marijuana 3.3 11 4.6 143
Cocaine 4.3 3 2.4 75
Crack 7.1 2 1.9 58
Heroin 8.3 1 2.2 67
Hallucinogens 1.4 1 1.5 47
Inhalants 7.2 6 1.2 56

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

Exhibit 6.5.2 Weighted Percentages of Past 12-Month Users Using 51 Days or More, by Age and Mode of Interview
  ACASI PAPI
Comparison Group
Age/Days of Use

Alcohol

12 to 17 Year Olds    
51 Days or More
34.86 9.83
50 Days or Less
65.14 90.17
18+ Year Olds    
51 Days or More
47.59 41.09
50 Days or Less
52.41 58.91
 

Marijuana

12 to 17 Year Olds    
51 Days or More
41.06 30.82
50 Days or Less
58.94 69.18
18+ Year Olds    
51 Days or More
41.73 33.70
50 Days or Less
58.27 66.30
 

Cocaine

12 to 17 Year Olds    
51 Days or More
28.15 3.62
50 Days or Less
71.85 96.38
18+ Year Olds    
51 Days or More
36.85 6.75
50 Days or Less
63.15 93.25
 

Hallucinogens

12 to 17 Year Olds    
51 Days or More
22.36 13.86
50 Days or Less
77.64 86.14
18+ Year Olds    
51 Days or More
36.94 13.93
50 Days or Less
63.06 86.07
 

Inhalants

12 to 17 Year Olds    
51 Days or More
31.33 14.37
50 Days or Less
68.67 85.63
18+ Year Olds    
51 Days or More
16.07 3.49
50 Days or Less
83.93 96.51

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

 
Exhibit 6.5.4 Number and Percentage of Respondents Selecting Weekly, Monthly, and Yearly Reporting Periods and Distributions of Responses Within Reporting Periods
  Alcohol
  12 to 17 Year Olds 18+ Year Olds
Period Frequency Percent Cumulative
Percent
Frequency Percent Cumulative
Percent
Per Week 95 23.8 23.8 164 28.9 28.9
Per Month 112 29.0 52.8 197 34.7 63.6
Per Year 182 47.2 100.0 207 36.4 100.0
Days Per Week for Weekly Reporters
Days Frequency Percent Cumulative
Percent
Frequency Percent Cumulative
Percent
1 48 52.2 52.2 60 36.6 36.6
2 17 18.5 70.7 39 23.8 60.4
3 15 16.3 87.0 21 12.8 73.2
4 4 4.3 91.3 16 9.8 82.9
5 4 4.3 95.7 16 9.8 92.7
6 1 1.1 96.7 8 4.9 97.6
7 3 3.3 100.0 4 2.4 100.0
Days Per Month for Monthly Reporters
1 25 22.3 22.3 27 13.7 13.7
2 16 14.3 36.6 31 15.7 29.4
3 16 14.3 50.9 38 19.3 48.7
4 8 7.1 58.0 25 12.7 61.4
5 16 14.3 72.3 23 11.7 73.1
6 5 4.5 76.8 6 3.0 76.1
7 4 3.6 80.4 5 2.5 78.7
8 6 5.4 85.7 1 0.5 79.2
9 1 0.9 86.6 1 0.5 79.7
10 5 4.5 91.1 12 6.1 85.8
11-15 4 3.6 94.6 11 5.6 91.4
16-20 4 3.6 98.2 4 2.0 93.4
21-25 0 0.0 98.2 3 1.5 94.9
26-30 1 0.9 99.1 8 4.1 99.0
31 1 0.9 100.0 2 1.0 100.0
Days Per Year for Yearly Reporters
1 36 19.8 19.8 17 8.2 8.2
2 33 18.1 37.9 24 11.6 19.8
3 35 19.2 57.1 26 12.6 32.4
4 9 4.9 62.1 15 7.2 39.6
5 12 6.6 68.7 26 12.6 52.2
6 4 2.2 70.9 16 7.7 59.9
7 2 1.1 72.0 6 2.9 62.8
8 3 1.6 73.6 4 1.9 64.7
9 1 0.5 74.2 2 1.0 65.7
10 12 6.6 80.8 15 7.2 72.9
11-15 6 3.3 84.1 13 6.3 79.2
16-20 13 7.1 91.2 8 3.9 83.1
21-30 8 4.4 95.6 14 6.8 89.9
31-50 2 1.1 96.7 7 3.4 93.2
50+ 6 3.3 100.0 14 6.8 100.0
 
  Marijuana
  12 to 17 Year Olds 18+ Year Olds
Period Frequency Percent Cumulative
Percent
Frequency Percent Cumulative
Percent
Per Week 54 30.2 30.2 40 34.2 34.2
Per Month 54 30.2 60.3 31 26.5 60.7
Per Year 71 39.7 100.0 46 39.3 100.0
Days Per Week for Weekly Reporters
Days Frequency Percent Cumulative
Percent
Frequency Percent Cumulative
Percent
1 16 29.6 29.6 12 30.0 30.0
2 6 11.1 40.7 3 7.5 37.5
3 6 11.1 51.9 5 12.5 50.0
4 5 9.3 61.1 2 5.0 55.0
5 4 7.4 68.5 3 7.5 62.5
6 5 9.3 77.8 5 12.5 75.0
7 12 22.2 100.0 10 25.0 100.0
Days Per Month for Monthly Reporters
1 15 27.8 27.8 5 16.1 16.1
2 10 18.5 46.3 8 25.8 41.9
3 11 20.4 66.7 1 3.2 45.2
4 2 3.7 70.4 2 6.5 51.6
5 2 3.7 74.1 4 12.9 64.5
6 1 1.9 75.9 0 0.0 64.5
7 0 0.0 75.9 1 3.2 67.7
8 0 0.0 75.9 0 0.0 67.7
9 1 1.9 77.8 0 0.0 67.7
10 1 1.9 79.6 2 6.5 74.2
11-15 5 9.3 88.9 4 12.9 87.1
16-20 1 1.9 90.7 2 6.5 93.5
21-25 1 1.9 92.6 0 0.0 93.5
26-30 2 3.7 96.3 2 6.5 100.0
31 2 3.7 100.0 0 0.0 100.0
Days Per Year for Yearly Reporters
1 19 26.8 26.8 9 19.6 19.6
2 6 8.5 35.2 6 13.0 32.6
3 3 4.2 39.4 8 17.4 50.0
4 10 14.1 53.5 4 8.7 58.7
5 6 8.5 62.0 5 10.9 69.6
6 2 2.8 64.8 0 0.0 69.6
7 2 2.8 67.6 0 0.0 69.6
8 2 2.8 70.4 0 0.0 69.6
9 2 2.8 73.2 0 0.0 69.6
10 1 1.4 74.6 6 13.0 82.6
11-15 4 5.6 80.3 2 4.3 87.0
16-20 4 5.6 85.9 0 0.0 87.0
21-30 2 2.8 88.7 2 4.3 91.3
31-50 4 5.6 94.4 1 2.2 93.5
50+ 4 5.6 100.0 3 6.5 100.0
 
  Cocaine
  12 to 17 Year Olds 18+ Year Olds
Period Frequency Percent Cumulative
Percent
Frequency Percent Cumulative
Percent
Per Week 6 30 30 6 17.1 17.1
Per Month 5 25.0 55 11 31.4 48.6
Per Year 9 45 100.0 18 51.6 100.0
Days Per Week for Weekly Reporters
Days Frequency Percent Cumulative
Percent
Frequency Percent Cumulative
Percent
1 3 5.0 50.0 2 33.3 33.3
2 2 33.3 83.3 0 0.0 33.3
3 0 0.0 83.3 1 16.7 50.0
4 0 0.0 83.3 1 16.7 66.7
5 0 0.0 83.3 2 33.3 100.0
6 0 0.0 83.3 0 0.0 100.0
7 1 16.7 100.0 0 0.0 100.0
Days Per Month for Monthly Reporters
1 1 20.0 20.0 0 0.0 0.0
2 1 20.0 40.0 2 18.2 18.2
3 0 0.0 40.0 3 27.3 45.5
4 1 20.0 60.0 1 9.1 54.5
5 0 0.0 60.0 1 9.1 63.6
6 0 0.0 60.0 0 0.0 63.6
7 0 0.0 60.0 1 9.1 72.7
8 0 0.0 60.0 0 0.0 72.7
9 0 0.0 60.0 0 0.0 72.7
10 0 0.0 60.0 0 0.0 72.7
11-15 1 20.0 80.0 1 9.1 81.8
16-20 0 0.0 80.0 1 9.1 90.9
21-25 1 20.0 100.0 1 9.1 100.0
26-30 0 0.0 100.0 0 0.0 100.0
31 0 0.0 100.0 0 0.0 100.0
Days Per Year for Yearly Reporters
1 5 55.6 55.6 3 16.7 16.7
2 3 33.3 88.9 4 22.2 38.9
3 0 0.0 88.9 2 11.1 50.0
4 0 0.0 88.9 4 22.2 72.2
5 0 0.0 88.9 1 5.6 77.8
6 0 0.0 88.9 0 0.0 77.8
7 0 0.0 88.9 0 0.0 77.8
8 0 0.0 88.9 0 0.0 77.8
9 0 0.0 88.9 0 0.0 77.8
10 0 0.0 88.9 0 0.0 77.8
11-15 0 0.0 88.9 0 0.0 77.8
16-20 0 0.0 88.9 1 5.6 83.3
21-30 0 0.0 88.9 0 0.0 83.3
31-50 0 0.0 88.9 1 5.6 88.9
50+ 1 11.1 100.0 2 11.1 100.0

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.

6.6 Comparison of ACASI Respondent Attitudes to Attitudes of PAPI Respondents: Results from Debriefing Interviews

All ACASI respondents in the 1997 field experiment were asked to complete a debriefing questionnaire at the end of the interview. ACASI was used so that respondents could answer the debriefing questions privately. A subsample of the respondents in Quarter 4 of the 1997 NHSDA also completed a debriefing interview using ACASI methodology. Using ACASI to answer debriefing questions gave the comparison group some minimal experience with answering questions using the computer and allowed us to query them as to their preferences for using a computer versus paper-and-pencil methods and about the relative privacy of the two modes of interview.

Confirming the results from the 1996 feasibility experiment, the debriefings clearly indicate that ACASI provides a more private setting for the interview and that most respondents prefer to use the computer. This is particularly true for 12 to 17 year olds.

6.6.1 Participation in the Debriefing Interviews

A total of 1,953 (99%) of the respondents in the 1997 field experiment completed the debriefing interview. We obtained debriefing interviews from 584 of the respondents in Quarter 4 of the 1997 NHSDA. A total of 713 respondents were selected for these interviews; the interviewers neglected to request participation in the ACASI debriefing interview from 25 respondents (4%); among the 688 respondents who were asked to complete the debriefing interview, 584 (85%) agreed to complete the interview. Exhibit 6.6.1 summarizes the characteristics of the debriefing respondents.

The CAI respondents answered 25 debriefing questions, and the PAPI/SAQ respondents answered 21. Eighteen of the questions were nearly identical in the two debriefing interviews in that they addressed the same topic for the two different interview modes.

6.6.2 Comparison of Respondent Attitudes

In Exhibits 6.6.2 through 6.6.22, we present unweighted tabulations of the debriefing responses for the 18 questions that were answered by both the ACASI and PAPI/SAQ respondents. Some of these exhibits present comparisons of debriefing interview respondents and interviewer debriefing responses.

Ability to complete the response task. Respondents in the 1997 Quarter 4 NHSDA debriefing sample were asked to rate the difficulty of recording their answers on the answer sheets; those in the 1997 field experiment were asked to rate the difficulty of using the computer. In both cases, the large majority of the respondents indicated that the task was not at all difficult (see Exhibit 6.6.2). An estimated 86% of the respondents who completed PAPI/SAQ answer sheets indicated that the task was not at all difficult, and there was a slight tendency for more of the ACASI respondents to indicate that the task was not at all difficult (91.4%). There was a larger difference between the two modes for the youths (aged 12 to 17); the corresponding percentages are 83.6% for those completing answer sheets and 91.4% for those using the computer. Among the adult respondents aged 18 or older, those with less than a high school education had a tendency to report more difficulty with the response task, and this was true for both interview modes.

Larger differences between the two interview modes were observed in response to the question concerning whether or not the respondent was able to complete the self-response task without the help of the interviewer. Data are shown in Exhibit 6.6.3. Among the ACASI respondents, 88.3% indicated that they did not need help, whereas only 73.5% of those who filled out the answer sheets indicated that they did not need help from the interviewer to do so. This difference was even larger for the youths, with 20% fewer of the ACASI respondents indicating that they needed help from the interviewer (33.2% for PAPI vs. 13.2% for ACASI). Again, among the adult respondents, those with less than a high school education were more likely to indicate that they needed help from the interviewer, and fewer of the ACASI respondents needed help.

One of the reasons for using ACASI is to ease the response task for poor readers. The results of the debriefing interviews indicated that ACASI did make it easier for poor readers to respond. To investigate this, we first asked respondents to rate their own reading ability. About the same proportion of people in the experimental and comparison group classified themselves as having fair to poor reading ability (15.2% and 16.8%, respectively). We also asked those who had received an ACASI interview how much the recorded voice helped them and how often they listened to it while completing the interview.15

Respondents who reported fair to poor reading ability were more likely to find the audio helpful and to listen to it during the interview (Exhibit 6.6.4). About 49% of the ACASI respondents reporting fair or poor reading abilities indicated that the recorded voice helped them a lot to understand the questions, whereas only about 15% of ACASI respondents reporting excellent reading abilities indicated that the recorded voice was of some help in understanding the questions. Among the ACASI respondents reporting fair or poor reading abilities, about 65% indicated that they listened to the recorded voice most of the time as compared to 37% of the ACASI respondents reporting excellent reading abilities (Exhibit 6.6.5).

We were concerned that respondents who did not routinely use computers would not like using them to complete the interview. To investigate this issue, we asked the comparison group who had used paper answer sheets to indicate whether they would rather use the computer or the paper answer sheets.

About 37% of the respondents who reported that they had not used the computer prior to the interview indicated that they would rather use the computer (Exhibit 6.6.6), and 32% of them indicated that it does not make any difference. ACASI respondents had no experience with the paper-and-pencil answer sheets (PAPI/SAQ); thus, we were not able to examine their preferences for an interview mode. We did have information, however, on their need for help and examined how this related to their computer experience. About 77% of ACASI respondents who reported they were using a computer for the first time indicated that they could easily enter their answers into the computer without asking the interviewer for help (Exhibit 6.6.7). An estimated 92% of the everyday users required no help.

Privacy of the interviewing environment. ACASI was designed to increase the privacy of the interviewing environment. Dramatic differences between the two modes of data collection were observed in response to the question on how many of the responses the interviewer saw (Exhibit 6.6.8). Only 41.3% of those completing answer sheets indicated that the interviewer saw none of their answers, whereas 82.6% of the ACASI respondents indicated such. In addition, under the paper-and-pencil mode, 42% of respondents indicated that the interviewer saw some of their answers, but only 13% of the ACASI respondents indicated such. This finding supports results from the 1996 feasibility experiment. In that experiment, we did not debrief respondents using a structured questionnaire but did ask interviewers to report, for each respondent, how often the respondent let them know his or her answer. In that study, interviewers indicated that they were aware of none of the respondent's answers in only 38% of the PAPI/SAQ cases, and for the two ACASI versions, the corresponding percentages were 63.7% and 73.8% for the MIRROR and the SKIP versions.

We also examined how the seating arrangement affected the privacy of the interview (Exhibit 6.6.9). About 73% of the ACASI respondents indicated that an interviewer seated next to them with no table space did not see any of their answers; about 84.2% indicated that the interviewer seated across with no table space did not see any of the answers. This indicates that the survey protocol still needs to stress obtaining a private setting, including making sure that the interviewer is not able to observe the respondent's answers.

Respondents were asked how many of their answers they thought that someone in their household other than the interviewer saw (see Exhibit 6.6.10). Under both modes, a large majority of the respondents indicated that no one other than the interviewer saw their answers (78.3% for those who used answer sheets vs. 83.6% for those using the computer). However, a more than desirable proportion of the respondents indicated that someone in the household saw some of their answers. For adult respondents, about 10% of those completing answer sheets indicated that someone else had seen their answers, and 7.5% of the ACASI respondents reported this as well. For the youths, the figures were even higher (21.2% for the PAPI; 18.9% for the ACASI).

Respondents also were asked how important it was to them that the interviewers and other household members not see their answers. Results for these two questions are presented in Exhibits 6.6.11 and 6.6.12. In both cases, the ACASI respondents were more likely to indicate that it was very important that their answers be concealed from either the interviewers or other household members. However, only about 26% of respondents indicated that this was very important. When asked about the importance of concealing their answers from the interviewer, nearly 59% of those completing the answer sheets reported that it was not at all important; the corresponding figure for the ACASI respondents was about 49%. Adults in both settings were more likely to say that it was not important to conceal their answers from the interviewer when compared to youths. Similar results were observed when respondents were asked about concealing their responses from other household members. Adults were less concerned than youths with concealing their answers from other household members, and ACASI respondents were somewhat more likely to report that it was very important to conceal the answers from other household members.

Exhibit 6.6.13 displays information on the interviewing environment and the respondents' reports as to whether or not someone in the household saw their answers. It is encouraging to note that among the 240 youths for whom a parent was present during the interview, 73% reported that none of their answers was seen by another household member. The fact that parents saw answers, however, in over 25% of the cases is a problem. The 240 youths from whom parents were present comprise 22% of the youth respondents. Overall, there was someone present during the interview for 497 (45%) of the youths. For 30% of these youths, at least someone in the household saw some of their answers. We note similar results for the adults. Exhibit 6.6.14 indicates that respondents for whom someone saw their answers find privacy just as important as those for whom no one saw their answers.

This series of exhibits has some worrisome findings. It indicates that respondents want a private setting for the interview and that this is not always achieved. The ACASI technology has the potential to achieve greater privacy; however, the interviewer also has some control over this. During interviewer training and supervision, the need for privacy needs to be continually emphasized.

Members of the 1997 Quarter 4 NHSDA debriefing sample were asked which method of completing the survey was best for protecting their privacy while they were completing the survey (see Exhibit 6.6.15). About 11% of these respondents indicated that the answer sheets were the best; 57.2% of youths and 42.5% of adults felt that the computer was best; and 23.3% and 28.4% of youths and adults, respectively, felt that the two methods were equal in protecting privacy. It is interesting to note that overall some 13% of the respondents felt that neither protected their privacy while they were completing the interview (9.2% of youths vs. 16.8% of adults).

Similar results (see Exhibit 6.6.16) were observed in response to the question on protection of privacy after the interviewer had left the household. Around 12% reported feeling that the answer sheets provided the best protection after the interviewer had left the household; 46.6% chose the computer; and 24.5% indicated that the methods were equal. The percentage reporting that neither protected their privacy after the interviewer left their home was around 16%, with adults slightly more likely than youths to report neither. These percentages are larger than those who reported that they were not confident that their results will not be linked with their names, indicating that respondents may be confident of the promises of confidentiality.

Respondents who completed the ACASI interview had no experience completing the interview using answer sheets. They were told, however, that in some cases people recorded their answers on answer sheets and sealed them in an envelope. They were asked which method they felt was most private. The responses are very similar (see Exhibit 6.6.17) to those of the 1997 NHSDA debriefing sample, with about 13% saying that the paper would be less private. About 47% chose the computer, and 39% said that there was no difference. Again, youths were more likely than adults to choose the computer.

Accuracy of responses. Two questions were asked about the need for accuracy and the overall accuracy of responses. In Exhibit 6.6.18, we compare the responses to the question on their opinions as to how accurate the sponsor wanted them to be. In both samples, about 80% of respondents thought that very accurate data were needed, and adults were more likely than youths to think that the accuracy was needed. Exhibit 6.6.19 presents respondents' ratings of the accuracy of their responses. The vast majority said that they were very accurate (80.8% of the 1997 Quarter 4 respondents vs. 83.1% of the 1997 field experiment respondents). Very small minorities indicated that their responses were not very accurate.

Interest in the interview and willingness to respond again. The ACASI respondents were more likely to say that the interview was very or somewhat interesting (55.3% vs. 46.9%). These results are presented in Exhibit 6.6.20. In both samples, adults found it more interesting than youths; however, the difference between the ACASI and PAPI respondents was greater for youths than adults. Among youths, 51.6% of ACASI respondents reported that the interview was somewhat or very interesting, and only 39.7% of those who completed the answer sheets gave such ratings. The corresponding percentages for adults were 60.2% and 54.1%. Adults with less education were most likely to find the interview interesting (35.5% of the ACASI respondents with less than a high school education found the interview interesting vs. 22.5% of the adults with a high school education and 11.8% of those with more than a high school education).

Most respondents said that they were very or somewhat likely to complete a similar interview if asked, and those responding by computer were slightly more likely to report willingness to participate in the future (79.9% vs. 75.1%). Youths were about 4% to 5% more likely than adults to say that they would do it again (Exhibit 6.6.21).

Under both modes, the majority of respondents were either very or somewhat confident that their answers would never be linked with their names (Exhibit 6.6.22). There was a small difference between the two samples in the percentage who indicated that they were not at all confident, with 8.7% of the 1997 NHSDA Quarter 4 debriefing respondents being not at all confident and 5.6% of the ACASI respondents.

 

 
Exhibit 6.6.1 Characteristics of Debriefing Interview Respondents
  1997 Field Experiment Comparison Group
Respondent Characteristics CAPI/ACASI 1997 Quarter 4
PAPI/SAQ
n % n %
Total 1,953 100 584 100
Age Group
12-17
1,102 56.4 292 50.0
18+
851 43.6 292 50.0
Gender
Male
915 46.9 246 42.1
Female
1,038 53.1 338 57.9
Race/Ethnicity
Hispanic
463 23.7 121 20.7
Non-Hisp., Black
521 26.7 193 33.0
Non-Hisp., All Other Races
969 49.6 270 46.2
Education1
< High School
197 23.1 47 16.1
High School
316 37.1 100 34.2
> High School
338 39.7 145 49.7

1Education includes only individuals aged 18 or older.

Sources: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment. 1997 National Household Survey on Drug Abuse: Quarter 4.





Exhibit 6.6.4 Comparison of Assistance Provided by the Recorded Voice, by Respondent's Rating of Reading Ability (Unweighted Percentages)
  1997 Field Experiment
  CAPI/ACASI
 

Some people believe that having a recorded voice read the questions will help respondents understand the questions better. How much did the recorded voice help you to understand the questions?

Reading Ability No Help Some Help A Lot of Help Did Not
Listen
DK/REF
Total 35.3 31.6 24.6 8.4 0.1
Excellent 46.0 26.5 14.7 12.8 0.0
Good 30.1 37.1 26.6 5.7 0.1
Fair/Poor 16.8 31.9 48.7 2.7 0.0
DK/Ref 25.0 50.0 0.0 0.0 25.0

Source: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment.

 

Exhibit 6.6.5 Use of Recorded Voice, by Respondent's Rating of Reading Ability (Unweighted Percentages)
  1997 Field Experiment
  CAPI/ACASI
  Did you listen to the recorded voice most or all of the time, some of the time, or little or none of the time?
Reading Ability Most of the Time Some of the Time Little or None
of the Time
DK/REF
Excellent 36.5 19.5 44.0 0.1
Good 48.7 26.7 24.3 0.1
Fair/Poor 64.8 22.5 12.8 0.0
DK/Ref 25.0 25.0 25.0 25.0

Source: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment.

 

Exhibit 6.6.6 Respondent's Preference to Use Computers or Answer Sheets, by Respondent's Computer Experience (Unweighted Percentages)
  Comparison Group
  1997 Field Experiment
 

Would you rather use the computer, fill out the answer sheet or wouldn't it matter to you?

Computer Experience Would Rather
Use the
Computer
Would Rather
Fill Out
Answer Sheet
Doesn't Make
Any Difference
DK/REF
First-Time User1 36.5 31.8 31.8 0.0
         
Prior User2
   Currently not using 46.2 3.9 50.0 0.0
   Less than once a month 39.2 20.0 41.2 0.0
   One to a few days a month 55.4 18.5 26.1 0.0
   One to four days a week 47.9 9.4 42.7 0.0
   Everyday 59.4 6.8 33.8 0.0
         
DK/REF 0.0 0.0 0.0 100

1Respondents who answered "yes" when asked, "Is this the first time you have ever used a computer?"

2Respondents who answered "no" when asked, "Is this the first time you have ever used a computer?"

Source: 1997 National Household Survey on Drug Abuse; Quarter 4.

 

Exhibit 6.6.7 Respondent's Ability to Enter Answers into the Computer Without the Interviewer's Help, by Respondent's Computer Experience (Unweighted Percentages)
  1997 Field Experiment
  CAPI/ACASI
 

Were you able to enter answers into the computer easily, without having to ask the interviewer for help?

Computer Experience Yes No
First-Time User1 76.7 23.2
Prior User2
Currently not using
82.8 17.2
Less than once a month
90.4 9.6
One to a few days a month
89.2 10.8
One to four days a week
87.5 12.5
Everyday
91.7 8.3
DK/REF 71.4 28.6

1Respondents who answered "yes" when asked, "Is this the first time you have ever used a computer?"

2Respondents who answered "no" when asked, "Is this the first time you have ever used a computer?"

Source: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures: 1997 Field Experiment.

 



 
Exhibit 6.6.9 Comparison of Debriefing Interview Respondents to Interviewer Debriefing Responses: Seating Arrangement Versus Answers Interviewer Saw (Unweighted Percentages)
  1997 Field Experiment
  CAPI/ACASI
Place Where the Interviewer Was Seated During ACASI According to the Interviewer

How many of the answers that you entered into the computer do you think the interviewer saw?

None of the Answers Some of the Answers A Lot of the Answers All of the Answers DK/REF
Seated Next to the Respondent with Table Space Available 79.3 14.7 2.8 2.8 0.5
Seated Next to the Respondent with No Table Space Available 72.9 20.8 2.1 3.5 0.7
Seated Across from Respondent with Table Space Available 86.3 10.8 1.3 1.1 0.5
Seated Across from Respondent with No Table Space Available 84.2 11.2 2.0 2.0 0.7
Some Other Arrangement 83.1 13.2 2.1 1.1 0.5

Source: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1997 Field Experiment.

 
Exhibit 6.6.15 Comparison of Debriefing Interview Respondents on Selected Debriefing Questions: Privacy Protection (Unweighted Percentages)
  Comparison Group
  1997 Quarter 4 NHSDA
 

Which method do you think is best for protecting your privacy while completing the survey?

Respondent Characteristics Computer Answer
Sheets
Equally
Well
Neither DK/REF
Total 49.8 11.0 25.9 13.0 0.4
Age Group
12-17
57.2 10.3 23.3 9.2 0.0
18+
42.5 11.6 28.4 16.8 0.6
Gender
Male
51.2 10.2 23.6 14.6 0.4<
Female
48.8 11.5 27.5 11.8 0.3
Race/Ethnicity
Hispanic
52.9 10.7 25.6 10.7 0.0
Non-Hisp., Black
51.8 13.5 19.2 15.0 0.5
Non-Hisp., All Other Races
47.0 9.3 30.7 12.6 0.4
Education1
< High School
46.8 12.8 23.4 14.9 2.1
High School
43.0 13.0 29.0 15.0 0.0
> High School
40.7 10.3 29.7 18.6 0.7

1Education includes only individuals aged 18 or older.

Source: 1997 National Household Survey on Drug Abuse: Quarter 4.

12 More detailed tables by treatment, age group, gender, race/ethnicity, and education can be found in Chapter 7.

13 See Chapter 4 for a discussion of the cognitive laboratory testing of the 12-month frequency of use question.

14 If asked, interviewers are trained to tell the respondent to record "DK" or "REF" in the right margin next to the particular item.

15 In the ACASI application used, respondents could turn off the voice if they wished.

Top Of PageTable of Contents

This page was last updated on June 16, 2008.