Skip To Content

Table Of Contents

Click for DHHS Home Page
Click for the SAMHSA Home Page
Click for the OAS Drug Abuse Statistics Home Page
Click for What's New
Click for Recent Reports and HighlightsClick for Information by Topic Click for OAS Data Systems and more Pubs Click for Data on Specific Drugs of Use Click for Short Reports and Facts Click for Frequently Asked Questions Click for Publications Click to send OAS Comments, Questions and Requests Click for OAS Home Page Click for Substance Abuse and Mental Health Services Administration Home Page Click to Search Our Site

Computer Assisted Interviewing for SAMHSA's National Household Survey on Drug Abuse

12. Refinement of Procedures: 1998 Laboratory and Field Testing and Final 1999 NHSDA CAI

The research conducted in 1996 and the 1997 field experiment and its subsequent analysis in early 1998 provided most of the information that was used to make decisions on the structure and content of the 1999 NHSDA. However, several issues remained and some new ones arose that required additional testing in the summer of 1998. Because the electronic screener application that was tested in the 1997 field experiment had not included the case management system (CMS) on the Newton, this component of the application needed to be tested under field conditions. This testing was done during the August 1998 field test and is described in Chapter 8.

The main reason that additional testing was needed was that the Department of Health and Human Services (DHHS) decided that a special youth tobacco module should be included in the 1999 NHSDA and that this module needed to be administered using audio computer-assisted self-interviewing (ACASI). This decision was made in the late spring of 1998 to address data requirements in proposed legislation related to tobacco use. Before this decision was made, SAMHSA had been planning to phase in the CAI implementation over a two-year period with the 1999 NHSDA conducted using PAPI methodology. The decision to include the new tobacco module in 1999 made it necessary to conduct the entire 1999 NHSDA using CAI. This required (a) developing and testing a new tobacco module that would provide the data required by the legislation and (b) making decisions as to the structure and content of the remainder of the CAI application. To investigate the former, we conducted a series of laboratory studies that focused on the ability of youths to complete a new tobacco module that asked about brand of tobacco use. To assist in decisions concerning the remainder of the application, timing tests of a draft of the instrument were conducted, and a version of the interview was field tested in August 1998.

12.1 Development and Testing of the Tobacco Module

The tobacco module was to be designed to provide more detailed information on the use of tobacco products by youths, including the brands that they used. Although the major interest was in the use of tobacco by youths, SAMHSA decided that the detailed tobacco module would be administered to all respondents and that it would become the tobacco component of the core NHSDA interview. Thus, the Research Triangle Institute (RTI) was instructed to develop this module as an ACASI interview and to do this very quickly. In addition, because the main concern was that the module be designed so that youths could complete it, the testing and development focused on youths rather than adults.

Several activities were undertaken to develop the module, including (a) a review of the suggested content and wording by an expert panel, (b) two rounds of cognitive laboratory testing, and (c) testing of the module in the August 1998 NHSDA field test.

12.2 Tobacco Module Development and Evaluation: Overview

12.2.1 Content of the Tobacco Module

To have time for testing, the content and basic approach of the module needed to be drafted very quickly. Based on a meeting between RTI and SAMHSA staff on June 8, 1998, an initial draft of the questionnaire was completed on June 10, 1998, and circulated for review. A conference call was held on June 16, 1998, and comments were gathered. Additional comments were gathered by e-mail, and the final changes for the initial round of laboratory testing were made on June 22, 1998.

The draft tobacco module included separate questions on cigarettes, chewing tobacco, snuff, cigars, and pipe tobacco. The cigarette section included questions on a variety of topics:

  1. ever smoking in one's lifetime;

  2. likelihood of initiating smoking for those who had never smoked;

  3. smoking within the past 30 days;

  4. recency of use for those who reported lifetime use but no use in the past 30 days;

  5. days smoked in the past 30 days for 30-day smokers;

  6. average number smoked per day on days smoked in the past 30 days;

  7. brand of cigarette smoked most often in the past 30 days, which was presented as a pick-list;

  8. open-ended question on brand for those who had used a brand not on the pick-list;

  9. verification question on brand chosen;

  10. use of roll-your-own-tobacco cigarettes;

  11. daily smoking for 30 days or more; and

  12. age when daily smoking began.

The cigarette questions were followed by an introduction that distinguished between chewing tobacco and snuff. This was followed by a series of questions on lifetime use of chewing tobacco, age at first use, use in the past 30 days, recency of use, days used in the past 30 days, brand used most often during the past 30 days, and verification ofthe reported brand. Similar questions were then asked for snuff and cigars. The module ended with two questions on using a pipe to smoke tobacco.

12.2.2 Expert Review of Tobacco Module Questions

A panel of seven independent consultants who are experts on measurement of cigarette use and selected substance use experts at RTI were asked to comment on the reliability and validity of the tobacco use questions and to make suggestions for improving the tobacco module. These expert opinions were documented in a memorandum that was submitted to SAMHSA and discussed by DHHS staff (including representatives from the DHHS Office of the Assistant Secretary for Planning and Evaluation [ASPE], the Centers for Disease Control and Prevention's [CDC's] Office on Smoking and Health [OSH], the CDC's National Center for Health Statistics [NCHS], the National Cancer Institute [NCI], and SAMHSA) in a meeting convened on July 2, 1998. At that meeting, the comments were discussed and decisions were made as to the wording that was to be used in the August testing.

The expert reviewers were generally positive about the questions. Although several reviewers suggested additional questions and editorial changes, no reviewer indicated that the proposed questions failed to meet current standards in terms of reliability and validity. The experts suggested additional questions that they thought would improve the utility of the NHSDA to provide information on the epidemiology of tobacco use by youths:

  1. more detailed information on the initiation of smoking,

  2. smoking different brands of tobacco products,

  3. versions of brand smoked (e.g., Regular, Lite, Ultra Lite),

  4. issues of youths' access to tobacco products, and

  5. "blunting" (joint use of tobacco and other drugs, such as smoking marijuana in a cigar wrapper).

Because of the need to limit the overall length of the NHSDA interview, additional questions were not added for the 1999 NHSDA.

The performance of the tobacco module was evaluated by means of both laboratory and field testing. Two rounds of cognitive interviews were conducted in RTI's Laboratory for Survey Methods and Measurement. A number of cognitive interviews were also conducted outside of laboratory locations in order to interview hard-to-locate Spanish-speaking respondents. Field testing was done as part of the August 1998 field test.

12.2.3 Programming the Tobacco Module

Programming began on June 12, 1998, and WAV files for the laboratory version were recorded between June 18 and June 22, 1998. We tested the ACASI program from June 18 to 22, 1998, and the final version tested in Round 1 was fully programmed on June 24, 1998.

12.3 Tobacco Module: Round 1 Laboratory Testing (Cognitive Issues)

The new questions went beyond the previous core tobacco questions by asking respondents to identify and select the brands of tobacco products they used. Laboratory testing focused on the ability of respondents to use the computer to select the brand of cigarettes or other tobacco products that they used as well as their ability to answer the other tobacco use questions. The testing focused on the following issues:

  1. Comprehension: Is the language of the questionnaire simple and easy to understand?

  2. Usability: Do respondents understand how to use the computer to complete the questionnaire (entering/changing answers, backing up, etc.)?

  3. Brand Identification: Can respondents consistently choose the brand of cigarettes/tobacco products that they use? Can they distinguish among brands? Do the questions enable respondents to describe their smoking preferences and choices? What is the best way to array lists of brands so that respondents can quickly identify the one they use?

  4. Smokeless Tobacco: Can respondents distinguish between chewing tobacco and snuff? Do the terms and descriptions provided in the questionnaire adequately define smokeless tobacco products? Are brand identification issues the same for smokeless tobacco as for cigarettes?

Respondents for the first round of laboratory testing were recruited by posting flyers at various locations in the Triangle area of North Carolina, including Raleigh, Durham, Cary, and Chapel Hill. Flyers were posted at locations where teenagers congregate during summertime (parks, recreation centers, shopping malls, restaurants and well-known spots for "hanging out"). The flyers directed interested volunteers to call in to enlist as participants. Upon calling, the laboratory interview procedures were explained to potential respondents, preliminary interview times were discussed, and the names and phone numbers of parents were elicited. After parental permission was obtained in a separate phone call made by RTI staff, final scheduling of respondentsvisits was confirmed.

A laboratory interview protocol was designed to elicit information from respondents about key cognitive issues relating to understanding and usability. The protocol called for using think-aloud interviewing techniques. Specific probe questions designed to elicit information about the above topics were asked, and answers were noted by interviewers on a paper copy of the interview as the respondent used the computer. RTI's Institutional Review Board (IRB) reviewed the laboratory procedures, as well as respondent recruiting methods, to make sure that respondents were able to give their informed consent to participate and that parental permission was adequately obtained for interviewing minor respondents.

A total of 10 respondents participated-5 males ranging in age from 13 to 17 years old and 5 females ranging in age from 14 to 17 years old.

The results clearly indicated that respondents were able to identify the brands of cigarettes, cigars, and (in one case) smokeless tobacco using the brand identification pick-list that was programmed. Respondents selected the brand of tobacco product they used most often from an alphabetical list of brand names. Separate lists were used for each type of tobacco product (i.e., cigarettes, chewing tobacco, snuff, and cigars). Respondents all found the lists easy to use; the alphabetical order made it easy for them to quickly find the brand they were looking for. Respondents were also able to type in names of brands not listed.

The only difficulty that the respondents had selecting the brand they used most often arose not from the way in which the question was asked but from their uncertainty about which brand was used most. Youths who borrowed cigarettes from others were not always certain what brands they were given. The only problem identified with the listing was that respondents sometimes failed to see the "some other brand" option and, instead, chose the brand among those presented that they used most. However, when the option was pointed out, they were able to use it correctly. This was later revised to make the response option clearer.

Youths found the computer screen easy to read and had no difficulty with listening to audio files or using the keyboard. They had little difficulty answering questions about how many of the past 30 days they smoked and the average number of cigarettes they smoked on those days.

Other surveys had used the term "smokeless tobacco" to refer to both chewing tobacco and snuff. The respondents were queried about this term, and many of them expressed uncertainty or confusion when asked what the term meant. They either did not recognize the term or else considered it to apply to part of the general category. They had no difficulty with the definitions of chewing tobacco and snuff that were provided in the questionnaire.

12.4 Tobacco Module: Round 2 Laboratory Testing

A second round of cognitive interviews with teenagers was planned for August 1998. Round 2 testing was aimed at checking the effectiveness of several refinements in the wording, attempted to include more youths who had used smokeless tobacco, and included a Spanish version of the module.

12.4.1 Changes in Content of the Tobacco Module from Round 1

The tobacco module tested was similar to that used in the first round with a few changes:

  1. Additional brands of cigarettes and cigars were presented, and there was a change in the way in which cigarette brand names were presented. Two alphabetical lists of brand names were presented instead of the one that was used in the prior round of testing. The second list of more rarely used brands was presented to respondents who did not identify their most used brand in the first list. As was done in Round 1, respondents whose cigarette brand was not found on either the first or second list were asked to type in the brand name.

  2. At the end of all brand name lists, the last choice was changed from "some other brand" (Round 1 wording) to "a brand not on this list."

  3. An additional question about smoking 100 cigarettes in one's lifetime was asked.

  4. Respondents who gave an age of initiation for any tobacco product that was the same or 1 year less than their present age were asked the year and month of first use.

  5. The definition of snuff was presented a second time for respondents who answered questions about chewing tobacco, prior to asking about snuff use. For respondents who did not answer chewing tobacco questions, the definition of snuff was not repeated.

  6. A Spanish-language version was developed.

12.4.2 Round 2 Research Design

Cognitive issues for Round 2. We examined several questions during Round 2:

  1. Was the modified method of presenting cigarette brands usable? The modified method involved using two lists of cigarettes, the second one shown if the target brand was not on the first list, followed by a provision to type in any unlisted brand using the keyboard.

  2. Was the redefinition of snuff after answering questions about chewing tobacco helpful?

  3. Was the language of the tobacco module questions understandable? Specifically, how the phrase "all or part of a cigarette" was interpreted received specific attention, as did the phrase "on the days that you smoke." Although not part of the tobacco module wording, respondents were also asked how they would interpret a question reading "In the past 30 days, have you smoked any cigarettes?"

  4. Could respondents read and understand the computer screen, keyboard, and sound equipment?

Round 2 data collection procedures. The data collection plan for Round 2's laboratory interviews called for interviewing 50 youths: 40 in English and 10 in Spanish. Of the 40 English interviews, 10 were to use smokeless tobacco (in the past 30 days, if possible, but recently enough to be able to identify the brand they used) and 10 respondents who were not tobacco users. Additionally, half were to be with youths 12 to 14 years old and half 15 to 17 years of age.

Recruiting and cognitive interview procedures for Round 2 were the same as those for Round 1, except for the Spanish interviews. These interviews took place either in respondents' homes or at a local Hispanic cultural fair. Although the original research plan called for interviewing only teenagers (12 to 17 year olds), several adults were interviewed in order to expedite the completion of enough interviews to make recommendations. Furthermore, only 4 of the 10 interviews were done using the ACASI computerized interviewing method. Because the remaining interviews were conducted at a fair, it was inconvenient to use the computer in that setting. However, because the primary interest in the Spanish version of the questionnaire involved the wording of the questions, rather than use of the computer, we felt that valuable information could still be gained without the computer. These remaining respondents were interviewed using a translated version of the latest tobacco module.

Recruiting of youth respondents for Round 2 proved more difficult than anticipated and more difficult than for the Round 1 testing. It may have been that the beginning of the school year, in the middle of August, made students unsure of their schedules and unwilling to devote their free time to something that sounded like work. Our original plan was to interview 50 youths, but we only interviewed about two thirds of that number.

12.4.3 Round 2 Testing Results

A total of 36 respondents who met the criteria of age and parental consent were interviewed. English- language respondents were 8 females and 18 males ranging in age from 13 to 17. Three respondents (all males) between the ages of 12 and 14 were interviewed, as well as five 15 year olds. An additional 10 respondents completed a Spanish-language version of the cognitive interview. The age range for Spanish interviews was from 12 to 57, with 8 of the 10 interviews being conducted with respondents in the target age range.

Brand identification. The results clearly indicated that respondents were able to identify the brands of cigarettes, cigars, and smokeless tobacco using the brand identification questions programmed for the youth tobacco module. The English and Spanish versions of these questions performed equally well. These questions were asked by having respondents select the brand of tobacco product they used most often from one of two alphabetical lists of brand names of cigarettes, and single lists for chewing tobacco, snuff and cigars (separate lists for each type of tobacco product). Brand names for the Spanish instrument were not translated, although the ACASI recordings were made using Spanish-accented pronunciation.

Respondents all found the lists easy to use. The alphabetical order made it easy for them to quickly find the brand they were looking for. Respondents were also able to type in names of brands not listed. Respondents who did not smoke cigarettes, or who had only smoked one or two, were as easily able to locate specific brand names as were regular smokers. Again, the only difficulty respondents had selecting a brand stemmed from the uncertainty about which brand was used most among the youths who borrowed cigarettes from others.

Those youths who had experience with smokeless tobacco found identifying brands of those products just as easy as identification of cigarettes and cigars.

Wording and terminology.

  1. The change of "some other brand" to "a brand not on this list" seemed to make the intention of that response option clearer.

  2. The wording "all or part of a cigarette" was almost uniformlyinterpreted as referring to any level of smoking beyond passive second-hand smoke. However, the wording "... smoke any cigarettes" was interpreted by a substantial minority as not referring to smoking only part of a cigarette.

  3. The restatement of the definition of snuff drew mixed reviews. Although some respondents were not bothered by it, a couple said it was too long and the restatement might be shortened and emphasize only the differences from chewing tobacco.

  4. The question of smoking 100 cigarettes in a lifetime was easy to answer by our respondents who had either smoked for a long time or else were non-smokers, or youths who had only smoked one or two times.

Several of the Spanish translations of tobacco-related terms were not well understood.

  1. In Spanish, cigarro can mean cigar or cigarette (also called cigarillo, which in English is a form of small cigar). Central and South Americans preferred the term puro to cigarro for a cigar. Puro means "pure," referring to the fact that cigars are made completely out of tobacco, rather than tobacco wrapped in paper, like cigarettes. Puerto Ricans preferred cigarros to puros. Clearly, both terms (along with appropriate definitions) are needed in referring to cigars and cigarettes in the general population.

  2. The concepts of plug and twist of chewing tobacco (translated literally as tapado and atado in the Spanish translation) were not understood either by younger respondents or adult smokers (some of whom had experience with chewing tobacco). A suggested solution would be to leave the English words in quotation marks and simply describe, in Spanish, the look of a plug/twist. An alternative would be to provide a picture of these forms of chewing tobacco along with the definitions.

Use of the computer. Respondents had no difficulty using the computer either for categorical answers or for typing in numerical or text answers. Respondents also had no difficulty providing the year and month of first use of tobacco products.

12.5 Tobacco Module: Discussion of Laboratory Results

The results of the laboratory testing clearly indicated that youths were able to answer questions about their own tobacco use using the tobacco module. Most could identify all the brands they had ever smoked and even brands that they never smoked (brands used by friends or brands seen in ads). This finding was clearly evident even though the target number of respondents was not reached.

Identifying smokeless tobacco brands was no different than for cigarette or cigar brands. However, respondents were not always correct in identifying a smokeless tobacco product as chewing tobacco vs. snuff. One respondent thought Kodiak was a brand of chewing tobacco and entered it by typing in the chewing tobacco section. However, when he completed the snuff section, he found it again. That made it somewhat confusing to answer the final smokeless tobacco question about which was used more: the brand of chewing tobacco or the brand of snuff. Although this finding might suggest that presenting both chewing tobacco and snuff brands together should be considered, it does not indicate that the alphabetical listing of brand names was difficult for respondents to use.

Based on these findings, it was recommended that the tobacco module be used in more or less the same form as it was tested. The findings from the cognitive laboratory testing regarding specific issues suggested the following:

  1. The use of alphabetical lists of brand names should be retained. There was no evidence that any other means of presenting brand name information, such as pictures, was necessary.

  2. The use of "... smoke all or part of a cigarette" wording should be retained. Some respondents did not think the "...smoke any cigarettes" wording referred to smoking part of a cigarette.

  3. The other wording changes introduced in Round 2 worked well enough, although the second definition of snuff could be shortened. They also suggested that "a brand not on this list" was an improvement over the "some other brand" wording used in Round 1.

  4. The computer screen was easy to read, and youths had no difficulty with listening to audio files or using the keyboard. They had little difficulty answering questions about how many cigarettes they smoked over a variety of time periods.

12.6 Tobacco Module: August 1998 Field Test Results

Data on tobacco use from 177 respondents who completed the August field test are available (see Exhibit 12.6.1). There were 91 youths (12 to 17 years old) and 86 adults (18+ years old). Among the youths, there were 53 males and 38 females, while among the adults, 35 were men and 51 were women. The following highlights the results for tobacco use.

  1. Among the 62 youths who had never smoked, 53 (85%) said that they would not smoke if one of their best friends offered them a cigarette; 50 (81%) said that they would definitely not smoke a cigarette in the next 12 months. (Adults were not asked these questions.)

  2. Among the 29 youths who had ever smoked a cigarette, the youngest age at which they first smoked was 5 years and the oldest was 16. Fifteen (52%) were 12 or under when they first smoked a cigarette. Six entered an age of first use of 10 or below, and this triggered a consistency check that they were all able to complete. Age of initiation for the 58 adult lifetime smokers ranged from 7 to 22 years, and 12 (21%) were 12 or younger when they first smoked a cigarette.

  3. Of the 12 youths who had smoked within the past 30 days, 10 named a brand on the list, one did not know the brand, and one person typed in Black and Mild (a brand of cigar). Of the 31 adult 30-day smokers, 30 smoked a named brand and one typed in a brand name (Pyramid).

  4. Among the 12 youths who had smoked in the past 30 days, three gave "don't know" answers to the questions on the number of days smoked, one gave a "don't know" answer to the number smoked on the days that they smoked, and one did not know the brand. These may have been youths who smoked very little and obtained the cigarettes that they smoked from others.

  5. One youth reported smoking a roll-your-own brand as did six adults.

  6. A total of 3 youths and 12 adults reported smoking cigarettes every day for 30 days at some time in their life.

A series of respondent and interviewer debriefing questions was also included in the August 1998 field test. Of note is the following:

  1. Among past 30-day smokers, 10 youths and all 31 of the adults reported that they had no difficulty answering the questions on the brand of cigarettes that they smoked most often in the past 30 days. One youth reported some difficulty, and the other youth did not get to this section.

  2. Only 2 of the 91 youths were rated by the interviewer as having some difficulty answering the tobacco questions on the computer. One did not understand the distinction between chewing tobacco and snuff, and one could not remember the information asked for. Of the 86 adults, 8 reported having at least some difficulty answering the tobacco questions on the computer: 2 had difficulty understanding some of the questions, 1 had difficulty remembering the required information, 1 had trouble responding appropriately and 4 had some other difficulty.

  3. Two youths and one adult made comments on tobacco legislation.

Overall, the field test respondents appeared to have little procedural difficulties answering the questions in the tobacco module. However, a total of 20 "don't know" or "refused" answers were entered. These were most frequently entered for questions asking about the number of cigarettes (five "don't know" answers) and ages when first used tobacco products (four total). Thus, the difficulties that respondents had appeared to be more associated with the inability to recall answers rather than an inability to complete the procedures required to answer the questions. Respondents provided responses on frequency of tobacco use that were comparable to previous NHSDA data, although the small sample size makes any statistical comparisons unstable. The tobacco module appeared to perform well in the field, bolstering the conclusions from the laboratory testing that the questions were adequate for the purpose for which they were designed. Certainly, the brand use questions, the major change from the prior tobacco modules, performed well.

Exhibit 12.6.1 Reported Use of Tobacco Products from August 1998 Field Test

Tobacco Product Usage

Youth

Adults

Recency of Smoking Cigarettes (Frequencies)

   

Within past 30 days

12

29

More than 30 days, but less than 12 months ago

5

5

More than 12 months, less than 3 years ago

9

8

More than 3 years ago

3

13

Never

62

29

     

Cigarette Smoking Frequency (Days Smoked in Past 30 Days)

   

1 - 10 days

6

7

11 - 20 days

2

1

21 - 30 days

1

21

     

Number of Cigarettes Smoked (Past 30 Days)

   

<1 cigarette

6

2

1 - 5 cigarettes

5

8

6 - 15 cigarettes

0

5

16 - 25 cigarettes

0

13

26 - or more cigarettes

0

2

     

Recency of Chewing Tobacco Use

   

Within past 30 days

0

3

More than 30 days, but less than 12 months ago

1

1

More than 12 months, less than 3 years ago

1

2

More than 3 years ago

3

5

Never

86

73

     

Recency of Snuff Use

   

Within past 30 days

0

3

More than 30 days, but less than 12 months ago

0

0

More than 12 months, less than 3 years ago

3

0

More than 3 years ago

2

3

Never

86

80

     

Recency of Smoking Cigars

   

Within past 30 days

6

5

More than 30 days, but less than 12 months ago

5

9

More than 12 months, less than 3 years ago

2

9

More than 3 years ago

2

7

Never

75

56

     

Frequency of Smoking Cigars (Days Smoked in Past 30 Days)

   

    1 - 10 days

5

5

    11 - 20 days

1

0

Source: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1998 Field Test.

12.7 Additional Field and Laboratory Testing

Other components of the 1999 CAI application were tested in the summer of 1998.

12.7.1 Field Testing of the Full Instrument

The August 1998 field test was initially designed as the final test of the software and procedures for using the Apple Newton 2100 for screening households and case management. However, when it was decided to implement the CAI in 1999 to accommodate the addition of the expanded tobacco module, the testing plan was revised to allow testing of a CAI application that included this new module and a draft CAI application for other sections of the interview. The original plan for field testing called for employing 4 field supervisors (FSs) to direct a total of 20 field interviewers (FIs) to collect data in areas that cut across all four U.S. time zones and regions. Subsequent to the decision to include testing of the full CAI application including the new tobacco module, a total of 8 FSs were selected and the field force was expanded to include 40 FIs. The sample design was modified to yield an expected total of 150 CAPI/ACASI interviews. Of those interviews, half were to be with 12 to 17 year olds and half with respondents 18 years old or older.

In the following paragraphs, we describe the findings for the CAI interview. Newton results are discussed in Chapter 8. Note that an earlier version of the Apple Newton was used for the 1997 field experiment.

Response rates. Overall, 836 households were screened; 683 (81.7%) were eligible; within those households, 1,313 individuals (2.06 persons per household) were listed using the Newton application. From these eligible individuals, 256 sampled persons were selected, and 183 (71.5%) of them completed interviews. A higher response rate could have been achieved; however, interviewing was ended early because yields were higher than expected. Exhibit 12.7.1 displays the age distribution of the people who were eligible and sampled, as well as those who completed interviews.

Hardware and software performance. The computer support system at RTI logged problems that were handled during the course of the August 1998 field test.

Problems with the laptop application, which used a Gateway computer, included e-mail communication (14 times), data transmission (13 times), and closing the ACASI program (14 times):

  1. E-mail communication problems were mainly due to phone number problems or problems entering the correct software commands.

  2. Most of the transmission problems were the result of FI errors (usually problems with passwords).

  3. Problems closing the ACASI program were generally solved with a couple of keystrokes or by restarting the interview and jumping to the end.

Interview data analysis. Due to the small sample size of the field test, little could learned about the population distribution of behaviors that are as rare as substance use. Thus, we examined operational aspects of the survey. There were no unusual patterns of "don't know" and "refusals" (such responses were quite rare), and the only questions where "other-specify" was an answer were the race/ethnicity questions, for which a total of 10 answers were specified by respondents.

The completion time data for the ACASI interview were analyzed for the sample as a whole and separately for youths (aged 12 to 17) and adults (18 or older). The results of these analyses are displayed in Exhibit 12.7.2. Overall, the entire interview took approximately 47 minutes, averaged across all field test respondents. The introductory sections averaged about 8 minutes, the core ACASI substance use section took about 14 minutes, the non-core ACASI section took about 15.5 minutes, and the final interviewer administered section took about 9.5 minutes. Youths averaged about 5 minutes more than adults. The difference was in the introductory section (a little over 1 minute more for youths) and the non-core ACASI section (more than 5 minutes more, which was mainly due to the Youth Experience module). The core and the final interviewer-administered sections took slightly less time for youths. The maximum total time was nearly 85 minutes, and the minimum total time was 16 minutes. Three fourths of the interviews were conducted in less than 55 minutes.

Raw, unweighted frequencies of drug use were examined to make sure that there were no surprises in this new application. In general, these showed no differences from prior implementations of the NHSDA.

Interviewer debriefing analysis. During the first 2 weeks of August 1998, conference calls were held with the FIs, and a final call was held during the first week of September. Five calls were held each week with groups of interviewers attending each call. The purpose of these calls was to discuss experiences and problems with data collection.

At the outset of the weekly calls, FIs spontaneously voiced their concerns about the computerized data collection procedures. Concerns about the use of the laptop computer for the CAI interview included the following:

  1. worry that the laptop computer would overheat and be uncomfortable to use if operated in the respondent's lap;

  2. reluctance of respondents to use the headphones (respondents' concerns about cleanliness or refusal to use them);

  3. disliking passing the computer back and forth more than once during transitions between CAPI and ACASI portions of the interview;

  4. complaints that the tutorial is too slow, is insulting to respondents, or other negative reactions to the tutorial;

  5. questions as to the necessity of having separate pill cards and other show cards instead of having computerized presentation of that information; and

  6. various software problems with the laptop (mainly entering final codes).

There were also many cases of problems in transmission between the laptop computer and the mainframe computer at RTI.

Interviewers suggested shortening the tutorial or making it optional, getting rid of calendar and pill cards, simplifying questionnaire language, putting all CAPI questions together, and moving certain additional questions to the ACASI section. They also suggested changes to the way the computers are programmed to transmit data and putting games on the Newtons to amuse FIs during ACASI sections of the interview. The testing was helpful in identifying changes in procedures, particularly transmission of data, and training that addressed FI concerns.

12.7.2 Laboratory Testing of Full Instrument

The full instrument was also tested in the laboratory during July and August 1998. This was necessary because the 1997 field experiment had used eight different versions of the core drug use questions, and testing of a version close to that planned for the 1999 survey was needed. The respondents for the laboratory full interview testing were a convenience sample, divided between clients at drug treatment centers and RTI employees and their acquaintances. A total of 53 people participated in the laboratory testing of the full ACASI instrument. Although some of the interviews were conducted in RTI's Laboratory for Survey Methods and Measurement, the majority were conducted by having RTI staff meet participants at convenient places (either at treatment centers for treatment center clients or in the respondent's home).

The laboratory testing of the full instrument was conducted primarily for the purpose of investigating procedural issues (difficulties with question wording, time to completion) rather than for estimating the prevalence of substance use. Because aconvenience sample was used, we do not report levels of substance use but rather report only the data the interviews were designed to study.

Time to completion. Time to completion was measured in two ways: (a) by calculating completion time from the time stamps that the computer was programmed to record, and (b) by having the interviewer note the beginning and end times of the interview. The first way is more precise and the only way to segment the total time into completion times for different sections of the interview. However, the laboratory version of the full interviews contained modifications in the instrument that were not present in the field test instrument. There was limited time to complete testing of the computer program before beginning the laboratory interviews. Programmers were not certain that the time-recording function was successfully programmed. Therefore, the second method of timing was also used as a backup for possible failures of the programmed timing, even though it yielded only relatively rough estimates of the length of the entire interview.

Laboratory interview sampling was done in a way that would tend to inflate the average interview completion time. Half of the respondents were clients at substance use treatment centers. These respondents would be expected to take longer because they would be more likely to complete sections on at least some substances were bypassed by nontreatment respondents, as well as sections on substance use treatment. These respondents were recruited in order to maximize the likelihood that all sections of the instrument would be completed by at least some respondents. The small size of the field test made it improbable that any drug users would be interviewed. This deliberate heterogeneity made it difficult to generalize the time to completion of the full interview to other populations.

Upon retrieval of the laptop computers used in the interviewing, it was found that there were timing data for only 38 of the 53 participants. Exhibit 12.7.3 displays the time to completion results for these respondents. It is not clear whether the failure to record the times for all interviews was due to equipment malfunctioning, to problems with programming, or to procedural irregularities by interviewers.

Completion times for the laboratory interviews were longer than for the field test19 (about 47 minutes for field test interviews vs. 64 minutes for laboratory interviews). Some of the difference can be attributed to the presence of treatment center respondents (whose interviews averaged over 70 minutes). However, the completion times for the nontreatment laboratory respondents were more than 15 minutes longer than the comparable field interview sample: respondents 18 years of age or older (all nontreatment laboratory interviews with timing information were from that age group). Most of the difference was in the initial CAPI questioning (approximately 7 minutes) and the non-core ACASI section (approximately 6 minutes, despite the absence of parenting questions answered by the nontreatment laboratory interview respondents). It may be that the interviewers for the laboratory interviews, who did not receive formal training on the use of the laptop computer, may have been less adept at using it, making the initial section longer. Furthermore, respondents in laboratory interviews were encouraged to discuss any problems they may have experienced, which would also serve to extend the duration of the interviews.

Specific issues from planned debriefing interviews. During the debriefing of the laboratory respondents, we asked about comprehension of specific questions or instructions in the interview and about selected procedural issues. None of the answers suggested that there were any problems with understanding the specific terms that were investigated ("nonmedical drug use," "treatment"). Results regarding procedural issues were as follows:

  1. Length of the interview: All nontreatment respondents, and a majority (15 of 19) of the treatment clients, did not consider the interview to be overly long. The four clients who did consider it to be too long thought 30 to 60 minutes would be a reasonable length.

  2. Adequacy of the tutorial: The tutorial was not perceived as overlong, and it seemed adequate at instructing nontreatment respondents how to use the computer. Some problems (pushing the wrong keys or inability to go back and change answers) did occur infrequently. Treatment clients found the computer tutorial an adequate introduction to using the computer, although a couple thought it could be shorter. Some respondents did not remember how to do some of the functions addressed in the tutorial, so it would be helpful to have labels for the function keys to serve as memory aids.

  3. Usefulness of the calendar: For nontreatment respondents and treatment clients, reactions to the calendar were mixed. About 33% used the calendar and found it useful, the remainder did not consider it useful and did not use it. Other criticisms of the calendar mainly involved seeing it as too small and hard to read.

Other difficulties observed or mentioned by respondents. There were a few suggestions that the vocabulary for the survey may still be at a relatively high level in a few places. However the ACASI format seemed to make the questions easy enough to answer so that no respondents asked for clarification during the interviews. Most of the other difficulties mentioned by respondents related to their unique circumstances rather than the shortcomings of the instrument (e.g., a respondent who lived in a shelter, who answered that he lived with 65 other people, had this answer rejected as being out of range by the computer).

Exhibit 12.7.1 Number of Persons Listed, Selected, and Responding, by Age

Status/Age in Years

Sample Size

Percent

Eligible Persons Listed in Eligible Household

1,313

100.0

    12-17

147

11.2

    18-25

165

12.6

    26-34

218

16.6

    35-49

366

27.9

    50+

416

31.7

Persons Selected (percent = % of total eligibles by age group)

256

19.5

    12-17

129

87.8

    18-25

69

41.8

    26-34

25

11.5

    35-49

17

4.6

    50+

16

3.8

Completed Interviews (percent = % of sampled persons by age group)

183

71.5

    12-17

93

72.1

    18-25

47

68.1

    26-34

14

56.0

    35-49

15

88.2

    50+

13

81.2

Source: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1998 Field Test

Exhibit 12.7.2 Field Test: Interview Completion Time (Minutes)

Module/Answer Sheet

n

Min

Q1

Median

Q3

Max

Mean

Total Sample

Total Interview

170

16.02

36.90

45.83

54.68

84.68

46.93

Pre-Answer Sheet

170

1.65

5.18

7.05

9.88

26.85

7.92

Total Core

170

2.21

9.10

12.89

17.37

33.86

13.85

    Tobacco

170

0.37

1.77

2.81

4.03

14.53

3.08

    Alcohol

170

0.03

0.37

1.08

2.52

8.75

1.77

    Marijuana

170

0.03

0.10

0.18

0.35

3.33

0.35

    Cocaine

170

0.03

0.07

0.10

0.17

1.57

0.16

    "Crack"

11

0.05

0.07

0.08

0.15

1.35

0.21

    Heroin

170

0.05

0.17

0.27

0.55

2.32

0.40

    Hallucinogen

170

0.18

0.60

0.96

1.47

3.72

1.14

    Inhalants

170

0.28

1.25

1.98

2.83

5.83

2.14

    Analgesics

170

0.23

0.80

1.36

2.13

4.78

1.56

    Tranquilizers

170

0.17

0.55

0.91

1.52

4.12

1.14

    Stimulants

170

0.25

0.50

0.87

1.53

3.68

1.11

    Sedatives1

170

0.22

0.55

0.81

1.26

3.34

0.98

Non-Core ACASI

170

4.39

10.75

14.28

19.30

34.21

15.48

    Special Drugs

170

0.08

0.21

0.31

0.49

1.28

0.37

    Risk Assessment

170

1.02

2.32

2.79

3.98

12.03

3.37

    Drug Experience

93

0.25

0.75

1.22

1.95

6.20

1.56

    Special Topics

170

0.13

0.35

0.52

0.85

4.90

0.70

    Treatment

169

0.08

0.15

0.22

0.37

7.73

0.37

    Youth Experience

84

3.58

9.97

12.02

15.75

24.33

12.73

    Social Environment

86

2.45

4.00

5.32

7.40

18.02

6.29

    Parenting

6

1.63

2.18

2.69

3.77

3.88

2.81

Interviewer-Administered

170

2.75

6.98

9.15

11.80

27.22

9.68

12 to 17 Sample

Total Interview

84

21.40

39.87

47.23

56.91

81.17

49.50

Pre-Answer Sheet

84

1.65

5.46

8.08

10.77

26.85

8.47

Total Core

84

2.21

9.55

12.89

16.66

31.74

13.65

    Tobacco

84

0.47

1.88

2.90

4.03

6.58

2.99

    Alcohol

84

0.03

0.20

0.58

1.39

6.07

1.17

    Marijuana

84

0.03

0.11

0.18

0.28

1.75

0.27

    Cocaine

84

0.03

0.07

0.10

0.15

0.55

0.13

    "Crack"

0

.

.

.

.

.

.

    Heroin

84

0.05

0.17

0.30

0.69

2.32

0.47

    Hallucinogen

84

0.18

0.68

1.04

1.65

3.72

1.27

    Inhalants

84

0.28

1.33

2.08

3.18

5.83

2.23

    Analgesics

84

0.30

0.82

1.47

2.42

4.78

1.75

    Tranquilizers

84

0.20

0.58

1.01

1.66

4.12

1.22

    Stimulants

84

0.25

0.46

0.92

1.58

3.42

1.18

    Sedatives

84

0.22

0.58

0.82

1.26

2.90

0.97

Non-Core ACASI

84

5.78

14.02

17.96

20.60

34.21

18.12

    Special Drugs

84

0.08

0.22

0.31

0.49

1.11

0.37

    Risk Assessment

84

1.03

2.33

2.95

4.26

10.10

3.41

    Drug Experience

25

0.25

0.75

1.12

1.62

6.20

1.49

    Special Topics

84

0.13

0.34

0.45

0.76

4.90

0.64

    Treatment

84

0.08

0.17

0.25

0.40

0.93

0.30

    Youth Experience

84

3.58

9.97

12.02

15.75

24.33

12.73

    Social Environment

0

.

.

.

.

.

.

    Parenting

0

.

.

.

.

.

.

Interviewer-Administered

84

3.42

6.06

7.95

11.43

27.22

9.26

18+ Sample

Total Interview

86

16.02

33.82

44.23

52.05

84.68

44.41

Pre-Answer Sheet

86

1.92

5.03

6.81

9.10

20.92

7.38

Total Core

86

4.24

8.66

12.95

17.85

33.86

14.04

    Tobacco

86

0.37

1.73

2.72

4.25

14.53

3.17

    Alcohol

86

0.07

0.90

1.93

3.23

8.75

2.36

    Marijuana

86

0.05

0.10

0.21

0.62

3.33

0.43

    Cocaine

86

0.03

0.07

0.11

0.18

1.57

0.20

    "Crack"

11

0.05

0.07

0.08

0.15

1.35

0.21

    Heroin

86

0.07

0.15

0.22

0.40

2.07

0.33

    Hallucinogen

86

0.22

0.53

0.78

1.32

3.00

1.02

    Inhalants

86

0.47

1.13

1.92

2.62

5.20

2.06

    Analgesics

86

0.23

0.78

1.21

1.80

3.87

1.37

    Tranquilizers

86

0.17

0.53

0.81

1.42

4.10

1.05

    Stimulants

86

0.25

0.52

0.82

1.52

3.68

1.05

    Sedatives

86

0.24

0.54

0.79

1.26

3.34

0.98

Non-Core ACASI

86

4.39

8.58

11.16

15.50

28.37

12.90

    Special Drugs

86

0.09

0.21

0.30

0.49

1.28

0.38

    Risk Assessment

86

1.02

2.17

2.78

3.77

12.03

3.34

    Drug Experience

68

0.35

0.74

1.28

1.98

5.83

1.59

    Special Topics

86

0.13

0.40

0.63

1.00

3.60

0.77

    Treatment

85

0.08

0.13

0.18

0.30

7.73

0.43

    Youth Experience

0

.

.

.

.

.

.

    Social Environment

86

2.45

4.00

5.32

7.40

18.02

6.29

    Parenting

6

1.63

2.18

2.69

3.77

3.88

2.81

Interviewer-Administered

86

2.75

7.73

9.87

11.93

27.10

10.10

1There was no time stamp between the sedative and special drugs sections, so the recorded time included both sections. Times for each section were interpolated by splitting the total time into the proportions of the 1997 Field Test #2, Treatment Version3, for the two sections.

Source: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1998 Field Test.

Exhibit 12.7.3 Full Laboratory Interviews: Interview Completion Time (Minutes)

Module/Answer Sheet

n

MIN

Q1

MEDIAN

Q3

MAX

MEAN

Total Sample

Total Interview

39

32.1

46.3

60.3

78.1

121.9

64.1

Pre-Answer Sheet

39

3.3

9.1

11.3

15.0

40.4

13.38

Total Core

39

7.9

12.0

16.5

20.6

43.4

17.98

    Tobacco

39

0.8

2.0

3.6

5.6

9.1

4.10

    Alcohol

39

0.3

1.6

2.1

3.1

8.3

2.45

    Marijuana

39

0.1

0.3

0.5

1.0

3.2

0.72

    Cocaine

39

0.0

0.1

0.3

1.1

4.7

0.81

    "Crack"

23

0.01

0.2

0.8

1.1

2.1

0.77

    Heroin

39

0.1

0.2

0.4

0.7

2.9

0.59

    Hallucinogen

39

0.4

0.7

0.9

1.4

3.9

1.17

    Inhalants

39

0.9

1.4

1.8

2.6

3.6

1.95

    Analgesics

39

0.6

1.0

1.4

3.0

5.6

1.98

    Tranquilizers

39

0.5

0.9

1.3

2.2

5.8

1.63

    Stimulants

39

0.5

0.7

1.00

1.3

3.2

1.16

    Sedatives

39

0.4

0.6

0.8

1.1

2.7

0.96

Non-Core ACASI

39

7.1

11.7

18.1

28.0

60.2

21.53

    Special Drugs

39

0.2

0.2

0.3

1.0

4.6

0.81

    Risk Assessment

39

1.6

2.2

2.9

4.0

14.8

3.57

    Drug Experience

39

1.0

1.8

3.7

5.8

20.2

4.99

    Special Topics

39

0.0

0.5

0.8

1.7

4.8

1.20

    Treatment

39

0.0

0.4

1.8

5.0

17.0

3.34

    Social Environment

38

0.0

5.0

7.7

9.1

12.8

7.18

Interviewer-Administered

39

5.5

8.6

11.7

14.0

19.5

11.24

Treatment Center Sample

Total Interview

12

34.3

48.9

65.4

89.4

121.9

70.44

Pre-Answer Sheet

12

5.2

7.5

10.2

12.1

29.3

11.20

Total Core

12

8.3

14.1

19.6

27.7

43.4

21.24

    Tobacco

12

2.0

3.3

4.9

7.4

9.0

5.22

    Alcohol

12

0.3

1.3

1.6

3.4

8.3

2.59

    Marijuana

12

0.1

0.3

0.6

0.9

3.2

0.78

    Cocaine

12

0.2

0.3

0.7

1.9

2.8

1.05

    "Crack"

10

0.2

0.6

0.9

1.2

2.1

0.99

    Heroin

12

0.1

0.3

0.6

1.6

2.9

0.96

    Hallucinogen

12

0.4

0.8

1.1

1.7

3.4

1.32

    Inhalants

12

0.9

1.5

2.0

2.7

3.6

2.06

    Analgesics

12

0.8

1.0

1.8

3.3

5.6

2.25

    Tranquilizers

12

0.5

0.7

1.4

2.6

4.4

1.73

    Stimulants

12

0.6

0.8

1.0

2.7

3.2

1.35

    Sedatives

12

0.5

0.7

0.9

1.4

2.7

1.12

Non-Core ACASI

12

11.0

14.2

25.9

34.5

60.2

27.61

    Special Drugs

12

0.2

0.3

0.7

2.3

4.6

1.34

    Risk Assessment

12

1.6

2.2

3.4

4.0

8.8

3.81

    Drug Experience

12

1.4

3.7

4.8

7.3

20.2

6.71

    Special Topics

12

0.3

0.7

1.6

2.1

4.2

1.66

    Treatment

12

1.7

2.3

4.1

9.8

17.0

6.29

    Social Environment

12

3.4

4.8

8.4

9.2

12.8

7.49

Interviewer-Administered

12

6.7

7.6

9.2

12.9

17.9

10.40

Nontreatment Sample

Total Interview

27

32.1

45.6

60.3

74.1

110.8

61.33

Pre Answer Sheet

27

3.3

9.3

12.4

15.4

40.4

14.35

Total Core

27

7.9

11.3

16.5

20.1

34.2

16.54

    Tobacco

27

0.8

1.8

3.4

4.4

9.1

3.61

    Alcohol

27

0.7

1.8

2.2

3.0

5.4

2.39

    Marijuana

27

0.1

0.3

0.4

1.0

3.0

0.70

    Cocaine

27

0.0

0.1

0.2

0.9

4.7

0.71

    "Crack"

13

0.0

0.1

0.5

1.1

1.4

0.60

    Heroin

27

0.1

0.2

0.4

0.5

1.4

0.43

    Hallucinogen

27

0.4

0.6

0.9

1.4

3.9

1.10

    Inhalants

27

1.0

1.4

1.7

2.4

2.9

1.90

    Analgesics

27

0.6

1.1

1.4

2.6

4.2

1.86

    Tranquilizers

27

0.6

0.9

1.2

1.8

5.8

1.59

    Stimulants

27

0.5

0.7

1.0

1.3

2.6

1.07

    Sedatives

27

0.4

0.6

0.8

1.1

2.2

0.89

Non-Core ACASI

27

7.1

11.6

15.2

25.2

55.9

18.82

    Special Drugs

27

0.1

0.2

0.3

0.5

2.4

0.57

    Risk Assessment

27

1.7

2.2

2.9

3.8

14.8

3.46

    Drug Experience

27

1.0

1.6

3.1

4.4

14.6

4.22

    Special Topics

27

0.0

0.5

0.6

1.2

4.8

1.00

    Treatment

27

0.0

0.4

0.6

3.5

10.0

2.04

    Social Environment

26

0.0

5.0

7.5

8.7

12.8

7.03

Interviewer-Administered

27

5.5

9.2

11.9

14.0

19.5

11.62

Source: National Household Survey on Drug Abuse: Development of Computer-Assisted Interviewing Procedures; 1998 Field Test.

12.8 The 1999 NHSDA CAI

The NHSDA CAI fielded in 1999 incorporated those features that were tested throughout the prior two years of development, as described in this report, that were determined to decrease respondent burden and increase data quality. Decisions resulting from this research regarding survey methodology and subject matter content for the NHSDA beginning in 1999 are described in the next two sections.

12.8.1 Methodology

Major features of NHSDA methodology implemented in 1999 are summarized below:

  1. Electronic screening and case management were implemented using the Apple Newton hand-held computer (see Chapters 5.3 and 8 for a full description of the electronic screener development and testing). A Spanish translation of the screener instrument was added.

    Screening is conducted as follows: at the beginning of the data collection period, each field interviewer's (FI's) list of assigned cases (case ID numbers and addresses) is transmitted via phone lines to his or her Newton computer from RTI's central office. On each visit or attempted visit to an address, whether for screening or interviewing, the FI enters the result code for the visit and any relevant notes into the Newton. If a household respondent is contacted, the FI asks the appropriate screening questions, enters responses, and the Newton computer selects zero, one, or two sample person(s) for interviewing.

  1. The CAPI/ACASI methodology was adopted for the NHSDA interview. Both software and hardware were upgraded for the August, 1998 pretest and subsequent field work beginning in January, 1999. The interview was programmed in Blaise 4.0 for Windows and loaded on Gateway SoloTM 5100 Multimedia Notebook laptop computers. This computer has a 14" monitor, built-in sound card, 56K modem, 64 MB of RAM, 4 GB hard disk space, and a 233 MHZ Pentium II processor.

    A Spanish translation of the questionnaire was added.

    The CAI interview is conducted as follows: for each sample person selected, a unique questionnaire ID number is generated by the Newton, which must be entered into the laptop computer by the FI in order to start the interview. The FI selects the language to be used (English or Spanish) and conducts the initial CAPI portion of the interview, then turns the laptop over to the respondent, pointing out the keys the respondentwill use and giving instructions in use of the headphones.

    The function keys that respondents are instructed to use for backing up one question at a time, turning off the sound, replaying the sound, and entering "don't know" and "refused" responses are labeled with a template.

    Before the actual ACASI portion of the interview begins, the respondent is presented with the short, interactive ACASI tutorial described earlier that provides basic instructions and practice in entering responses to different types of questions, changing responses, and having questions repeated. The FI makes every effort to ensure that no person other than the respondent can see or hear the questions. After the respondent completes the ACASI portion, the FI administers the remaining CAPI portion of the interview. Finally, the FI enters responses to a few questions about his/her impressions of the interview. These "FI debriefing items" were revised in 1999 to include an assessment of the effect of the computer on the respondent's decision to participate and on the privacy of the interview.

    Each day, FIs transmit all screening work from the Newton and interviews from the laptop to RTI headquarters over telephone lines. Screening data are updated daily and made available on the project website to RTI and SAMHSA staff for monitoring field progress. (See Chapters 9.1.4 and 9.1.5 for a description of the data transmission, capture and monitoring systems.)

  2. Experimental Factor 1: Structure of contingent questioning in the CAI questionnaire: a single gate question for all core substances was selected over multiple gate questions. (See Chapters 5 and 7.1.)

  3. Experimental Factor 2: Data quality checks were implemented throughout the CAI questionnaire, including range edits and inconsistency resolution in CAPI and ACASI. (See Chapters 5 and 7.2.)

  4. Experimental Factor 3: A single opportunity to report 30 day and 12 month substance use was selected over multiple opportunities to report use. (See Chapters 5 and 7.3.)

  5. A two-stage question was implemented for asking 12-month frequency of use, in which respondents first choose the metric that is easiest for them (days per year, month or week) and then report number of days.

  6. The voice used for the ACASI sections of the interview was the preferred female voice from our cognitive laboratory study (see Caspar and Edwards, 1997).

12.8.2 Questionnaire Content

Significant new questionnaire content was introduced and tested in the final pretest in August 1998, and fielded in January, 1999. This new content is summarized below.

  1. An entirely new tobacco module was fielded, with questions on several smoking-related topics:

     usual (past 30 day) brand of cigarettes, snuff, chewing tobacco, and cigars: wording for cigarettes and cigars was changed from "even a few puffs" to "part or all of a _____," and snuff and chewing tobacco were asked about separately;

     questions on use of pipe tobacco and "roll-your-own" cigarettes; and

     a set of questions for adolescents who do not smoke, asking about the likelihood of initiating smoking in the next year.

  2. Response categories for the recency of use questions were shortened from four to three: "within the past 30 days," "more than 30 days ago but within the past 12 months," and "more than 12 months ago".

  3. A new question on "month of first use" of all drugs was added for better incidence data on persons recently initiating substance use.

  4. The "pill cards" (showing prescription drugs) were updated.

  5. Questions on non-medical use of prescription drugs were revised and tested, and the term "pain killer" for analgesics was changed to "pain reliever."

  6. New questions were added to the drug dependence section to assess withdrawal symptoms resulting from use of specific drugs. These questions were based on criteria for withdrawal as described in the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) published by the American Psychiatric Association (1994).

  7. Two questions traditionally asked in the NHSDA core drug sections were deleted as a result of field and lab testing: lifetime number of days a substance was used and number of days in past 12 monthsthat a respondent got very high or drunk. The former question was determined to be too difficult for most respondents to answer accurately, and the latter was poorly understood, based on lab testing and on data from consistency checks in the field experiment.

  8. Because of increasing concerns about methamphetamine abuse, new questions were added to the core "stimulants" section to estimate the incidence (age at first use, including month and year for recent initiates) and recency of use of methamphetamines.

  9. New content in the socio-demographic sections includes additional questions on employment status, country of birth, and the opportunity to report multiple racial and Hispanic groups.

12.8.3 Summary

We believe the new CAI instrument and procedures implemented in 1999 represent a significant improvement over the PAPI instrument and procedures employed from 1971 through 1998. The new methodology was implemented in 1999 on a sample of approximately 67,000 respondents, with an additional PAPI sample of approximately 14,000. A report on the implementation and assessment of the new design, "Methodological Issues in the 1999 Redesign of the NHSDA" will be published later in 2001. This report discusses particular problems and issues addressed during the redesign, as well as the effect of interview mode (CAI vs. PAPI) on the reporting of drug use data. Additional information on the NHSDA, including the 1999 questionnaire, details on all data collection procedures, and the Summary of Findings from the 1999 NHSDA can be accessed through the World Wide Web, at http://www.oas.samhsa.gov.

19 Recall that no formal hypothesis testing of comparisons of means was made.

Top Of PageTable of Contents

This page was last updated on June 16, 2008.