Appendix A: State Estimation Methodology

This report includes estimates of 23 substance use and mental health measures (see Section A.1) using the combined data from the 2006 and 2007 National Surveys on Drug Use and Health (NSDUHs). Also included in this report are estimates of change between 2005-2006 and 2006-2007 State estimates. As discussed in Chapter 1 (Section 1.1), several changes were introduced to the survey in 2002; thus, estimates for 2001 and prior years are not comparable with estimates from 2002 and later years.

The survey-weighted hierarchical Bayes (SWHB) methodology used in the production of State estimates from the 1999-2006 surveys also was used in the production of the 2006-2007 State estimates. The SWHB methodology is described in Appendix E of the 2001 State report (Wright, 2003b) and by Folsom, Shah, and Vaish (1999). The list of predictors used in the 2006-2007 small area estimation (SAE) modeling is given in Section A.2. No new variable selection was done for the 2006-2007 data (as discussed in Section A.3). The goals of SAE modeling, general model description, and the implementation of SAE modeling remain the same and are described in Appendix E of the 2001 State report (Wright, 2003b). At the end of this appendix, tables showing the 2005, 2006, 2007, pooled 2005-2006, and pooled 2006-2007 survey response rates are included (Tables A.1 to A.12).

Small area estimates obtained using the SWHB methodology are design consistent (i.e., for States with large sample sizes, the small area estimates are close to the robust design-based estimates). The State small area estimates when aggregated by using the appropriate population totals result in national small area estimates that are very close to the national design-based estimates. However, for numerous reasons (including internal consistency), it is desirable to have national small area estimates exactly match the national design-based estimates. Beginning in 2002, exact benchmarking was introduced as described in Section A.4. Tables of estimated numbers of persons associated with each outcome (in thousands) are available on the web in the form of HTML tables (see http://www.oas.samhsa.gov/2k7State/toc.cfm). An explanation of how these counts and their respective prediction intervals (PIs) are calculated can be found in Section A.5. The definition and explanation of the formula used in estimating the marijuana incidence rate is given in Section A.6.

For all outcomes, the age groups for which estimates are provided in this report are 12 to 17, 18 to 25, and 26 or older. Estimates for those aged 12 or older also are provided in this report. Because it was determined that States may find estimates for 18 or older useful, estimates for that age group will be available on the web in the form of HTML tables (see http://www.oas.samhsa.gov/2k7State/toc.cfm).

Also included in this report are estimates of underage (aged 12 to 20) alcohol use and binge alcohol use. Alcohol consumption is expected to differ significantly across the 18 to 25 age group because of the legalization of alcohol at age 21. Therefore, it was decided that it would be useful to produce small area estimates for persons aged 12 to 20. A short description of the methodology used to produce underage drinking estimates is provided in Section A.7.

Section A.8 discusses the criteria used to define dependence on and abuse of illicit drugs and alcohol. Section A.9 discusses how the serious psychological distress (SPD) estimates were produced. Section A.10 discusses the production of major depressive episode (MDE) estimates. The methodology used to produce estimates of change between the 2005-2006 and the 2006-2007 State estimates is described in Section A.11. Section A.12 discusses the method to compare prevalence rates of a particular outcome between two States.

A.1 Variables Modeled

The 2007 NSDUH data were pooled with the 2006 NSDUH data, and age group–specific State prevalence estimates for 23 binary (0, 1) outcome variables were produced and presented in this report in Appendix B. These estimates were produced for the following outcomes:

1. past month use of illicit drugs,
   
2. past year use of marijuana,
   
3. past month use of marijuana,
   
4. perception of great risk of smoking marijuana once a month,
   
5. average annual rate of first use of marijuana,
   
6. past month use of illicit drugs other than marijuana,
   
7. past year use of cocaine,
   
8. past year nonmedical use of pain relievers,
   
9. past month use of alcohol,
   
10. past month binge alcohol use,
    
11. perception of great risk of having five or more drinks of an alcoholic beverage once or twice a week,
    
12. past month use of tobacco products,
    
13. past month use of cigarettes,
    
14. perception of great risk of smoking one or more packs of cigarettes per day,
    
15. past year alcohol dependence or abuse,
    
16. past year alcohol dependence,
    
17. past year illicit drug dependence or abuse,
    
18. past year illicit drug dependence,
    
19. past year dependence on or abuse of illicit drugs or alcohol,
    
20. needing but not receiving treatment for illicit drug use in the past year,
    
21. needing but not receiving treatment for alcohol use in the past year,
    
22. past year serious psychological distress (SPD), and
    
23. past year major depressive episode (MDE).

Estimates of change between the 2005-2006 and 2006-2007 State estimates were produced for all of these outcomes and are included in this report in Appendix C. Also included at the end of this appendix is a table listing all outcomes and the years for which small area estimates were produced going back to the 2002 NSDUH (Table A.13).

A.2 Predictors Used in Mixed Logistic Regression Models

Local area data used as potential predictor variables in the mixed logistic regression models were obtained from several sources, including Claritas Inc., the U.S. Census Bureau, the Federal Bureau of Investigation (FBI) (Uniform Crime Reports), Health Resources and Services Administration (Area Resource File), the Bureau of Labor Statistics, the Bureau of Economic Analysis, the Substance Abuse and Mental Health Services Administration (SAMHSA) (National Survey of Substance Abuse Treatment Services [N-SSATS]), and the National Center for Health Statistics (mortality data). The values of these predictor variables are updated every year (when possible). Major sources and potential data items used in the modeling are provided in the following text and lists.

• Claritas. The demographic data package called Building Block Basic, Age by Race for 2002 with projections to 2007. The projections are used to update these predictor variables each year. The 2006 and 2007 projections were used this year.
  
  
• U.S. Census Bureau. The 2000 census (demographic and socioeconomic variables) and 2004 food stamp participation rates (http://www.census.gov/hhes/www/saipe/inputdata/cntyfs.xls).
  
  
• Federal Bureau of Investigation. Uniform Crime Report (UCR) arrest totals from http://www.icpsr.umich.edu/NACJD/ucr.html. The most current data used are 2005 for most counties, with previous years' data substituted in a few cases.
  
  
• Health Resources and Services Administration. Families below the poverty-level rates from the 2000 census data are obtained from the Area Resource File (ARF) February 2005 release from the Bureau of Health Professions, Office of Research and Planning.
  
  
• Bureau of Labor Statistics (BLS). The 2006 county-level unemployment rates (ftp://ftp.bls.gov/pub/special.requests/la/laucnty06.txt).
  
  
• Bureau of Economic Analysis (BEA). The 2006 county-level per capita income rates (http://www.bea.gov/regional/reis/).
  
  
• National Center for Health Statistics. Mortality data using International Classification of Diseases, 10th revision (ICD-10), 2000-2005. The ICD-10 death rate data are from the National Center for Health Statistics at the Centers for Disease Control and Prevention.
  
  
• SAMHSA, Office of Applied Studies (OAS). National Survey of Substance Abuse Treatment Services (N-SSATS), formerly known as the Uniform Facility Data Set (UFDS). The 2006-2007 data on drug and alcohol treatment rates are obtained from Synectics for Management Decisions, Inc.
  

The following lists provide the specific independent variables that were potential predictors in the models.

Claritas Data (Description)	Claritas Data (Level)
% Population aged 0-19 in block group	Block group
% Population aged 20-24 in block group	Block group
% Population aged 25-34 in block group	Block group
% Population aged 35-44 in block group	Block group
% Population aged 45-54 in block group	Block group
% Population aged 55-64 in block group	Block group
% Population aged 65+ in block group	Block group
% Blacks in block group	Block group
% Hispanics in block group	Block group
% Other race in block group	Block group
% Whites in block group	Block group
% Males in block group	Block group
% Females in block group	Block group
% American Indian, Eskimo, Aleut in tract	Tract
% Asian, Pacific Islander in tract	Tract
% Population aged 0-19 in tract	Tract
% Population aged 20-24 in tract	Tract
% Population aged 25-34 in tract	Tract
% Population aged 35-44 in tract	Tract
% Population aged 45-54 in tract	Tract
% Population aged 55-64 in tract	Tract
% Population aged 65+ in tract	Tract
% Blacks in tract	Tract
% Hispanics in tract	Tract
% Other race in tract	Tract
% Whites in tract	Tract
% Males in tract	Tract
% Females in tract	Tract
% Population aged 0-19 in county	County
% Population aged 20-24 in county	County
% Population aged 25-34 in county	County
% Population aged 35-44 in county	County
% Population aged 45-54 in county	County
% Population aged 55-64 in county	County
% Population aged 65+ in county	County
% Blacks in county	County
% Hispanics in county	County
% Other race in county	County
% Whites in county	County
% Males in county	County
% Females in county	County

2000 Census Data (Description)	2000 Census Data (Level)
% Population who dropped out of high school	Tract
% Housing units built in 1940-1949	Tract
% Persons aged 16-64 with a work disability	Tract
% Hispanics who are Cuban	Tract
% Females 16 years or older in labor force	Tract
% Females never married	Tract
% Females separated/divorced/widowed/other	Tract
% One-person households	Tract
% Female head of household, no spouse, child under 18	Tract
% Males 16 years or older in labor force	Tract
% Males never married	Tract
% Males separated/divorced/widowed/other	Tract
% Housing units built in 1939 or earlier	Tract
Average persons per room	Tract
% Families below poverty level	Tract
% Households with public assistance income	Tract
% Housing units rented	Tract
% Population with 9-12 years of school, no high school diploma	Tract
% Population with 0-8 years of school	Tract
% Population with associate's degree	Tract
% Population with some college and no degree	Tract
% Population with bachelor's, graduate, professional degree	Tract
Median rents for rental units	Tract
Median value of owner-occupied housing units	Tract
Median household income	Tract

Uniform Crime Report Data (Description)	Uniform Crime Report Data (Level)
Drug possession arrest rate	County
Drug sale/manufacture arrest rate	County
Drug violations' arrest rate	County
Marijuana possession arrest rate	County
Marijuana sale/manufacture arrest rate	County
Opium or cocaine possession arrest rate	County
Opium or cocaine sale/manufacture arrest rate	County
Other drug possession arrest rate	County
Other dangerous non-narcotics arrest rate	County
Serious crime arrest rate	County
Violent crime arrest rate	County
Driving under influence arrest rate	County

Other Categorical Data (Description)	Other Categorical Data (Source)	Other Categorical Data (Level)
=1 if Hispanic, =0 otherwise	NSDUH sample	Person
=1 if non-Hispanic black, =0 otherwise	NSDUH sample	Person
=1 if non-Hispanic other, =0 otherwise	NSDUH sample	Person
=1 if male, =0 if female	NSDUH sample	Person
=1 if MSA with ≥1 million, =0 otherwise	2000 Census	County
=1 if MSA with <1 million, =0 otherwise	2000 Census	County
=1 if non-MSA urban, =0 otherwise	2000 Census	Tract
=1 if urban area, =0 if rural area	2000 Census	Tract
=1 if no Cubans in tract, =0 otherwise	2000 Census	Tract
=1 if no arrests for dangerous non-narcotics, =0 otherwise	UCR	County

Miscellaneous Data (Variable Description)	Miscellaneous Data (Source)	Miscellaneous Data (Level)
Alcohol death rate, underlying cause	NCHS-ICD-10	County
Cigarettes death rate, underlying cause	NCHS-ICD-10	County
Drug death rate, underlying cause	NCHS-ICD-10	County
Alcohol treatment rate	N-SSATS (formerly called UFDS)	County
Alcohol and drug treatment rate	N-SSATS (formerly called UFDS)	County
Drug treatment rate	N-SSATS (formerly called UFDS)	County
% Families below poverty level	ARF	County
Unemployment rate	BLS	County
Per capita income (in thousands)	BEA	County
Average suicide rate (per 10,000)	NCHS-ICD-10	County
Food stamp participation rate	Census Bureau	County
Single State agency maintenance of effort	National Association of State Alcohol and Drug Abuse Directors (NASADAD)	State
Block grant awards	SAMHSA	State
Cost of Services Factor Index	SAMHSA	State
Total Taxable Resources Per Capita Index	U.S. Department of Treasury	State

A.3 Selection of Independent Variables for the Models

No new variable selection was done for any outcome variables in 2006-2007. The updated versions of fixed-effect predictors that were used in modeling the 2005-2006 data were used to model the 2006-2007 data. Because the interest was to estimate change between the 2005-2006 and 2006-2007 State estimates, the same set of fixed-effect predictors was used for producing both sets of estimates.

A.4 Benchmarking the Age Group–Specific Small Area Estimates

The self-calibration built into the SWHB solution ensures that the population-weighted average of the State small area estimates will closely match the national design-based estimates. Given the self-calibration ensured by the SWHB solution, for State reports prior to 2002, the standard Bayes prescription was followed; specifically, the posterior mean was used for the point estimate, and the tail percentiles of the posterior distribution were used for the prediction interval (PI) limits.

Singh and Folsom (2001) extended Ghosh's (1992) results on constrained Bayes estimation to include exact benchmarking to design-based national estimates. In the simplest version of this constrained Bayes solution where only the design-based mean is imposed as a benchmarking constraint, each of the State-by-age group small area estimates (for 2006-2007) is adjusted by adding the common factor Δ_a = (D_a - P_a), where D_a is the design-based national prevalence estimate and P_a is the population-weighted mean of the State small area estimates (P_sa) for age group-a. The exactly benchmarked State-s and age group-a small area estimates then are given by θ_sa = P_sa+ Δ_a. Experience with such additive adjustments suggests that the resulting exactly benchmarked State small area estimates will always be between 0 and 100 percent because the SWHB self-calibration ensures that the adjustment factor is small relative to the size of the State-level small area estimates.

Relative to the Bayes posterior mean, these benchmark-constrained State small area estimates are biased by the common additive adjustment factor. Therefore, the posterior mean-squared error for each benchmarked State small area estimate has the square of this adjustment factor added to its posterior variance. To achieve the desirable feature of exact benchmarking, this constrained Bayes adjustment factor was implemented for the State-by-age group small area estimates. The associated credible intervals can be recentered at the benchmarked small area estimates on the logit scale with the symmetric interval end points based on the posterior root mean-squared errors. The adjusted 95 percent PIs (Lower_sa , Upper_sa) are defined below:

Lower_sa = exp(L_sa)/[1 + exp(L_sa)] and Upper_sa = exp(U_sa)/[1 + exp(U_sa)],

where

L_sa = ln[θ_sa/(1 - θ_sa)] - 1.96 * ,     D

U_sa = ln[θ_sa/(1 - θ_sa)] + 1.96 * and      D

MSE_sa = (ln[P_sa/(1 - P_sa)]- ln[θ_sa/(1 - θ_sa)])² + posterior variance of ln[P_sa/(1 - P_sa)].     D

The associated posterior coverage probabilities for these benchmarked intervals are very close to the prescribed 0.95 value because the State small area estimates have posterior distributions that can be approximated exceptionally well by a Gaussian distribution.

A.5 Calculation of Estimated Number of Persons Associated with Each Outcome

Tables 1 to 24, available at http://www.oas.samhsa.gov/2k7State/toc.cfm, show the estimated numbers of persons (in thousands) associated with each of the 23 outcomes of interest. To calculate these estimated numbers of persons, the benchmarked small area estimates and the associated 95 percent PIs are multiplied by the average population (across the 2 years, in this case 2006 and 2007) of the State by age group of interest.

For example, past month use of alcohol among 18 to 25 year olds in Alabama was 52.05 percent (see Table B.9). The corresponding PIs ranged from 48.17 to 55.90 percent. The population count for 18 to 25 year olds averaged across 2006-2007 in Alabama was 500,168 (see Table A.10). Hence, the estimated number of 18 to 25 year olds using alcohol in past month in Alabama was 0.5205 * 500,168, which is 260,337 (see Table 9). The associated PIs ranged from 0.4817 * 500,168 (i.e., 240,931) to 0.5590 * 500,168 (i.e., 279,594). Note that when estimates of the number of persons are calculated for Tables 1 to 24, the unrounded prevalence estimates and population counts are used. Hence, the number obtained by multiplying the published prevalence rate with the published population estimate may not exactly match the counts that are published in these tables due to rounding differences.

A.6 Calculation of Average Annual Incidence of Marijuana Use

Incidence rates typically are calculated as the number of new initiates of a substance during a period of time (such as in the past year) divided by an estimate of the number of person years of exposure (in thousands). The incidence definition used in this report employs a simpler form of the at-risk population based on the model-based methodology. This model-based average annual incidence rate is defined as follows:

Average annual rate = 100*{[X₁ ÷ (0.5 * X₁ + X₂)] ÷ 2},     D

where X₁ is the number of marijuana initiates in the past 24 months and X₂ is the number of persons who never used marijuana.

In this report, the incidence rate is expressed as a percentage or rate per 100 person years of exposure. Note that this estimate uses a 2-year time period to accumulate incidence cases from each annual survey. By assuming further that the distribution of first use for the incidence cases is uniform across the 2-year interval, the total number of person years of exposure is 1 year on average for the incidence cases plus 2 years for all the "never users" at the end of the time period. This approximation to the person years of exposure permits one to recast the incidence rate as a function of two population prevalence rates, namely, the fraction of persons who first used marijuana in the past 2 years and the fraction who had never used marijuana. Both of these prevalence estimates were estimated using the SWHB estimation approach.

The count of persons who first used marijuana in the past 2 years is based on a "moving" 2-year period that ranges over 3 calendar years. Subjects were asked when they first used marijuana. If a person indicated first use of marijuana between the day of the interview and 2 years prior, the person was included in the count. Thus, it is possible for a person interviewed in the first part of 2007 to indicate first use as early as the first part of 2005 or as late as the first part of 2007. Similarly, a subject interviewed in the last part of 2007 could indicate first use as early as the last part of 2005 or as late as the last part of 2007. Therefore, in the 2007 survey, the reported period of first use ranged from early 2005 to late 2007 and was "centered" in 2006. About half of the 12 to 17 year olds who reported first use in the past 24 months reported first use in 2006, while a quarter each reported first use in 2005 and 2007. Persons who responded in 2007 that they had never used marijuana were included in the count of "never used." Similarly, reports of first use in the past 24 months from the 2006 survey ranged from early 2004 to late 2006 and were centered in 2005. Half of the 12 to 17 year olds who reported first use in the past 24 months reported first use in 2005, while a quarter each reported first use in 2004 and 2006. Note that only incidence rates for marijuana use are provided in this report.

A.7 Underage Drinking

To obtain small area estimates for persons aged 12 to 20 for past month alcohol and binge alcohol use, a separate set of models was fit for these two outcomes for the 12 to 17 age group and the 18 to 20 age group. For the 2006-2007 models, no new variable selection was done. Updated versions of the predictors were used to produce the small area estimates.

Model-based estimates for persons aged 12 to 20 were produced by taking the population-weighted average of the individual age group (12 to 17 and 18 to 20) estimates. Estimates for underage drinking for past month alcohol and binge alcohol use were benchmarked to match national design-based estimates for that age group using the process described in Section A.4. Estimates of change between the 2005-2006 and 2006-2007 estimates for underage drinking in the States also are presented in this report.

A.8 Illicit Drug and Alcohol Dependence and Abuse

The NSDUH computer-assisted interviewing (CAI) instrumentation includes questions that are designed to measure dependence on and abuse of illicit drugs and alcohol. For these substances,⁷ dependence and abuse questions were based on the criteria in the DSM-IV (APA, 1994).

Specifically, for marijuana, hallucinogens, inhalants, and tranquilizers, a respondent was defined as having dependence if he or she met three or more of the following six dependence criteria:

1. Spent a great deal of time over a period of a month getting, using, or getting over the effects of the substance.
   
   
2. Used the substance more often than intended or was unable to keep set limits on the substance use.
   
   
3. Needed to use the substance more than before to get desired effects or noticed that the same amount of substance use had less effect than before.
   
   
4. Inability to cut down or stop using the substance every time tried or wanted to.
   
   
5. Continued to use the substance even though it was causing problems with emotions, nerves, mental health, or physical problems.
   
   
6. The substance use reduced or eliminated involvement or participation in important activities.
   

For alcohol, cocaine, heroin, pain relievers, sedatives, and stimulants, a seventh withdrawal criterion was added. A respondent was defined as having dependence if he or she met three or more of seven dependence criteria. The seventh withdrawal criterion is defined by a respondent reporting having experienced a certain number of withdrawal symptoms that vary by substance (e.g., having trouble sleeping, cramps, hands tremble).

For each illicit drug and alcohol, a respondent was defined as having abused that substance if he or she met one or more of the following four abuse criteria and was determined not to be dependent on the respective substance in the past year:

1. Serious problems at home, work, or school caused by the substance, such as neglecting your children, missing work or school, doing a poor job at work or school, or losing a job or dropping out of school.
   
   
2. Used the substance regularly and then did something that might have put you in physical danger.
   
   
3. Use of the substance caused you to do things that repeatedly got you in trouble with the law.
   
   
4. Had problems with family or friends that were probably caused by using the substance and continued to use the substance even though you thought the substance use caused these problems.

For additional details on how respondents were classified as being dependent on or having abused illicit drugs and alcohol, see Section B.4.3 in Appendix B of the 2007 NSDUH's national results report (OAS, 2008a, pp. 131-133).

A.9 Serious Psychological Distress

In 2005, 2006, and 2007, serious psychological distress (SPD) was measured using the "short-form" module consisting only of the K6 screening instrument for nonspecific psychological distress (Kessler et al., 2003). This SPD module was only administered to persons 18 or older. For details on the module for SPD, see Section B.4.4 in Appendix B of the 2007 NSDUH's national results report (OAS, 2008a, pp. 133-135).

In the 2004 NSDUH, the sample of respondents aged 18 or older was split evenly between the "long-form" module, which included all items in the mental health module used in the 2003 NSDUH (sample A), and a "short-form" module consisting only of the K6 items (sample B). In order to produce the pooled 2004-2005 SPD estimates, the 2004 sample A "long-form" scores were transformed to match the distributional characteristics of the 2004 sample B "short-form" scores using the cumulative distribution function (CDF) adjustment method described by Wright and Sathe (2006). These adjusted 2004 sample A scores were used in conjunction with the 2004 sample B "short-form" scores and the 2005 "short-form" SPD scores to produce the 2004-2005 SPD estimates. The SPD estimates in the 2003-2004 State report and other prior State reports are, therefore, not comparable with the 2004-2005, 2005-2006, and 2006-2007 SPD State estimates.

To produce the pooled 2006-2007 SPD estimates, data from both years (2006 and 2007) were combined. No transformations were required because both survey years used the same K6 scale for getting information on SPD. Estimates of change between the 2005-2006 and the 2006-2007 prevalence rates also are presented in this report.

A.10 Major Depressive Episode

Beginning in 2004, a module was included in the questionnaire that was related to having a major depressive episode (MDE); it was derived from the criteria specified for major depression in the DSM-IV (APA, 1994). These questions permit estimates to be calculated for lifetime and past year prevalence of MDE, treatment for MDE, and role impairment resulting from MDE. In this report, estimates of having at least one MDE in the past year are reported.

In 2005, 2006, and 2007 all adult and adolescent respondents were administered their respective depression modules. Due to minor wording differences in the questions in the adult and adolescent MDE modules, data from youths aged 12 to 17 were not combined with data from persons aged 18 or older to get an overall estimate for those aged 12 or older. Instead, an estimate for those aged 18 or older was produced. To produce the pooled 2006-2007 MDE estimates, the 2006 MDE data were pooled with the 2007 MDE data. Estimates of change between 2005-2006 and 2006-2007 MDE estimates also are included in this report.

According to DSM-IV, a person is defined as having had MDE in his or her lifetime if he or she has had at least five or more of the following nine symptoms nearly every day in the same 2-week period, where at least one of the symptoms is a depressed mood or loss of interest or pleasure in daily activities (APA, 1994): (1) depressed mood most of the day; (2) markedly diminished interest or pleasure in all or almost all activities most of the day; (3) significant weight loss when not sick or dieting, or weight gain when not pregnant or growing, or decrease or increase in appetite; (4) insomnia or hypersomnia; (5) psychomotor agitation or retardation; (6) fatigue or loss of energy; (7) feelings of worthlessness; (8) diminished ability to think or concentrate or indecisiveness; and (9) recurrent thoughts of death or suicidal ideation. Respondents who have had MDE in their lifetime are asked if, during the past 12 months, they had a period of depression lasting 2 weeks or longer while also having some of the other symptoms mentioned. Those reporting that they have are defined as having had MDE in the past year.

For details on the adult and adolescent modules for MDE, see Section B.4.5 in Appendix B of the 2007 NSDUH's national results report (OAS, 2008a, pp. 135-139).

A.11 Measuring Change in State Estimates between 2005-2006 and 2006-2007

The estimates of change between State small area estimates displayed in Appendix C are based on the 2005 through 2007 NSDUHs. The State estimates for 2005-2006 are the previously published model-based small area estimates (Hughes et al., 2008). The State estimates for 2006-2007 are the small area estimates given in Appendix B. The moving average State prevalence estimates for the overlapping 2005-2006 and 2006-2007 time periods were obtained from independent applications of RTI's SWHB methodology; that is, the 2006-2007 models were fit independently of the previously fitted 2005-2006 models. This independent analysis approach was followed because there was no desire to revise the previously published 2005-2006 estimates. Moreover, the same fixed predictor variables were used in the 2005-2006 and 2006-2007 models, but annual updates were made when more current versions became available. The age group–specific fixed predictor variables were defined at five levels (namely, person-level, census block group-level, tract-level, county-level, and State-level). Also, each age group model had 51 State-level random effects and 300 "within-State" area-level random effects.

To estimate change in State estimates, let π_sa(1) and π_sa(2) denote 2005-2006 and 2006-2007 prevalence rates, respectively, for State-s and age group-a. The change between π_sa(1) and π_sa(2) is defined in terms of the log-odds ratio (lor_sa) as opposed to the simple difference because the posterior distribution of the lor_sa is closer to Gaussian than the posterior distribution of the simple difference (π_sa(2) – π_sa(1)). The lor_sa is defined as

,     D

where ln denotes the natural logarithm. The p value given in the Appendix C tables is computed to test the null hypothesis of no change (i.e., π_sa(2) = π_sa(1) or equivalently lor_sa = 0). An estimate of lor_sa is given by

     D

where the p_sa(1) are previously published 2005-2006 State estimates and the p_sa(2) are the 2006-2007 State estimates presented in this report (see Appendix B). To compute the variance of i.e., let and then

     D

where denotes the covariance between and . This covariance is defined in terms of the associated correlation as follows:

.     D

Note that the and used here to calculate are the same variances used in calculating the previously published 2005-2006 PIs and the 2006-2007 PIs given in this report, respectively.

The correlation between and was obtained by simultaneously modeling the 2005, 2006, and 2007 NSDUH data. This simultaneous modeling approach was adopted based on the results of the validation study (see Appendix E, Section E.2, of Wright, 2003b) conducted for measuring change in 1999-2000 and 2000-2001 State estimates. For this simultaneous model, four age groups by 3 years (i.e., 12 subpopulation-specific models) were fitted, each with its own set of fixed and random effects. In this case, the general covariance matrices for the State and within-State random effects were 12 by 12 matrices corresponding to the 12 element (age group by year) vectors of random effects. Note that the survey-weighted Bernoulli-type log likelihood employed in SWHB methodology was appropriate for this simultaneous model because the 12 age group by year subpopulations were nonoverlapping. The correlation [, ] was approximated by the correlation calculated using the posterior distributions of ln[π_sa(1) / (1 – π_sa(1))] and ln[π_sa(2) / (1 – π_sa(2))] from the simultaneous model.

To calculate the p value for testing the null hypothesis of no difference (lor = 0), it is assumed that the posterior distribution of lor is normal with and With the null value of lor = 0, the Bayes p value or posterior probability of no difference is p value = 2*P [Z ≥ abs(z)], where Z is a standard normal random variate, , and abs(z) denotes the absolute value of z.

A.12 Method for Determining Differences between Two State Estimates for 2006-2007

This section describes a method for determining whether differences between two 2006-2007 State estimates are statistically significant. This procedure can be used for any two State estimates representing the same age group (e.g., young adults aged 18 to 25) and time period (e.g., 2006-2007).

Let π_1a and π_2a denote the 2006-2007 age group-a specific prevalence rates for two different States, s1 and s2, respectively. The null hypothesis of no difference, i.e., π_1a = π_2a, is equivalent to the log-odds ratio equal to zero, i.e., lor_a = 0, where lor_a is defined as , where ln denotes the natural logarithm. An estimate of lor_a is given by where p_1a and p_2a are the 2006-2007 State estimates given in Appendix B. To compute the variance of i.e., let and then

where denotes the covariance between and . This covariance is defined in terms of the associated correlation as follows:

.     D

The quantities and can be obtained by using the 95 percent PIs given in Appendix B. For this purpose, let (lower₁,upper₁) and (lower₂,upper₂) denote the 95 percent PIs for the two States, s1 and s2 respectively. Then

     D

where

For all practical purposes, the correlation between and is assumed to be negligible; hence, can be approximated by The correlation is assumed to be negligible because each State was a stratum in the first level of stratification; therefore, each State sample is selected independently. However, the correlation between the two State estimates is theoretically nonzero because State estimates share common fixed-effect parameters in the SAE models. Hence, the test statistic z (defined below) might result in a different conclusion in a few cases when the correlation between the State estimates is incorporated in calculating To calculate the p value for testing the null hypothesis of no difference (lor_a = 0), it is assumed that the posterior distribution of lor_a is normal with and With the null value of lor_a = 0, the Bayes p value or posterior probability of no difference is p value = 2*P [Z ≥abs(z)], where Z is a standard normal random variate, and abs (z) denotes the absolute value of z.

When comparing prevalence rates for two States, it is tempting and often convenient to look at their 95 percent PIs to decide whether the difference in the State prevalence rates is significant. If the two PIs overlap, one would conclude that the difference is not statistically significant. If the two PIs do not overlap, it implies that the State prevalence rates are significantly different from each other. However, the type-I error for the overlapping 95 percent PIs test is 0.6 percent (assuming that the two State estimates are uncorrelated and have the same variances) as compared with the 5 percent type-I error of the test based on the z statistics defined above (Payton, Greenstone, & Schenker, 2003). Thus, using the overlap method with 95 percent PIs implies a type-I error that is much less than the 5 percent level that is typically prescribed for such tests.

As discussed in Schenker and Gentleman (2001), the method of overlapping PIs is more conservative (i.e., it rejects the null hypothesis of no difference less often) than the standard method based on z statistics when the null hypothesis is true. Even if PIs for two States overlap, the two prevalence rates may be declared significantly different by the test based on z statistics. Hence, the method of overlapping PIs is not recommended to test the equivalence of two State prevalence rates. A detailed description of the method of overlapping confidence intervals (CIs) and its comparison with the standard methods for testing of a hypothesis is given in Schenker and Gentleman (2001) and Payton et al. (2003).

Example. The prevalence rates for past month alcohol use among 12 to 17 year olds in Alabama and Montana are shown in the exhibit below and also in Table B.9 in Appendix B. Looking at the two 95 percent PIs, it would appear that the Alabama and Montana prevalence rates for past month alcohol use are not statistically different at the 5 percent level of significance because the two PIs overlap:

State	Point Estimate (%)	95% PI (%)
Alabama	14.93	(12.80, 17.35)
Montana	18.81	(16.49, 21.38)

However, as shown below, the test based on the z statistic described earlier concludes that they are significantly different at the 5 percent level of significance.

Let p_1a = 0.1493, lower₁ = 0.1280, upper₁ = 0.1735, p_2a = 0.1881, lower₂ = 0.1649, upper₂ = 0.2138. Then,

     D

     D

     D

     D

Because the computed absolute value of z is greater than or equal to 1.96 (the critical value of the z statistic), then at the 5 percent level of significance, the hypothesis of no difference (Alabama prevalence rate = Montana prevalence rate) is rejected. Thus, the two State prevalence rates are statistically different. The Bayes p value or posterior probability of no difference is p value = 2*P [Z ≥ 2.268] = 0.023.