Appendix A: State Estimation Methodology

This report includes estimates of 25 substance use and mental health measures (see Section A.2) using the combined data from the 2008 and 2009 National Surveys on Drug Use and Health (NSDUHs). Also included in this report are comparisons between the 2007-2008 and the 2008-2009 State estimates and comparisons between the 2002-2003 and the 2008-2009 State estimates. As discussed in Chapter 1 (Section 1.1), several changes were introduced to the survey in 2002; thus, estimates for 2001 and prior years are not comparable with estimates from 2002 and later years.

The survey-weighted hierarchical Bayes (SWHB) methodology used in the production of State estimates from the 1999-2008 surveys also was used in the production of the 2008-2009 State estimates. The SWHB methodology is described in Appendix E of the 2001 State report (Wright, 2003b) and by Folsom, Shah, and Vaish (1999). The goals of small area estimation (SAE) modeling and the implementation of SAE modeling remain the same and are described in Appendix E of the 2001 State report (Wright, 2003b). A general model description is given in Section A.1. A list of outcomes for which small area estimates are produced in this report is given in Section A.2. The list of predictors used in the 2008-2009 SAE modeling is given in Section A.3. Information on the new population projections obtained from Claritas that were used for the first time in producing the 2007-2008 small area estimates and how they were used to create SAE model predictors is given in Section A.4. New variable selection was done for the mental health outcomes using the 2008-2009 data. For other outcomes, no new variable selection was done (as discussed in Section A.5).

Small area estimates obtained using the SWHB methodology are design consistent (i.e., the small area estimates for States with large sample sizes are close to the robust design-based estimates). The State small area estimates when aggregated using the appropriate population totals result in national small area estimates that are very close to the national design-based estimates. However, for numerous reasons (including internal consistency), it is desirable to have national small area estimates exactly match the national design-based estimates. Beginning in 2002, exact benchmarking was introduced as described in Section A.6.⁹ Tables of estimated numbers of persons associated with each outcome (in thousands) are available on the Web in the form of HTML tables (see http://www.samhsa.gov/data/2k9State/TOC.htm). An explanation of how these counts and their respective Bayesian confidence intervals¹⁰ are calculated can be found in Section A.7. The definition and explanation of the formula used in estimating the marijuana incidence rate is given in Section A.8.

For all outcomes except major depressive episode (i.e., depression), serious mental illness, any mental illness, and past year serious thoughts of suicide, the age groups for which estimates are provided in this report are 12 to 17, 18 to 25, and 26 or older. Estimates for those aged 12 or older also are provided in this report. Because it was determined that States may find estimates for persons aged 18 or older useful, estimates for that age group are available on the Web in the form of HTML tables (see http://www.samhsa.gov/data/2k9State/TOC.htm). Also included in this report are estimates of underage (aged 12 to 20) alcohol use and binge alcohol use. Alcohol consumption is expected to differ significantly across the 18 to 25 age group because of the legalization of alcohol at age 21. Therefore, it was decided that it would be useful to produce small area estimates for persons aged 12 to 20. A short description of the methodology used to produce underage drinking estimates is provided in Section A.9.

Section A.10 discusses the criteria used to define dependence on and abuse of illicit drugs and alcohol. Section A.11 discusses the production of estimates for major depressive episode (i.e., depression), serious mental illness, any mental illness, and suicidal thoughts. Note that for major depressive episode, there are no 12 or older estimates published; also, for serious mental illness, any mental illness, and serious thoughts of suicide, no 12 to 17 estimates are produced because youths are not asked these questions. Section A.12 discusses the method to compare prevalence rates of a particular outcome between two States. The methodology used to compare the 2007-2008 and the 2008-2009 State estimates and the 2002-2003 and the 2008-2009 State estimates is described in Section A.13.

At the end of this appendix, tables showing the 2007, 2008, 2009, pooled 2007-2008, and pooled 2008-2009 survey sample sizes, population estimates, and response rates are included (Tables A.1 to A.14). Table A.15 lists all outcomes and the years for which small area estimates were produced going back to the 2002 NSDUH.

A.1 General Model Description

The model can be characterized as a complex mixed model (including both fixed and random effects) of the following form:

,     D

where is the probability of engaging in the behavior of interest (e.g., using marijuana in the past month) for person-k belonging to age group-a in State sampling (SS) region-j of State-i. Let denote a vector of auxiliary (predictor) variables associated with age group-a (12 to 17, 18 to 25, 26 to 34, and 35 or older) and denote the associated vector of regression parameters. The age group-specific vectors of auxiliary variables are defined for every block group in the Nation and also include person-level demographic variables, such as race/ethnicity and gender. The vectors of State-level random effects and SS region-level random effects are assumed to be mutually independent with and where A is the total number of individual age groups modeled (generally, A=4). For hierarchical Bayes (HB) estimation purposes, an improper uniform prior distribution is assumed for , and proper Wishart prior distributions are assumed for and . The HB solution for involves a series of complex Markov Chain Monte Carlo (MCMC) steps to generate values of the desired fixed and random effects from the underlying joint posterior distribution. The basic process is described in Folsom et al. (1999), Shah, Barnwell, Folsom, and Vaish (2000), and Wright (2003a, 2003b).

Once the required number of MCMC samples for the parameters of interest are generated and tested for convergence properties (see Raftery & Lewis, 1992), the small area estimates for each age group × race/ethnicity × gender cell within a block group can be obtained. These block group-level small area estimates then can be aggregated using the appropriate population count projections to form State-level small area estimates for the desired age group(s). These State-level small area estimates are benchmarked to the national design-based estimates as described in Section A.6.

A.2 Variables Modeled

The 2009 NSDUH data were pooled with the 2008 NSDUH data, and age group-specific State prevalence estimates for 25 binary (0, 1) outcome variables were produced and presented in this report in Appendix B. Estimates were produced for the following outcomes:

1. past month use of illicit drugs,
   
   
2. past year use of marijuana,
   
   
3. past month use of marijuana,
   
   
4. perception of great risk of smoking marijuana once a month,
   
   
5. average annual rate of first use of marijuana,
   
   
6. past month use of illicit drugs other than marijuana,
   
   
7. past year use of cocaine,
   
   
8. past year nonmedical use of pain relievers,
   
   
9. past month use of alcohol,
   
   
10. past month binge alcohol use,
    
    
11. perception of great risk of having five or more drinks of an alcoholic beverage once or twice a week,
    
    
12. past month use of tobacco products,
    
    
13. past month use of cigarettes,
    
    
14. perception of great risk of smoking one or more packs of cigarettes per day,
    
    
15. past year alcohol dependence or abuse,
    
    
16. past year alcohol dependence,
    
    
17. past year illicit drug dependence or abuse,
    
    
18. past year illicit drug dependence,
    
    
19. past year dependence on or abuse of illicit drugs or alcohol,
    
    
20. needing but not receiving treatment for illicit drug use in the past year,
    
    
21. needing but not receiving treatment for alcohol use in the past year,
    
    
22. serious mental illness in the past year,
    
    
23. any mental illness in the past year,
    
    
24. serious thoughts of suicide in the past year, and
    
    
25. past year major depressive episode (i.e., depression).

Comparisons between the 2007-2008 and the 2008-2009 State estimates were produced for all of these outcomes except serious mental illness, any mental illness, and serious thoughts of suicide and are included in this report in Appendix C. In addition, tests of change between the 2002-2003 and the 2008-2009 State estimates were produced for all outcomes except major depressive disorder, serious mental illness, any mental illness, and past year serious thoughts of suicide and are included in this report in Appendix D. Note that the mental health outcomes included in this report are either being reported for the first time or are not comparable with estimates from prior years (except for the major depressive episode estimates for youths that are comparable with estimates from previous years).

A.3 Predictors Used in Mixed Logistic Regression Models

Local area data used as potential predictor variables in the mixed logistic regression models were obtained from several sources, including Claritas Inc., the U.S. Census Bureau, the Federal Bureau of Investigation (FBI) (Uniform Crime Reports), Health Resources and Services Administration (Area Resource File), the Bureau of Labor Statistics, the Bureau of Economic Analysis, the Substance Abuse and Mental Health Services Administration (SAMHSA) (National Survey of Substance Abuse Treatment Services [N-SSATS]), and the National Center for Health Statistics (mortality data). The values of these predictor variables are updated every year (when possible). Sources and potential data items used in the modeling are provided in the following text and lists.

• Claritas. This demographic data package contains data for 2008 with projections to 2012. The projections are used to update these predictor variables each year. The 2008 and 2009 projections were used this year. For more information on these data, see Section A.4.
  
  
• U.S. Census Bureau. The 2000 census (demographic and socioeconomic variables) and 2007 food stamp participation rates (http://www.census.gov/hhes/www/saipe/inputdata/cntyfs.xls). The Census Bureau's small area income and poverty estimates division obtains Food Stamp Program (now known as the Supplemental Nutrition Assistance Program) participation rates from the U.S. Department of Agriculture, Food and Nutrition Service.
  
  
• Federal Bureau of Investigation. Uniform Crime Report (UCR) arrest totals from http://www.icpsr.umich.edu/icpsrweb/NACJD/support/faqs/597165391602908571. The most current data used are from 2007 for most counties, with previous years' data substituted in a few cases.
  
  
• Health Resources and Services Administration. Families below the poverty-level rates from the 2000 census data are obtained from the Area Resource File (ARF) February 2005 release from the Bureau of Health Professions, Office of Research and Planning.
  
  
• Bureau of Labor Statistics (BLS). The 2009 county-level unemployment rates (ftp://ftp.bls.gov/pub/special.requests/la/laucnty09.txt). The BLS uses results from the Current Population Survey (CPS) to provide county-level unemployment rates. The CPS is a monthly survey of households conducted by the Census Bureau for the BLS.
  
  
• Bureau of Economic Analysis (BEA). The 2008 county-level per capita income rates (http://www.bea.gov/regional/index.htm). These county-level per capita income rates are produced by the Regional Income Division of the BEA.
  
  
• National Center for Health Statistics. Mortality data using International Classification of Diseases, 10th revision (ICD-10), 2002-2007. The ICD-10 death rate data are from the National Center for Health Statistics at the Centers for Disease Control and Prevention.
  
  
• SAMHSA, Center for Behavioral Health Statistics and Quality (CBHSQ), formerly the Office of Applied Studies (OAS). National Survey of Substance Abuse Treatment Services (N-SSATS), formerly known as the Uniform Facility Data Set (UFDS). The 2008-2009 data on drug and alcohol treatment rates are obtained from Synectics for Management Decisions, Inc.

The following lists provide the specific independent variables that were potential predictors in the models.

Claritas Data (Description)	Claritas Data (Level)
% Population Aged 0 to 19 in Block Group	Block Group
% Population Aged 20 to 24 in Block Group	Block Group
% Population Aged 25 to 34 in Block Group	Block Group
% Population Aged 35 to 44 in Block Group	Block Group
% Population Aged 45 to 54 in Block Group	Block Group
% Population Aged 55 to 64 in Block Group	Block Group
% Population Aged 65 or Older in Block Group	Block Group
% Non-Hispanic Blacks in Block Group	Block Group
% Hispanics in Block Group	Block Group
% Non-Hispanic Other Races in Block Group	Block Group
% Non-Hispanic Whites in Block Group	Block Group
% Males in Block Group	Block Group
% Females in Block Group	Block Group
% American Indians, Eskimos, Aleuts in Tract	Tract
% Asians, Pacific Islanders in Tract	Tract
% Population Aged 0 to 19 in Tract	Tract
% Population Aged 20 to 24 in Tract	Tract
% Population Aged 25 to 34 in Tract	Tract
% Population Aged 35 to 44 in Tract	Tract
% Population Aged 45 to 54 in Tract	Tract
% Population Aged 55 to 64 in Tract	Tract
% Population Aged 65 or Older in Tract	Tract
% Non-Hispanic Blacks in Tract	Tract
% Hispanics in Tract	Tract
% Non-Hispanic Other Races in Tract	Tract
% Non-Hispanic Whites in Tract	Tract
% Males in Tract	Tract
% Females in Tract	Tract
% Population Aged 0 to 19 in County	County
% Population Aged 20 to 24 in County	County
% Population Aged 25 to 34 in County	County
% Population Aged 35 to 44 in County	County
% Population Aged 45 to 54 in County	County
% Population Aged 55 to 64 in County	County
% Population Aged 65 or Older in County	County
% Non-Hispanic Blacks in County	County
% Hispanics in County	County
% Non-Hispanic Other Races in County	County
% Non-Hispanic Whites in County	County
% Males in County	County
% Females in County	County

2000 Census Data (Description)	2000 Census Data (Level)
% Population Who Dropped Out of High School	Tract
% Housing Units Built in 1940 to 1949	Tract
% Persons Aged 16 to 64 with a Work Disability	Tract
% Hispanics Who Are Cuban	Tract
% Females 16 Years or Older in Labor Force	Tract
% Females Never Married	Tract
% Females Separated, Divorced, Widowed, or Other	Tract
% One-Person Households	Tract
% Female Heads of Household, No Spouse, Child #under 18	Tract
% Males 16 Years or Older in Labor Force	Tract
% Males Never Married	Tract
% Males Separated, Divorced, Widowed, or Other	Tract
% Housing Units Built in 1939 or Earlier	Tract
Average Persons per Room	Tract
% Families below Poverty Level	Tract
% Households with Public Assistance Income	Tract
% Housing Units Rented	Tract
% Population with 9 to 12 Years of School, No High School Diploma	Tract
% Population with 0 to 8 Years of School	Tract
% Population with Associate's Degree	Tract
% Population with Some College and No Degree	Tract
% Population with Bachelor's, Graduate, Professional Degree	Tract
Median Rents for Rental Units	Tract
Median Value of Owner-Occupied Housing Units	Tract
Median Household Income	Tract

Uniform Crime Report Data (Description)	Uniform Crime Report Data (Level)
Drug Possession Arrest Rate	County
Drug Sale or Manufacture Arrest Rate	County
Drug Violations' Arrest Rate	County
Marijuana Possession Arrest Rate	County
Marijuana Sale or Manufacture Arrest Rate	County
Opium or Cocaine Possession Arrest Rate	County
Opium or Cocaine Sale or Manufacture Arrest Rate	County
Other Drug Possession Arrest Rate	County
Other Dangerous Non-Narcotics Arrest Rate	County
Serious Crime Arrest Rate	County
Violent Crime Arrest Rate	County
Driving under Influence Arrest Rate	County

Other Categorical Data (Description)	Other Categorical Data (Source)	Other Categorical Data (Level)
= 1 if Hispanic, = 0 Otherwise	NSDUH Sample	Person
= 1 if Non-Hispanic Black, = 0 Otherwise	NSDUH Sample	Person
= 1 if Non-Hispanic Other, = 0 Otherwise	NSDUH Sample	Person
= 1 if Male, = 0 if Female	NSDUH Sample	Person
= 1 if MSA with ≥ 1 Million, = 0 Otherwise	2000 Census	County
= 1 if MSA with < 1 Million, = 0 Otherwise	2000 Census	County
= 1 if Non-MSA Urban, = 0 Otherwise	2000 Census	Tract
= 1 if Urban Area, = 0 if Rural Area	2000 Census	Tract
= 1 if No Cubans in Tract, = 0 Otherwise	2000 Census	Tract
= 1 if No Arrests for Dangerous Non-Narcotics, = 0 Otherwise	UCR	County

Miscellaneous Data (Variable Description)	Miscellaneous Data (Source)	Miscellaneous Data (Level)
Alcohol Death Rate, Underlying Cause	NCHS-ICD-10	County
Cigarette Death Rate, Underlying Cause	NCHS-ICD-10	County
Drug Death Rate, Underlying Cause	NCHS-ICD-10	County
Alcohol Treatment Rate	N-SSATS (Formerly Called UFDS)	County
Alcohol and Drug Treatment Rate	N-SSATS (Formerly Called UFDS)	County
Drug Treatment Rate	N-SSATS (Formerly Called UFDS)	County
% Families below Poverty Level	ARF	County
Unemployment Rate	BLS	County
Per Capita Income (in Thousands)	BEA	County
Average Suicide Rate (per 10,000)	NCHS-ICD-10	County
Food Stamp Participation Rate	Census Bureau	County
Single State Agency Maintenance of Effort	National Association of State Alcohol and Drug Abuse Directors (NASADAD)	State
Block Grant Awards	SAMHSA	State
Cost of Services Factor Index	SAMHSA	State
Total Taxable Resources Per Capita Index	U.S. Department of Treasury	State

A.4 Updated Claritas Data

For the State and substate reports published using the 2002 to 2007 NSDUH data, Claritas data obtained in 2002 were used to produce the small area estimates. In reports published using the 2008 and 2009 NSDUH data, Claritas data obtained in 2008 were used. The 2002 Claritas data had 2000 and 2002 population counts, as well as 2007 population projections. The 2008 Claritas data had 2008 population counts, as well as 2012 population projections. Claritas data were used for the following in the NSDUH SAE process:

1. Creating demographic predictor variables (age group, race × ethnicity, and gender) at the block group, tract, and county levels (predictors such as percentage of the population aged 0 to 19 in a block group, percentage of population who are males in a tract). There are 13 such variables defined for each of the census geographies (block group, tract, and county). See Section A.3 for a complete list of these predictors.
   
   
2. Creating census block group-level population projections at the age group × race/ethnicity × gender level (4 age groups, 4 races/ethnicities, and 2 genders = 32 cells) that are used in producing aggregate State-level small area estimates.¹¹
   
   

1. In the 2008 SAE process (and subsequent years), new Claritas data with 2008 population counts and 2012 population projections were used. The new Claritas data will be henceforth referred to as the 2008-2012 Claritas data, and the 2002 Claritas data will be referred to as the 2002-2007 Claritas data. After doing some data exploration on the 2008-2012 Claritas data and comparing them with the 2002-2007 Claritas data, some differences were observed when comparing the 2007 population counts (from the 2002-2007 Claritas data) with the 2008 population counts (from the 2008-2012 Claritas data). For example, the distributions of the population aged 20 to 24 in block groups were very different for the two datasets. Another difference was that there were more block groups that had a 0 population count for some of the 32 cells in 2008 as compared with the 32 cells in 2007.
   
   
2. The format of the race/ethnicity data was also different for the two sets of Claritas data. To generate age group × race × Hispanicity × gender population counts at the block group level using the 2002-2007 Claritas data, two separate population distributions (age × gender × race and race × Hispanicity) at the block group level had to be used. The assumption that each of the age × gender cells within a race group had the same Hispanicity distribution was made. So, the data were manipulated to get the desired four-way cross of demographic domains. The 2008-2012 Claritas data had age group × race × Hispanicity × gender population distributions, so no assumptions or manipulations to the data had to be made.
   
   

3. In State reports prior to 2008 when creating the 32 cells using the 2002-2007 Claritas data, the population from the 2 or more races category was distributed among the black, white, and other race categories. Starting in 2008 and subsequent years, a decision was made to merge the two or more races category with the other race category. This was based on a decision to discontinue creating a NSDUH sample variable that split the two or more races respondents into black, white, or other. Because NSDUH respondents with 2 or more races were now being grouped into the other category, the same technique was used to produce the 32 cell counts.

Some of the data differences can be attributed to reasons (2b) and (3), and the rest are most likely attributed to the fact that the 2008-2012 Claritas projections are based on updated population information. Because of these differences in the 2007 population projections based on 2002-2007 Claritas data and the 2008 population counts based on 2008-2012 Claritas data, it was decided that "new" 2007 population projections would be obtained by "projecting back" the 2008-2012 Claritas data. Population projections for 2006 also were obtained in the same manner, so that they could be used in the 2006-2008 SAE reports.

Based on the information above, the following steps were taken for the 2008-2009 SAE process (for more information on the steps taken for the 2007-2008 SAE process, see Appendix A of Hughes et al., 2010):

1. Using the 2008-2012 Claritas data, 2008 and 2009 population counts were obtained (the 2008 counts were extracted directly from the file and the 2009 counts were obtained by using linear interpolation between the 2008 and 2012 counts) and used to create the predictors that were merged onto the 2008 and 2009 sample and universe files (the universe file is a census block-group level file containing SAE predictor variables and population counts).
   
   
2. All block group, tract, and county-level continuous predictors were converted into 10-category, semicontinuous variables by using the corresponding 2007-2008 decile values created by pooling the 2007 and 2008 NSDUH data. The same 2007-2008 decile values will be used for future SAE analyses until new Claritas data containing the 2013 population counts and projections are obtained. Using the same decile values year after year makes it possible to keep track of any temporal changes occurring in the predictor variables, which may help in detecting any changes in State prevalence rates across years in an efficient manner. The 10-category predictor variables subsequently were used to form linear, quadratic, and cubic orthogonal polynomials eventually used in the SAE modeling process.
   
   
3. The updated population counts for the 32 cells (age group × race/ethnicity × gender population counts) were used to create the universe files for both years (i.e., 2008 and 2009).
   
   
4. The updated 2007 sample and universe files based on the 2008-2012 Claritas data were used in simultaneous modeling (see Section A.13) to produce the correlations required to estimate change between the 2007-2008 and 2008-2009 State prevalence rates.
   
   
5. For any analysis using 2002 through 2005 NSDUH data, the old projections based on the 2002-2007 Claritas data were used (i.e., for the 2002-2003 vs. 2008-2009 comparisons, the 2002-2007 Claritas projections were used on the 2002-2003 sample and universe files, whereas the 2008-2012 Claritas projections were used on the 2008-2009 sample and universe files).

A.5 Selection of Independent Variables for the Models

New variable selection was done for serious mental illness, any mental illness, serious thoughts of suicide, and major depressive episode (i.e., depression) for persons aged 18 or older using the pooled 2008-2009 NSDUH data. Estimates for serious mental illness, any mental illness, and serious thoughts of suicide are being produced in this report for the first time; hence, no prior fixed-effect predictors were available. The serious mental illness estimates in prior State reports (e.g., 2001, 2002, and 2003 State reports) were based on a different definition. For major depressive episode, the variable selection for adults aged 18 or older is based on an adjusted major depressive episode variable (for details, see Section A.11). Fixed-effect predictors for the new outcome variables were selected using the method described by Wright and Sathe (2005).

For all of the other outcomes (including major depressive episode for youths aged 12 to 17), no new variable selection was done. The updated versions of fixed-effect predictors that were used in modeling the 2007-2008 data were used to model the 2008-2009 data. Because the interest was to estimate change between the 2007-2008 and 2008-2009 State estimates, the same set of fixed-effect predictors was used for producing both sets of estimates.

A.6 Benchmarking the Age Group-Specific Small Area Estimates

The self-calibration built into the SWHB solution ensures that the population-weighted average of the State small area estimates will closely match the national design-based estimates. The national design-based estimates in NSDUH are based entirely on survey-weighted data using a direct estimation approach, whereas the State and census region estimates in this report are model-based. Given the self-calibration ensured by the SWHB solution, for State reports prior to 2002, the standard Bayes prescription was followed; specifically, the posterior mean was used for the point estimate, and the tail percentiles of the posterior distribution were used for the Bayesian confidence interval limits.

Singh and Folsom (2001) extended Ghosh's (1992) results on constrained Bayes estimation to include exact benchmarking to design-based national estimates. In the simplest version of this constrained Bayes solution where only the design-based mean is imposed as a benchmarking constraint, each of the State-by-age group small area estimates (for 2008-2009) is adjusted by adding the common factor where is the design-based national prevalence estimate and is the population-weighted mean of the State small area estimates for age group-a. The exactly benchmarked State-s and age group-a small area estimates then are given by Experience with such additive adjustments suggests that the resulting exactly benchmarked State small area estimates will always be between 0 and 100 percent because the SWHB self-calibration ensures that the adjustment factor is small relative to the size of the State-level small area estimates.

Relative to the Bayes posterior mean, these benchmark-constrained State small area estimates are biased by the common additive adjustment factor. Therefore, the posterior mean-squared error for each benchmarked State small area estimate has the square of this adjustment factor added to its posterior variance. To achieve the desirable feature of exact benchmarking, this constrained Bayes adjustment factor was implemented for the State-by-age group small area estimates. The associated Bayesian confidence (credible) intervals can be recentered at the benchmarked small area estimates on the logit scale with the symmetric interval end points based on the posterior root mean-squared errors. The adjusted 95 percent Bayesian confidence intervals are defined below:

,     D

where

,     D

, and     D

.     D

The associated posterior coverage probabilities for these benchmarked intervals are very close to the prescribed 0.95 value because the State small area estimates have posterior distributions that can be approximated exceptionally well by a Gaussian distribution.

A.7 Calculation of Estimated Number of Persons Associated with Each Outcome

Tables 1 to 26, available at http://www.samhsa.gov/data/2k9State/TOC.htm, show the estimated numbers of persons (in thousands) associated with each of the 25 outcomes of interest. To calculate these estimated numbers of persons, the benchmarked small area estimates and the associated 95 percent Bayesian confidence intervals are multiplied by the average population across the 2 years (in this case, 2008 and 2009) of the State by age group of interest.

For example, past month use of alcohol among 18 to 25 year olds in Alabama was 54.51 percent (see Table B.9 in Appendix B). The corresponding Bayesian confidence intervals ranged from 50.59 to 58.37 percent. The population count for 18 to 25 year olds averaged across 2008-2009 in Alabama was 505,718 (see Table A.10). Hence, the estimated number of 18 to 25 year olds using alcohol in the past month in Alabama was 0.5451 * 505,718, which is 275,667 (see Table 9). The associated Bayesian confidence intervals ranged from 0.5059 * 505,718 (i.e., 255,843) to 0.5837 * 505,718 (i.e., 295,188). Note that when estimates of the number of persons are calculated for Tables 1 to 26, the unrounded prevalence estimates and population counts are used. Hence, the number obtained by multiplying the published prevalence rate with the published population estimate may not exactly match the counts that are published in these tables due to rounding differences.

A.8 Calculation of Average Annual Incidence of Marijuana Use

Incidence rates typically are calculated as the number of new initiates of a substance during a period of time (such as in the past year) divided by an estimate of the number of person years of exposure (in thousands). The incidence definition used in this report employs a simpler form of the at-risk population based on the model-based methodology. This model-based average annual incidence rate is defined as follows:

,     D

where is the number of marijuana initiates in the past 24 months and is the number of persons who never used marijuana.

In this report, the incidence rate is expressed as a percentage or rate per 100 person years of exposure. Note that this estimate uses a 2-year time period to accumulate incidence cases from each annual survey. By assuming further that the distribution of first use for the incidence cases is uniform across the 2-year interval, the total number of person years of exposure is 1 year on average for the incidence cases plus 2 years for all the "never users" at the end of the time period. This approximation to the person years of exposure permits one to recast the incidence rate as a function of two population prevalence rates, namely, the fraction of persons who first used marijuana in the past 2 years and the fraction who had never used marijuana. Both of these prevalence estimates were estimated using the SWHB estimation approach.

The count of persons who first used marijuana in the past 2 years is based on a "moving" 2-year period that ranges over 3 calendar years. Subjects were asked when they first used marijuana. If a person indicated first use of marijuana between the day of the interview and 2 years prior, the person was included in the count. Thus, it is possible for a person interviewed in the first part of 2009 to indicate first use as early as the first part of 2007 or as late as the first part of 2009. Similarly, a subject interviewed in the last part of 2009 could indicate first use as early as the last part of 2007 or as late as the last part of 2009. Therefore, in the 2009 survey, the reported period of first use ranged from early 2007 to late 2009 and was "centered" in 2008. For example, about half of the 12 to 17 year olds who reported first use in the past 24 months reported first use in 2008, while a quarter each reported first use in 2007 and 2009. Persons who responded in 2009 that they had never used marijuana were included in the count of "never used." Similarly, reports of first use in the past 24 months from the 2008 survey ranged from early 2006 to late 2008 and were centered in 2007. Half of the 12 to 17 year olds who reported first use in the past 24 months reported first use in 2007, while a quarter each reported first use in 2006 and 2008. Note that only incidence rates for marijuana use are provided in this report.

A.9 Underage Drinking

To obtain small area estimates for persons aged 12 to 20 for past month alcohol and binge alcohol use, a separate set of models was fit for these two outcomes for the 12 to 17 age group and the 18 to 20 age group. For the 2008-2009 models, no new variable selection was done. Updated versions of the predictors were used to produce the small area estimates.

Model-based estimates for persons aged 12 to 20 were produced by taking the population-weighted average of the individual age group (12 to 17 and 18 to 20) estimates. Estimates for underage drinking for past month alcohol and binge alcohol use were benchmarked to match national design-based estimates for that age group using the process described in Section A.6. Comparisons between the 2007-2008 and the 2008-2009 small area estimates for underage drinking in the States also are presented in this report.

A.10 Illicit Drug and Alcohol Dependence and Abuse

The NSDUH computer-assisted interviewing (CAI) instrumentation includes questions that are designed to measure dependence on and abuse of illicit drugs and alcohol. For these substances,¹² dependence and abuse questions were based on the criteria in the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) (American Psychiatric Association [APA], 1994).

Specifically, for marijuana, hallucinogens, inhalants, and tranquilizers, a respondent was defined as having dependence if he or she met three or more of the following six dependence criteria:

1. Spent a great deal of time over a period of a month getting, using, or getting over the effects of the substance.
   
   
2. Used the substance more often than intended or was unable to keep set limits on the substance use.
   
   
3. Needed to use the substance more than before to get desired effects or noticed that the same amount of substance use had less effect than before.
   
   
4. Inability to cut down or stop using the substance every time tried or wanted to.
   
   
5. Continued to use the substance even though it was causing problems with emotions, nerves, mental health, or physical problems.
   
   
6. The substance use reduced or eliminated involvement or participation in important activities.

For alcohol, cocaine, heroin, pain relievers, sedatives, and stimulants, a seventh withdrawal criterion was added. A respondent was defined as having dependence if he or she met three or more of seven dependence criteria. The seventh withdrawal criterion is defined by a respondent reporting having experienced a certain number of withdrawal symptoms that vary by substance (e.g., having trouble sleeping, cramps, hands tremble).

For each illicit drug and alcohol, a respondent was defined as having abused that substance if he or she met one or more of the following four abuse criteria and was determined not to be dependent on the respective substance in the past year:

1. Serious problems at home, work, or school caused by the substance, such as neglecting your children, missing work or school, doing a poor job at work or school, or losing a job or dropping out of school.
   
   
2. Used the substance regularly and then did something that might have put you in physical danger.
   
   
3. Use of the substance caused you to do things that repeatedly got you in trouble with the law.
   
   
4. Had problems with family or friends that were probably caused by using the substance and continued to use the substance even though you thought the substance use caused these problems.

For additional details on how respondents were classified as being dependent on or having abused illicit drugs and alcohol, see Section B.4.3 in Appendix B of the 2009 NSDUH national findings report (OAS, 2010b, pp. 26-28).

A.11 Mental Health Measures

This section provides a summary of measurement issues associated with the four mental health outcome variables included in this report—serious mental illness, any mental illness, serious thoughts of suicide, and major depressive episode. Additional details can be found in Sections B.4.6 and B.4.7 of Appendix B in the 2008 NSDUH national findings report for serious mental illness and major depressive episode, respectively (OAS, 2009), and Sections B.4.2 to B.4.4 of Appendix B in the 2009 NSDUH mental health findings report for all four outcome variables (CBHSQ, 2010).

A.11.1 Serious Mental Illness

In the 2000-2001 and 2002-2003 NSDUH State reports, the Kessler-6 (K6) distress scale was used to measure serious mental illness (Kessler et al., 2003). However, SAMHSA discontinued producing State-level serious mental illness estimates beginning with the release of the 2003-2004 State report because of concerns about the validity of using only the K6 distress scale without an impairment scale; see Section B.4.4 of Appendix B in the 2004 NSDUH national findings report (OAS, 2005). The use of the K6 distress scale continued in the 2003-2004, 2004-2005, 2005-2006, and 2006-2007 State reports, but the outcome measure was changed from serious mental illness to serious psychological distress because it was determined that the K6 scale only measured serious psychological distress and only contributed to measuring serious mental illness (see details below).

In December 2006, a technical advisory group meeting of expert consultants was convened by SAMHSA's Center for Mental Health Services to solicit recommendations for mental health surveillance data collection strategies among the U.S. population. The panel recommended that NSDUH should be used to produce estimates of serious mental illness among adults using NSDUH's mental health measures and a gold-standard clinical psychiatric interview. In response, SAMHSA's CBHSQ initiated a Mental Health Surveillance Study (MHSS) under its NSDUH contract with RTI International to develop and implement methods to estimate serious mental illness. Based on recommendations from this panel, estimates of serious mental illness presented in this report for 2008 and 2009 are based on a revised methodology and, thus, are not comparable with serious mental illness estimates or serious psychological distress estimates shown in prior NSDUH State reports.

To develop methods for preparing the estimates of serious mental illness and any mental illness presented in this and other NSDUH reports, the MHSS was initiated as part of the 2008 NSDUH design and analysis. Because of constraints on the interview time in NSDUH and the need for trained mental health clinicians, it was not possible to administer a full structured diagnostic clinical interview to assess mental illness on approximately 45,000 adult respondents; therefore, the approach adopted by SAMHSA was to utilize short scales separately measuring psychological distress (K6) and functional impairment that could be used in a statistical model to accurately predict whether a respondent had a mental illness. Two impairment scales—the World Health Organization Disability Assessment Schedule (WHODAS) and the Sheehan Disability Scale (SDS)—were included in the 2008 survey for evaluation. The collection of clinical psychiatric interview data was achieved using a subsample of approximately 1,500 adult NSDUH participants in 2008. These participants were recruited for a follow-up clinical interview consisting of a gold-standard diagnostic assessment for mental disorders and functional impairment. In order to determine the optimal scale to measure functional impairment, a split-sample design was incorporated into the full 2008 NSDUH data collection in which half of the adult respondents received the WHODAS and half received the SDS. Statistical models using the data from the subsample of respondents collected as part of the MHSS then were developed for each half sample in which the short scales (the K6 in combination with the WHODAS or the K6 in combination with the SDS) were used as predictors in models of mental illness assessed via the clinical interviews. The model parameter estimates were then used to predict serious mental illness in the full 2008 NSDUH sample.

Kessler-6 Distress Scale

The K6 in NSDUH consists of two sets of six questions that asked adult respondents how frequently they experienced symptoms of psychological distress during two different time periods: (1) during the past 30 days, and (2) if applicable, the one month in the past year when they were at their worst emotionally. Respondents were asked about the second time period only if they indicated that there was a month in the past 12 months when they felt more depressed, anxious, or emotionally stressed than they felt during the past 30 days.

The six questions comprising the K6 scale for the past month are as follows:

NERVE30

During the past 30 days, how often did you feel nervous?

1   All of the time
2   Most of the time
3   Some of the time
4   A little of the time
5   None of the time
Don't know/Refused

Response categories are the same for the remaining questions shown below.

HOPE30

During the past 30 days, how often did you feel hopeless?

FIDG30

During the past 30 days, how often did you feel restless or fidgety?

NOCHR30

During the past 30 days, how often did you feel so sad or depressed that nothing could cheer you up?

EFFORT30

During the past 30 days, how often did you feel that everything was an effort?

DOWN30

During the past 30 days, how often did you feel down on yourself, no good or worthless?

To create a score, the six items (NERV30, HOPE30, FIDG30, NOCHR30, EFFORT30, and DOWN30) on the K6 scale were recoded from 0 to 4 so that "all of the time" was coded 4, "most of the time" 3, "some of the time" 2, "a little of the time" 1, and "none of the time" 0, with "don't know" and "refused" also coded as 0. Summing across the transformed responses in these six items resulted in a score with a range from 0 to 24.

If respondents were asked about a month in the past 12 months when they felt more depressed, anxious, or emotionally stressed than they felt during the past 30 days, they were asked comparable K6 items for that particular month in the past 12 months. The scoring procedures for these K6 items for the past 12 months were the same as those described above. The higher of the two K6 total scores for the past 30 days or past 12 months was used both for MHSS analysis purposes and in the adult respondents' final data.

An alternative K6 total score also was created in which K6 scores less than 8 were recoded as 0 and scores from 8 to 24 were recoded as 1 to 17. The rationale for creating the alternative past year K6 score was that serious mental illness prevalence was typically extremely low for respondents with past year K6 scores less than 8, and the prevalence rates started increasing only when scores were 8 or greater.

MHSS Clinical Interviews

As described previously, a subsample of approximately 1,500 adult NSDUH participants in 2008 completed follow-up clinical interviews to provide data for the statistical modeling of the NSDUH interview data of psychological distress and functional impairment on mental health status. The MHSS sample respondents were administered clinical interviews within 4 weeks of the NSDUH main interview to assess the presence of mental disorders and functional impairment. Specifically, each participant was assessed by a trained clinical interviewer (master's or doctoral-level clinician, counselor, or social worker) via paper-and-pencil interviewing (PAPI) over the telephone. The clinical interview used was an adapted version of the Structured Clinical Interview for DSM-IV-TR Axis I Disorders, Research Version, Non-patient Edition (SCID-I/NP) (First, Spitzer, Gibbon, & Williams, 2002). Past year disorders that were assessed through the SCID included mood disorders (e.g., major depressive episode, manic episode), anxiety disorders (e.g., panic disorder, generalized anxiety disorder, posttraumatic stress disorder), eating disorders (e.g., anorexia nervosa), intermittent explosive disorder, and adjustment disorder. In addition, the presence of psychotic symptoms was assessed. Substance use disorders also were assessed, although these disorders were not included in the estimates of mental illness.

Functional impairment ratings were assigned by clinical interviewers using the Global Assessment of Functioning (GAF) scale (Endicott, Spitzer, Fleiss, & Cohen, 1976). Mental illness, measured using the SCID and differentiated by the level of functional impairment, was defined in the MHSS as follows:

• Respondents were defined as having any mental illness if they were determined to have any of the mental disorders assessed in the SCID, regardless of the level of functional impairment.
  
  
• Respondents were defined as having serious mental illness if they had any of the mental disorders assessed in the SCID, and these disorders resulted in substantial impairment in carrying out major life activities, based on GAF scores of 50 or below.

The SCID and the GAF in combination were considered to be the gold standard for measuring mental illness.

Statistical modeling involved developing separate weighted logistic regression prediction models for the K6 and for each of the two impairment scales. With serious mental illness status based on having a SCID diagnosis plus a GAF less than or equal to 50, the response variable Y was defined so that

Y = 1 when a serious mental illness diagnosis is positive; otherwise, Y = 0.

If X is a vector of explanatory variables, then the response probability can be estimated using weighted logistic regression models for the WHODAS and SDS half samples. The final 2008 WHODAS and SDS calibration models, respectively, were determined as follows:

     D     (1)

,     D     (2)

where refers to an estimate of the serious mental illness response probability for the WHODAS and SDS models (indicated by the "w" subscript for the WHODAS and the "s" subscript for the SDS). The , , and terms refer to the alternative K6, WHODAS, and SDS scores:¹³

• = Alternative Past Year K6 Score: Past year K6 score less than 8 recoded as 0; past year K6 score 8 to 24 recoded as 1 to 17.
  
  
• = Alternative WHODAS Score: WHODAS item scores less than 2 recoded as 0; WHODAS item scores 2 to 3 recoded as 1, then summed for a score ranging from 0 to 8.
  
  
• = Alternative SDS Score: SDS item scores less than 7 recoded as 0; SDS item scores 7 to 10 recoded as 1, then summed for a score ranging from 0 to 4.

Rearranging terms of the two models provided a direct calculation of the predicted probability of serious mental illness:

,     D

.     D

Next, a cut point probability was determined, so that if for a particular respondent, then he or she was predicted to be serious mental illness positive; otherwise, he or she was predicted to be serious mental illness negative. Receiver operating characteristic (ROC) analyses were used to determine the cut point that resulted in the weighted number of false-positive and false-negative counts being (approximately) equal, thus ensuring unbiased estimates. The optimal cut points were determined to be 0.26972 and 0.26657 for the WHODAS and SDS models, respectively. See Aldworth et al. (2009) for further details.

Model fit statistics and various sensitivity analyses indicated that in combination with the K6, the WHODAS was a better predictor of serious mental illness than the SDS. Consequently, the decision was made to continue with the WHODAS as the measure of impairment for all adults in future NSDUHs. Nevertheless, for the final models, serious mental illness estimates based on the SDS in the 2008 full dataset were very similar to those based on the WHODAS, indicating that the estimates from the two half samples could be combined to form single estimates.

The 2008 prediction model parameters and cut points estimated using the 2008 WHODAS subsample were used to estimate serious mental illness in the 2009 NSDUH sample.

A.11.2 Any Mental Illness

Various methods to estimate any mental illness were investigated in the 2008 MHSS. These methods were subject to the constraint that they would have no effect on the serious mental illness estimates produced by the models discussed above. The methods investigated included logistic models based on any mental illness as the response variable, serious mental illness as the response variable, and multilogistic models based on a multilevel mental illness variable from which both serious mental illness and any mental illness could be derived. Analyses suggested that models based on serious mental illness as the response variable provided almost identical results to those of the other models, so this method was chosen to estimate any mental illness.

As noted previously, serious mental illness estimates for 2008 were based on both the WHODAS and SDS half samples because estimates of serious mental illness were comparable between half samples. Because estimates of any mental illness based on the SDS half sample were not comparable with those based on the WHODAS half sample, the decision was made to base estimates of any mental illness for 2008 only on the WHODAS half sample.

Estimates of any mental illness were obtained from the serious mental illness predicted probabilities calculated using the WHODAS model described above. Respondents with a predicted probability of serious mental illness greater than the cut point of 0.02400 were classified as having any mental illness.

A.11.3 Serious Thoughts of Suicide

Responding to a need for national data on the prevalence of suicidal thoughts and behavior, a set of questions was added beginning with the 2008 NSDUH questionnaire. These questions ask all adult respondents aged 18 or older if at any time during the past 12 months they had serious thoughts of suicide (suicidal ideation). State-level estimates of suicidal ideation are included in this report.

A.11.4 Major Depressive Episode (Depression)

According to the DSM-IV, a person is defined as having had major depressive episode in his or her lifetime if he or she has had at least five or more of the following nine symptoms nearly every day in the same 2-week period, where at least one of the symptoms is a depressed mood or loss of interest or pleasure in daily activities (APA, 1994): (1) depressed mood most of the day; (2) markedly diminished interest or pleasure in all or almost all activities most of the day; (3) significant weight loss when not sick or dieting, or weight gain when not pregnant or growing, or decrease or increase in appetite; (4) insomnia or hypersomnia; (5) psychomotor agitation or retardation; (6) fatigue or loss of energy; (7) feelings of worthlessness; (8) diminished ability to think or concentrate or indecisiveness; and (9) recurrent thoughts of death or suicidal ideation. Respondents who have had a major depressive episode in their lifetime are asked if, during the past 12 months, they had a period of depression lasting 2 weeks or longer while also having some of the other symptoms mentioned. Those reporting that they have are defined as having had major depressive episode in the past year and then are asked questions from the SDS to measure the level of functional impairment in major life activities reported to be caused by the major depressive episode in the past 12 months (Leon, Olfson, Portera, Farber, & Sheehan, 1997).

Beginning in 2004, modules related to major depressive episode, derived from DSM-IV (APA, 1994) criteria for major depression, were included in the questionnaire. These questions permit prevalence estimates of major depressive episode to be calculated. Separate modules were administered to adults aged 18 or older and youths aged 12 to 17. The adult questions were adapted from the depression section of the National Comorbidity Survey Replication (NCS-R; Harvard School of Medicine, 2005), and the questions for youths were adapted from the depression section of the National Comorbidity Survey Adolescent (NCS-A; Harvard School of Medicine, 2005). To make the modules developmentally appropriate for youths, there are minor wording differences in a few questions between the adult and youth modules. Revisions to the questions in both modules were made primarily to reduce its length and to modify the NCS questions, which are interviewer-administered, to the audio computer-assisted self-interviewing (ACASI) format used in NSDUH. In addition, some revisions, based on cognitive testing, were made to improve comprehension.

Since 2004, the NSDUH questions that determine major depressive episode have remained unchanged. In the 2008 questionnaire, however, changes were made in other mental health items that precede the major depressive episode questions for adults (K6, suicide, and impairment). Questions also were retained in 2009 for the WHODAS impairment scale, and the questions for the SDS impairment scale were deleted; see Sections B.4.2 and B.4.3 in CBHSQ (2010) for further details about these questionnaire changes. These questionnaire changes in 2008 appear to have affected the reporting on major depressive episode questions among adults.

Because the WHODAS was selected to be used in the 2009 survey, model-based adjustments were applied to major depressive episode estimates from the SDS half sample in 2008 to remove the context effect differential between the two half samples. Additionally, model-based adjustments were made to the 2005, 2006, and 2007 adult major depressive episode estimates to make them comparable with the 2008 and 2009 major depressive episode estimates (for more information on these adjustments, see Aldworth, Kott, Yu, Mosquin, & Barnett-Walker, 2011). Thus, estimates of major depressive episode were produced for this report using the adjusted 2008 major depressive episode variable along with the unadjusted 2009 major depressive episode variable. Separate tables showing State-level 2007-2008 estimates along with 2005-2006 and 2006-2007 estimates, all based on the adjusted major depressive episode variables, are available at http://www.samhsa.gov/data/states.htm. A comparison of the 2007-2008 and 2008-2009 major depressive episode estimates are included in Table C.23. However, note that the major depressive episode estimates shown in this report for adults are not comparable with estimates shown in prior NSDUH State reports. For further discussion, see Sections B.4.4 and B.4.7 of the 2008 NSDUH national findings report (OAS, 2009).

In addition, changes to the youth mental health service utilization module questions in 2009 that preceded the questions about adolescent depression could have affected adolescents' responses to the adolescent depression questions and estimates of adolescent major depressive episode. However, these changes in 2009 did not appear to affect the estimates of adolescent major depressive episode. Therefore, data on trends in past year major depressive episode from 2004 to 2009 are available for adolescents aged 12 to 17.

A.12 Method for Determining Differences between Two State Estimates for 2008-2009

This section describes a method for determining whether differences between two 2008-2009 State estimates are statistically significant. This procedure can be used for any two State estimates representing the same age group (e.g., young adults aged 18 to 25) and time period (e.g., 2008-2009).

Let and denote the 2008-2009 age group-a specific prevalence rates for two different States, s1 and s2, respectively. The null hypothesis of no difference, that is, , is equivalent to the log-odds ratio equal to zero, that is, , where is defined as , where ln denotes the natural logarithm. An estimate of is given by , where and are the 2008-2009 State estimates given in Appendix B. To compute the variance of that is, let and ,

then , where denotes the covariance between and . This covariance is defined in

terms of the associated correlation as follows:

.     D

The quantities and can be obtained by using the 95 percent Bayesian confidence intervals given in Appendix B. For this purpose, let and denote the 95 percent Bayesian confidence intervals for the two States, s1 and s2, respectively. Then

,     D

where .     D

For all practical purposes, the correlation between and is assumed to be negligible; hence, can be approximated by The correlation is assumed to be negligible because each State was a stratum in the first level of stratification; therefore, each State sample is selected independently. However, the correlation between the two State estimates is theoretically nonzero because State estimates share common fixed-effect parameters in the SAE models. Hence, the test statistic (defined below) might result in a different conclusion in a few cases when the correlation between the State estimates is incorporated in calculating To calculate the p value for testing the null hypothesis of no difference (), it is assumed that the posterior distribution of is normal with . With the null value of , the Bayes p value or posterior probability of no difference is , where Z is a standard normal random variate, , and denotes the absolute value of .

When comparing prevalence rates for two States, it is tempting and often convenient to look at their 95 percent Bayesian confidence intervals to decide whether the difference in the State prevalence rates is significant. If the two Bayesian confidence intervals overlap, one would conclude that the difference is not statistically significant. If the two Bayesian confidence intervals do not overlap, it implies that the State prevalence rates are significantly different from each other. However, the type-I error for the overlapping 95 percent Bayesian confidence intervals test is 0.6 percent (assuming that the two State estimates are uncorrelated and have the same variances) as compared with the 5 percent type-I error of the test based on the statistics defined above (Payton, Greenstone, & Schenker, 2003). Thus, using the overlap method with 95 percent Bayesian confidence intervals implies a type-I error that is much less than the 5 percent level that is typically prescribed for such tests.

As discussed in Schenker and Gentleman (2001), the method of overlapping Bayesian confidence intervals is more conservative (i.e., it rejects the null hypothesis of no difference less often) than the standard method based on statistics when the null hypothesis is true. Even if Bayesian confidence intervals for two States overlap, the two prevalence rates may be declared significantly different by the test based on statistics. Hence, the method of overlapping Bayesian confidence intervals is not recommended to test the equivalence of two State prevalence rates. A detailed description of the method of overlapping confidence intervals and its comparison with the standard methods for testing of a hypothesis is given in Schenker and Gentleman (2001) and Payton et al. (2003).

Example. The prevalence rates for past month alcohol use among 12 to 17 year olds in Arizona and North Dakota are shown in the exhibit below and also in Table B.9 in Appendix B. Looking at the two 95 percent Bayesian confidence intervals, it would appear that the Arizona and North Dakota prevalence rates for past month alcohol use are not statistically different at the 5 percent level of significance because the two Bayesian confidence intervals overlap:

State	Point Estimate (%)	95% Bayesian Confidence Interval (%)
Arizona	14.72	(12.35, 17.45)
North Dakota	18.91	(16.31, 21.81)

However, in the following example, the test based on the statistic described earlier concludes that they are significantly different at the 5 percent level of significance.

Let and Then,

     D

     D

     D

     D

Because the computed absolute value of is greater than or equal to 1.96 (the critical value of the statistic), then at the 5 percent level of significance, the hypothesis of no difference (Arizona prevalence rate = North Dakota prevalence rate) is rejected. Thus, the two State prevalence rates are statistically different. The Bayes p value or posterior probability of no difference is p value = .

A.13 Measuring Change in State Estimates

A.13.1 Change between 2007-2008 and 2008-2009 Small Area Estimates

Comparisons between State small area estimates displayed in Appendix C are based on the 2007 through 2009 NSDUHs. The State estimates for 2007-2008 are the previously published model-based small area estimates (Hughes et al., 2010). The State estimates for 2008-2009 are the small area estimates given in Appendix B. The moving average State prevalence estimates for the overlapping 2007-2008 and 2008-2009 time periods were obtained from independent applications of SWHB methodology; that is, the 2008-2009 models were fit independently of the previously fitted 2007-2008 models. This independent analysis approach was followed because there was no desire to revise the previously published 2007-2008 estimates. Moreover, the same fixed predictor variables were used in the 2007-2008 and 2008-2009 models, but annual updates were made when more current versions became available (see Section A.3 for details). The age group-specific fixed predictor variables were defined at five levels (namely, person-level, census block group-level, tract-level, county-level, and State-level). Also, each age group model had 51 State-level random effects and 300 "within-State" area-level random effects.

To estimate change in State estimates, let and denote 2007-2008 and 2008-2009 prevalence rates, respectively, for State-s and age group-a. The change between and is defined in terms of the log-odds ratio () as opposed to the simple difference because the posterior distribution of the is closer to Gaussian than the posterior distribution of the simple difference (). The is defined as

,     D

where ln denotes the natural logarithm. The p value given in the Appendix C tables is computed to test the null hypothesis of no change (i.e., or equivalently ) An estimate of is given by

,     D

where the are previously published 2007-2008 State estimates and the are the 2008-2009 State estimates presented in this report (see Appendix B). To compute the variance of that is, let and , then

,     D

where denotes the covariance between and . This covariance is defined in terms of the associated correlation as follows:

.     D

Note that and used here to calculate are the same variances used in calculating the previously published 2007-2008 Bayesian confidence intervals and the 2008-2009 Bayesian confidence intervals given in this report, respectively.

The correlation between and was obtained by simultaneously modeling the 2007, 2008, and 2009 NSDUH data. This simultaneous modeling approach was adopted based on the results of the validation study (see Appendix E, Section E.2, of Wright, 2003b) conducted for measuring change in the 1999-2000 and 2000-2001 State estimates. For this simultaneous model, 4 age groups (12 to 17, 18 to 25, 26 to 34, and 35 or older) by 3 years (2007, 2008, and 2009), that is, 12 subpopulation-specific models, were fitted, each with its own set of fixed and random effects. In this case, the general covariance matrices for the State and within-State random effects were 12 × 12 matrices corresponding to the 12 element (age group × year) vectors of random effects. Note that the survey-weighted, Bernoulli-type log likelihood employed in the SWHB methodology was appropriate for this simultaneous model because the 12 age group × year subpopulations were nonoverlapping. The correlation [, ] was approximated by the correlation calculated using the posterior distributions of and from the simultaneous model.

To calculate the p value for testing the null hypothesis of no difference , it is assumed that the posterior distribution of is normal with and . With the null value of , the Bayes p value or posterior probability of no difference is p value = , where Z is a standard normal random variate, , and denotes the absolute value of .

A.13.2 Change between 2002-2003 and 2008-2009 Small Area Estimates

The Bayes p values or posterior probabilities of no difference in prevalence rates for two nonoverlapping periods, 2002-2003 and 2008-2009, were calculated in a very similar manner to the method described in Section A.13.1. Borrowing from the notation above, let refer to the previously published 2002-2003 State estimates (Wright & Sathe, 2005), and let denote the 2008-2009 State estimates presented in this report (see Appendix B). The change between the two prevalence rates is defined in terms of the log-odds ratio as discussed in the prior section. The p value given in the Appendix D tables is computed to test the null hypothesis of no change, that is, to test = 0. An estimate of is given by

,     D

To compute the variance of that is, let and , then

,     D

where denotes the covariance between and This covariance is defined in terms of the associated correlation as follows:

.     D

Note that and used here to calculate are the same posterior variances used in calculating the previously published 2002-2003 Bayesian confidence intervals and the 2008-2009 Bayesian confidence intervals given in this report, respectively.

The difference in the method discussed in Section A.13.1 and the method discussed here is in the model that was fit to find the correlation between and . Here, the correlation between and was obtained by simultaneously modeling the pooled 2002-2003 and pooled 2008-2009 NSDUH data. For this simultaneous model, four age groups (12 to 17, 18 to 25, 26 to 34, and 35 or older) by two time periods (2002-2003 and 2008-2009), that is, eight subpopulation-specific models, were fitted, each with its own set of fixed and random effects. In this case, the general covariance matrices for the State and substate random effects were 8 × 8 matrices corresponding to the eight element (age group × time period) vectors of random effects.

The Bayes p value or posterior probability of no difference was calculated in a manner similar to that described in Section A.13.1.