2012-2013
National Survey on Drug Use and Health:
Guide to State Tables and Summary of Small
Area Estimation Methodology

Section A: Overview of NSDUH and Model-Based State Estimates

A.1 Introduction

This document provides information on the model-based small area estimates of substance use and mental disorders in States based on data from the combined 2012-2013 National Surveys on Drug Use and Health (NSDUHs). These estimates are available online along with other related information.¹ An annual survey of the civilian, noninstitutionalized population aged 12 or older, NSDUH is sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA). It collects information from individuals residing in households, noninstitutionalized group quarters (e.g., shelters, rooming houses, dormitories), and civilians living on military bases.

In 2012-2013, NSDUH collected data from 136,147 respondents aged 12 or older and was designed to obtain representative samples from the 50 States and the District of Columbia.² The survey is planned and managed by SAMHSA's Center for Behavioral Health Statistics and Quality (CBHSQ). Data collection and analysis were conducted under contract with RTI International.³ Nationally in 2012-2013, 314,198 addresses were screened, and 136,147 individuals responded within the screened addresses (see Table C.9). The survey is conducted from January through December each year. The screening response rate (SRR) for 2012-2013 combined averaged 85.0 percent, and the interview response rate (IRR) averaged 72.4 percent, for an overall response rate (ORR) of 61.5 percent (Table C.9). The ORRs for 2012-2013 ranged from 45.8 percent in New York to 75.2 percent in Utah. Estimates have been adjusted to reflect the probability of selection, unit nonresponse, poststratification to known census population estimates, item imputation, and other aspects of the estimation process. These procedures are described in the 2011, 2012, and 2013 NSDUH's methodological resource books (MRBs) (CBHSQ, 2013, 2014, in press). For additional details on NSDUH's methodology, see Section A.2 of the 2011-2012 State small area estimation (SAE) methodology document.⁴

Section A.2 of this document lists all of the tables and files associated with the 2012-2013 State small area estimates and when and where they can be found. Information is given in Section A.3 on the confidence intervals and margins of error and how to make interpretations with respect to the small area estimates. Section A.4 discusses related substance use measures and warns users about not drawing conclusions by subtracting small area estimates from two different measures.

The survey-weighted hierarchical Bayes (SWHB) estimation methodology used in the production of State estimates from the 1999 to 2012 surveys also was used in the production of the 2012-2013 State estimates. The SWHB methodology is described in Appendix E of the 2001 State report (Wright, 2003b) and in Folsom, Shah, and Vaish (1999). The goals of SAE modeling and the implementation of SAE modeling remain the same and are described in Appendix E of the 2001 State report (Wright, 2003b). A general model description is given in Section B.1 of this document. A list of measures for which small area estimates are produced is given in Section B.2. Predictors used in the 2012-2013 SAE modeling are listed and described in Section B.3.

Small area estimates obtained using the SWHB methodology are design consistent (i.e., the small area estimates for States with large sample sizes are close to the robust design-based estimates). The State small area estimates when aggregated using the appropriate population totals result in national small area estimates that are very close to the national design-based estimates. However, to ensure internal consistency, it is desirable to have national small area estimates⁵ exactly match the national design-based estimates. The benchmarked State-level estimates are also potentially less biased than the unbenchmarked State-level estimates. Beginning in 2002, exact benchmarking was introduced, as described in Section B.4.⁶ Tables of the estimated numbers of individuals associated with each measure are available online,⁷ and an explanation of how these counts and their respective Bayesian confidence intervals⁸ are calculated can be found in Section B.5. Section B.6 discusses the method to compare the estimates of a particular measure between two States. For all measures except major depressive episode (MDE, i.e., depression), serious mental illness (SMI), any mental illness (AMI), and past year serious thoughts of suicide, the age groups for which estimates are provided are 12 to 17, 18 to 25, 26 or older, 18 or older, and 12 or older.⁹ Estimates of underage (aged 12 to 20) alcohol use and binge alcohol use were also produced. Alcohol consumption is expected to differ significantly across the 18 to 25 age group because of the legalization of alcohol at age 21. Therefore, it was decided that it would be useful to produce small area estimates for individuals aged 12 to 20.

In Section C, the 2011, 2012, 2013, pooled 2011-2012, and pooled 2012-2013 survey sample sizes, population estimates, and response rates are included in Tables C.1 to C.14, respectively. Table C.15 lists all of the measures and the years for which small area estimates were produced going back to the 2002 NSDUH, and Table C.16 lists all of the measures by age groups for which small area estimates were produced. In addition, Table C.17 provides a summary of milestones implemented in the SAE production process from 2002 to 2013.

A.2 Presentation of Data

In addition to this methodology document, the following files are also available at https://www.samhsa.gov/data/population-data-nsduh/reports?tab=33:

• 2012-2013 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 26, by Age Group): Tables of percentages and associated 95 percent Bayesian confidence intervals are included for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) use of alcohol and underage binge alcohol use. These tables are available in Excel and PDF format. In order to increase the precision of small area estimates and rankings especially for small sample States and to detect year-to-year changes more efficiently, an SAE expert panel¹⁰ recommended producing annual State estimates based on 2 consecutive years of pooled NSDUH data and to base comparisons of estimates (to measure change) on 2-year moving averages.
• 2012-2013 NSDUH National Maps of Prevalence Estimates, by State (Figures 1a to 26d): The color of each State on the U.S. maps indicates how the State ranks relative to other States for each measure. States could fall into one of five groups according to their ranking by quintiles. Because there are 51 areas to be ranked for each measure, the middle quintile was assigned 11 areas and the remaining groups 10 each. In some cases, a "quintile" could have more or fewer States than desired because two (or more) States have the same estimate (to two decimal places). When such ties occurred at the "boundary" between two quintiles, all States with the same estimate were assigned to the lower quintile. Those States with the highest estimates for a given measure are in red, with the exception of the perceptions of risk measures, for which the lowest perceptions of great risk are in red. Those States with the lowest estimates are in white, with the exception of the perceptions of risk measures, for which the highest perceptions of great risk are in white.
  
  Note that because the average annual incidence of marijuana was so low for the 26 or older age group and had such an abbreviated range, no U.S. map has been included for it.
• 2012-2013 NSDUH State Estimates Categorized into Five Groups, by Age Group: This Excel table shows the ranges of percentages for each outcome categorized into five groups (used to form the U.S. maps described above) from lowest to highest estimate for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older.
• 2012-2013 NSDUHs: Model-Based Estimated Totals (in Thousands) (50 States and the District of Columbia) (Tables 1 to 26): Tables showing estimated numbers (counts in thousands) and confidence intervals are included for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) alcohol use and underage binge alcohol use. These tables are available in Excel and PDF format.
• 2012-2013 NSDUH State-Specific Tables (Tables 1 to 112): Tables are provided for each individual State and the District of Columbia, as well as for the total United States and the four census regions (i.e., Northeast, Midwest, South, and West). The tables (two per area) show the percentages and the numbers of individuals (counts in thousands).
• NSDUH: Comparison of 2011-2012 and 2012-2013 Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 26): Tables are presented that show the 2011-2012 (previously published data) and 2012-2013 NSDUH State estimates and an indication of the statistical significance of the difference or change (p value). Estimates are shown for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) alcohol use and underage binge alcohol use. Because annual State-level estimates are based on 2 years of pooled NSDUH data, two consecutive sets of estimates have a 1-year overlap (e.g., 2011-2012 and 2012-2013). If the population totals across the 3 years (e.g., 2011, 2012, and 2013) were the same, then the null hypothesis of no difference between the log odds of the 2011-2012 and 2012-2013 prevalence rates would be equivalent to testing the null hypothesis that the difference between the 2011-2012 and 2012-2013 prevalence rates is zero, which in turn would be equivalent to testing that the difference between the 2011 and 2013 prevalence rates is zero.
• NSDUH: Comparison of 2002-2003 and 2012-2013 Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 22): Tables are presented that show the 2002-2003 and 2012-2013 NSDUH State estimates and an indication of the statistical significance of the difference or change (p value). Estimates are shown for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) alcohol use and underage binge alcohol use. This comparison is done between the most recent estimates (in this case, 2012-2013) and the earliest comparable estimates (based on 2002-2003 NSDUH data).
• 2012-2013 NSDUH: Other Sources of State-Level Data: This document compares two outcomes (cigarette and alcohol use) from NSDUH with data from the Behavioral Risk Factor Surveillance System (BRFSS).

A.3 Confidence Intervals and Margins of Error

At the top of each of the 26 State model-based estimate tables¹¹ is the design-based national estimate along with a 95 percent design-based confidence interval, all of which are based on survey weights and the reported data. The State and regional estimates are model-based statistics (using SAE methodology) that have been adjusted such that the population-weighted mean of the estimates across the 50 States and the District of Columbia equals the design-based national estimate. For more details on this benchmarking, see Section B.4. Associated with each State and regional estimate is a 95 percent Bayesian confidence interval. These intervals indicate the uncertainty in the estimate due to both sampling variability and model bias. For example, the State with the highest estimate of past month use of marijuana for young adults aged 18 to 25 was Rhode Island, with an estimate of 29.8 percent and a 95 percent confidence interval that ranged from 26.3 to 33.5 percent (Table 3 of the State model-based estimates' tables). Therefore, the probability is 0.95 that the true percentage of past month marijuana use in Rhode Island for young adults aged 18 to 25 is between 26.3 and 33.5 percent. As noted earlier in a Section A.1 footnote, the term "prediction interval" (PI) was used in the 2004-2005 NSDUH State report and prior reports to represent uncertainty in the State and regional estimates. However, that term also is used in other applications to estimate future values of a parameter of interest. That interpretation does not apply to NSDUH State model-based estimates, so PI was replaced with "Bayesian confidence interval."

Margin of error is another term used to describe uncertainty in the estimates. For example, if is a 95 percent symmetric confidence interval for the population proportion (p) and is an estimate of p obtained from the survey data, then the margin of error of is given by or . Because is a symmetric confidence interval, will be the same as . In this case, the probability is 0.95 that the true population value (p) is within ± or ± of the survey estimate (). The margin of error defined above will vary for each estimate and will be affected not only by the sample size (e.g., the larger the sample, the smaller the margin of error), but also by the sample design (e.g., telephone surveys using random digit dialing and surveys employing a stratified multistage cluster design will, more than likely, produce a different margin of error) (Scheuren, 2004).

The confidence intervals shown in NSDUH reports are asymmetric, meaning that the distance between the estimate and the lower confidence limit will not be the same as the distance between the upper confidence limit and the estimate. For example, Utah's past month marijuana use estimate of 10.9 percent for adults aged 18 to 25 years with a 95 percent confidence interval equal to (8.7, 13.5) (see Table 3 of the State model-based estimates' tables). Therefore Utah's estimate is 2.2 (i.e., 10.9 − 8.7) percentage points from the lower 95 percent confidence limit and 2.6 (i.e., 13.5 − 10.9) percentage points from the upper limit. These asymmetric confidence intervals work well for small percentages often found in NSDUH tables and reports while still being appropriate for larger percentages. Some surveys or polls provide only one margin of error for all reported percentages. This single number is usually calculated by setting the sample percentage estimate () equal to 50 percent, which will produce an upper bound or maximum margin of error. Such an approach would not be feasible in NSDUH because the estimates vary from less than 1 percent to over 75 percent; hence, applying a single margin of error to these estimates could significantly overstate or understate the actual precision levels. Therefore, given the differences mentioned above, it is more useful and informative to report the confidence interval for each estimate instead of a margin of error.

When it is indicated that a State has the highest or lowest estimate, it does not imply that the State's estimate is significantly higher or lower than the next highest or lowest State. When comparing two State estimates, two overlapping 95 percent confidence intervals do not imply that their State estimates are statistically equivalent at the 5 percent level of significance. For details on a more accurate test to compare State estimates, see Section B.6.

A.4 Related Substance Use Measures

Small area estimates are produced for a number of related drug measures, such as marijuana use and illicit drug use. It might appear that one could draw conclusions by subtracting one from the other (e.g., subtracting the percentage who used illicit drugs other than marijuana in the past month from the percentage who used illicit drugs in the past month to find the percentage who only used marijuana in the past month). Because related measures have been estimated with different models (i.e., separate models by age group and outcome), subtracting one measure from another related measure at the State or census region level can give misleading results, perhaps even a "negative" estimate, and should be avoided. However, these comparisons can be made at the national level because these estimates are design-based estimates. For example, at the national level, subtracting cigarette use estimates from tobacco use estimates will give the estimate of individuals who did not use cigarettes, but used other forms of tobacco.

Section B: State Model-Based Estimation Methodology

B.1 General Model Description

The model can be characterized as a complex mixed¹² model (including both fixed and random effects) of the following form:

,     D

where is the probability of engaging in the behavior of interest (e.g., using marijuana in the past month) for person-k belonging to age group-a in grouped State sampling region (SSR)-j of State-i.¹³ Let denote a vector of auxiliary (predictor) variables associated with age group-a (12 to 17, 18 to 25, 26 to 34, and 35 or older) and denote the associated vector of regression parameters. The age group-specific vectors of auxiliary variables are defined for every block group in the Nation and also include person-level demographic variables, such as race/ethnicity and gender. The vectors of State-level random effects and grouped SSR-level random effects are assumed to be mutually independent with and , where is the total number of individual age groups modeled (generally, ). For hierarchical Bayes (HB) estimation purposes, an improper uniform prior distribution is assumed for , and proper Wishart prior distributions are assumed for and . The HB solution for involves a series of complex Markov Chain Monte Carlo (MCMC) steps to generate values of the desired fixed and random effects from the underlying joint posterior distribution. The basic process is described in Folsom et al. (1999), Shah, Barnwell, Folsom, and Vaish (2000), and Wright (2003a, 2003b).

Once the required number of MCMC samples (1,250 in all) for the parameters of interest are generated and tested for convergence properties (see Raftery & Lewis, 1992), the small area estimates for each age group × race/ethnicity × gender cell within a block group can be obtained. These block group-level small area estimates then can be aggregated using the appropriate population count projections to form State-level small area estimates for the desired age group(s). These State-level small area estimates are benchmarked to the national design-based estimates as described in Section B.4.

B.2 Variables Modeled

The 2013 NSDUH data were pooled with the 2012 NSDUH data, and age group-specific State estimates for 25 binary (0, 1) measures were produced for the following outcomes:

1. past month use of illicit drugs,
2. past year use of marijuana,
3. past month use of marijuana,
4. perception of great risk of smoking marijuana once a month,
5. average annual rate of first use of marijuana,¹⁴
6. past month use of illicit drugs other than marijuana,
7. past year use of cocaine,
8. past year nonmedical use of pain relievers,
9. past month use of alcohol,
10. past month binge alcohol use,
11. perception of great risk of having five or more drinks of an alcoholic beverage once or twice a week,
12. past month use of tobacco products,
13. past month use of cigarettes,
14. perception of great risk of smoking one or more packs of cigarettes per day,
15. past year alcohol dependence or abuse,
16. past year alcohol dependence,
17. past year illicit drug dependence or abuse,
18. past year illicit drug dependence,
19. past year dependence or abuse of illicit drugs or alcohol,
20. needing but not receiving treatment for illicit drug use in the past year,
21. needing but not receiving treatment for alcohol use in the past year,
22. serious mental illness (SMI) in the past year,
23. any mental illness (AMI) in the past year,
24. serious thoughts of suicide in the past year, and
25. past year major depressive episode (MDE, i.e., depression).

Estimates of underage (aged 12 to 20) alcohol use and binge alcohol use were also produced. Comparisons between the 2011-2012 and the 2012-2013 State estimates were produced for all of these measures as well. For details on how measures such as AMI, SMI, MDE, illicit drugs, dependence or abuse, and average annual rate of first use of marijuana are defined, see "2011-2012 National Surveys on Drug Use and Health: Guide to State Tables and Summary of Small Area Estimation Methodology" at https://www.samhsa.gov/data/population-data-nsduh/reports?tab=33.

Illicit drug use includes the use of methamphetamines. NSDUH questions on methamphetamine use are asked in the stimulant module of the core section of the questionnaire in the context of questions about the nonmedical use of prescription stimulants. Beginning in 2005, new questions were added to the special drug module in the noncore section of the NSDUH questionnaire to capture information from respondents who may have used methamphetamines but did not recognize it as a prescription drug and therefore did not report use in the core stimulant module. However, the illicit drug estimates (including dependence, abuse, and treatment of illicit drugs) shown in the 2012-2013 SAE documents include data from the original methamphetamine questions, but they do not include the new methamphetamine items added in 2005 and 2006 (i.e., the responses to the noncore questions). For more information on these new methamphetamine questions, see the findings from the methamphetamine analysis section of the 2005 NSDUH Methodological Resource Book (Center for Behavioral Health Statistics and Quality [CBHSQ], 2007).

B.3 Predictors Used in Mixed Logistic Regression Models

Local area data used as potential predictor variables in the mixed logistic regression models were obtained from several sources, including Claritas Inc., the U.S. Census Bureau, the Federal Bureau of Investigation (FBI) (Uniform Crime Reports [UCRs]), the Bureau of Labor Statistics, the Bureau of Economic Analysis, the Substance Abuse and Mental Health Services Administration (SAMHSA) (National Survey of Substance Abuse Treatment Services [N–SSATS]), and the National Center for Health Statistics (mortality data). Note that the predictors used to produce the 2012-2013 State small area estimates are the same as the predictors used to produce the 2011-2012 State small area estimates (however, values of the data were updated when possible). That is, no new variable selection was done for 2012-2013.

Sources and potential data items used in the modeling are provided in the following text and lists.

• Nielsen Claritas. This demographic data package contains data for 2013 with projections to 2018. The population projections are used to update these predictor variables each year. The 2012 and 2013 population estimates (the 2012 estimates were obtained by projecting back the data based on the provided 2013 population and 2018 population counts) were used for producing the 2012-2013 State estimates. For more information on these data, see Section B.4.
• U.S. Census Bureau. The 2010 census (demographic and geographic variables) and 2011 food stamp participation estimates were used (http://www.census.gov/did/www/saipe/inputdata/cntysnap.xls). The Census Bureau's small area income and poverty estimates division obtains Food Stamp Program (now known as the Supplemental Nutrition Assistance Program [SNAP]) participation estimates from the U.S. Department of Agriculture, Food and Nutrition Service.
• American Community Survey (ACS). The 2008-2012 5-year ACS demographic and socioeconomic variables at the tract level and poverty variable at the county level were used (http://www.census.gov/acs/www/data_documentation/data_main/).
• Federal Bureau of Investigation. Uniform Crime Report (UCR) arrest totals were obtained from http://www.icpsr.umich.edu/icpsrweb/NACJD/archive.jsp. The most current data used are from 2011 for most counties, with previous years' data substituted in a few cases.
• Bureau of Labor Statistics (BLS). The 2013 county-level unemployment estimates were used (http://www.bls.gov/lau/tables.htm). The BLS uses results from the Current Population Survey (CPS) to provide county-level unemployment estimates. The CPS is a monthly survey of households conducted by the Census Bureau for the BLS.
• Bureau of Economic Analysis (BEA). The 2012 county-level per capita income estimates were used (http://bea.gov/iTable/index.cfm). These county-level per capita income estimates are produced by the Regional Income Division of the BEA.
• National Center for Health Statistics (NCHS). Mortality data using International Classification of Diseases, 10th revision (ICD-10), 2005-2010, were used. The ICD-10 death data are from the NCHS at the Centers for Disease Control and Prevention.
• SAMHSA, Center for Behavioral Health Statistics and Quality (CBHSQ, formerly the Office of Applied Studies [OAS]). Data were used from the National Survey of Substance Abuse Treatment Services (N-SSATS), formerly known as the Uniform Facility Data Set (UFDS). The 2012-2013 data on drug and alcohol treatment estimates were obtained. Maintenance of effort expenditures, block grant awards, cost of services, and total taxable resources data were also used.

The following lists provide the specific independent variables that were potential predictors in the models.

B.4 Benchmarking the Age Group-Specific Small Area Estimates

The self-calibration built into the survey-weighted hierarchical Bayes (SWHB) solution ensures that the population-weighted average of the State small area estimates will closely match the national design-based estimates. The national design-based estimates in NSDUH are based entirely on survey-weighted data using a direct estimation approach, whereas the State and census region estimates are model-based. Given the self-calibration ensured by the SWHB solution, for State reports prior to 2002, the standard Bayes prescription was followed; specifically, the posterior mean was used for the point estimate, and the tail percentiles of the posterior distribution were used for the Bayesian confidence interval limits.

Singh and Folsom (2001) extended Ghosh's (1992) results on constrained Bayes estimation to include exact benchmarking to design-based national estimates. In the simplest version of this constrained Bayes solution where only the design-based mean is imposed as a benchmarking constraint, each of the 2012-2013 State-by-age group small area estimates is adjusted by adding the common factor where is the design-based national estimate and is the population-weighted mean of the State small area estimates for age group-a. The exactly benchmarked State-s and age group-a small area estimates then are given by . Experience with such additive adjustments suggests that the resulting exactly benchmarked State small area estimates will always be between 0 percent and 100 percent because the SWHB self-calibration ensures that the adjustment factor is small relative to the size of the State-level small area estimates.

Relative to the Bayes posterior mean, these benchmark-constrained State small area estimates are biased by the common additive adjustment factor. Therefore, the posterior mean squared error for each benchmarked State small area estimate has the square of this adjustment factor added to its posterior variance. To achieve the desirable feature of exact benchmarking, this constrained Bayes adjustment factor was implemented for the State-by-age group small area estimates. The associated Bayesian confidence (credible) intervals can be re-centered at the benchmarked small area estimates on the logit scale with the symmetric interval end points based on the posterior root mean squared errors. The adjusted 95 percent Bayesian confidence intervals are defined below:

 ,     D

where

 ,     D

 ,     D     and

 .     D

The associated posterior coverage probabilities for these benchmarked intervals are very close to the prescribed 0.95 value because the State small area estimates have posterior distributions that can be approximated exceptionally well by a Gaussian distribution.

B.5 Calculation of Estimated Number of Individuals Associated with Each Outcome

Tables 1 to 26 of "2012-2013 NSDUHs: Model-Based Estimated Totals (in Thousands) (50 States and the District of Columbia)" show the estimated numbers of individuals associated with each of the 25 outcomes of interest.¹⁵ To calculate these numbers, the benchmarked small area estimates and the associated 95 percent Bayesian confidence intervals are multiplied by the average population across the 2 years (in this case, 2012 and 2013) of the State by the age group of interest.

For example, past month use of alcohol among 18 to 25 year olds in Alabama was 50.89 percent.¹⁶ The corresponding Bayesian confidence intervals ranged from 47.07 to 54.71 percent. The population count for 18 to 25 year olds averaged across 2012 and 2013 in Alabama was 536,933 (see Table C.10 in Section C of this methodology document). Hence, the estimated number of 18 to 25 year olds using alcohol in the past month in Alabama was 0.5089 × 536,933, which is 273,245.¹⁷ The associated Bayesian confidence intervals ranged from 0.4707 × 536,933 (i.e., 252,734) to 0.5471 × 536,933 (i.e., 293,756). Note that when estimates of the number of individuals are calculated for Tables 1 to 26 in "2012-2013 NSDUHs: Model-Based Estimated Totals (in Thousands) (50 States and the District of Columbia)" (follow the link in footnote 17), the unrounded percentages and population counts are used, then the numbers are reported to the nearest thousand. Hence, the number obtained by multiplying the published estimate with the published population estimate may not exactly match the counts that are published in these tables because of rounding differences.

The only exception to this calculation is the production of the estimated numbers of marijuana initiates. Those estimates cannot be directly calculated as the product of the percentage estimate of first use of marijuana and the population counts available in Section C. That is because the denominator of that percentage estimate is defined as the number of person years at risk for marijuana initiation, which is a combination of individuals who never used marijuana and one half of the individuals who initiated in the past 24 months.

B.6 Comparison of Two 2012-2013 Small Area Estimates

This section describes a method for determining whether differences between two 2012-2013 State estimates are statistically significant. This procedure can be used for any two State estimates representing the same age group (e.g., young adults aged 18 to 25) and time period (e.g., 2012-2013).

Let and denote the 2012-2013 age group-a specific prevalence rates for two different States, and , respectively. The null hypothesis of no difference, that is,  , is equivalent to the log-odds ratio equal to zero, that is, , where is defined as ,

where ln denotes the natural logarithm. An estimate of is given by ,

where and are the 2012-2013 State estimates given in the "2012-2013 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 26, by Age Group)" (follow the link in footnote 16). To compute the variance of , that is, , let and ,

then , where denotes the covariance between and . This covariance is defined in terms of the associated correlation as follows:

.     D

The quantities and can be obtained by using the 95 percent Bayesian confidence intervals given in the "2012-2013 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 26, by Age Group)" (follow the link in footnote 16). For this purpose, let and denote the 95 percent Bayesian confidence intervals for the two States, and , respectively. Then

     D

where .

For all practical purposes, the correlation between and is assumed to be negligible; hence, can be approximated by The correlation is assumed to be negligible because each State was a stratum in the first level of stratification; therefore, each State sample is selected independently. However, the correlation between the two State estimates is theoretically nonzero because State estimates share common fixed-effect parameters in the SAE models. Hence, the test statistic (defined below) might result in a different conclusion in a few cases when the correlation between the State estimates is incorporated in calculating . To calculate the p value for testing the null hypothesis of no difference (), it is assumed that the posterior distribution of is normal with and . With the null value of , the Bayes p value or posterior probability of no difference is , where is a standard normal random variate, , and denotes the absolute value of .

Hence, to test whether differences between two 2012-2013 State estimates are statistically significant, the test statistic and the associated p value can be used. If p ≤ 0.05, then the two State estimates can be considered different at the 5 percent level of significance.

When comparing estimates for two States, it is tempting and often convenient to look at their 95 percent Bayesian confidence intervals to decide whether the difference in the State estimates is significant. If the two Bayesian confidence intervals overlap, one would conclude that the difference is not statistically significant. If the two Bayesian confidence intervals do not overlap, it implies that the State estimates are significantly different from each other. However, the type-I error for the overlapping 95 percent Bayesian confidence intervals test is 0.6 percent (assuming that the two State estimates are uncorrelated and have the same variances) as compared with the 5 percent type-I error of the test based on the statistics defined above (Payton, Greenstone, & Schenker, 2003). Thus, using the overlap method with 95 percent Bayesian confidence intervals implies a type-I error that is much less than the 5 percent level that is typically prescribed for such tests.

As discussed in Schenker and Gentleman (2001), the method of overlapping Bayesian confidence intervals is more conservative (i.e., it rejects the null hypothesis of no difference less often) than the standard method based on z statistics when the null hypothesis is true. Even if Bayesian confidence intervals for two States overlap, the two estimates may be declared significantly different by the test based on z statistics. Hence, the method of overlapping Bayesian confidence intervals is not recommended to test the equivalence of two State estimates. A detailed description of the method of overlapping confidence intervals and its comparison with the standard methods for testing of a hypothesis is given in Schenker and Gentleman (2001) and Payton et al. (2003).

Example. The percentages for past month alcohol use among 12 to 17 year olds in New Jersey and Oklahoma are shown in the following exhibit and also in Table 9 of the "2012-2013 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia)" at https://www.samhsa.gov/data/population-data-nsduh/reports?tab=33. Looking at the two 95 percent Bayesian confidence intervals, it would appear that the Oklahoma and New Jersey percentages for past month alcohol use are not statistically different at the 5 percent level of significance because the two Bayesian confidence intervals overlap:

However, in the following example, the test based on the statistic described earlier concludes that they are significantly different at the 5 percent level of significance.

Let , , , , , . Then,

,     D

,     D

,     D

,     D

,     D     and

.     D

Because the computed absolute value of is greater than or equal to 1.96 (the critical value of the statistic), then at the 5 percent level of significance, the hypothesis of no difference (Oklahoma estimate = New Jersey estimate) is rejected. Thus, the two State estimates are statistically different. The Bayes p value or posterior probability of no difference is .

Section C: Sample Sizes, Response Rates, and Population Estimates