2013-2014
National Survey on Drug Use and Health:
Guide to State Tables and Summary of Small
Area Estimation Methodology

Section A: Overview of NSDUH and Model-Based State Estimates

A.1 Introduction

This document provides information on the model-based small area estimates of substance use and mental disorders in states based on data from the combined 2013-2014 National Surveys on Drug Use and Health (NSDUHs). These estimates are available online along with other related information.¹ An annual survey of the civilian, noninstitutionalized population aged 12 or older, NSDUH is sponsored by the Substance Abuse and Mental Health Services Administration (SAMHSA), and it collects information from individuals residing in households, noninstitutionalized group quarters (e.g., shelters, rooming houses, dormitories), and civilians living on military bases. In 2013-2014, NSDUH collected data from 135,739 respondents aged 12 or older and was designed to obtain representative samples from the 50 states and the District of Columbia. NSDUH is planned and managed by SAMHSA's Center for Behavioral Health Statistics and Quality (CBHSQ). Data collection and analysis are conducted under contract with RTI International.² The survey is conducted annually from January through December. A summary of NSDUH's methodology is given in Section A.2. Section A.3 lists all of the tables and files associated with the 2013-2014 state small area estimates and when and where they can be found. Information is given in Section A.4 on the confidence intervals and margins of error and how to make interpretations with respect to the small area estimates. Section A.5 discusses related substance use measures and warns users about not drawing conclusions by subtracting small area estimates from two different measures.

The survey-weighted hierarchical Bayes (SWHB) estimation methodology used in the production of state estimates from the 1999 to 2013 surveys also was used in the production of the 2013-2014 state estimates. The SWHB methodology is described in Appendix E of the 2001 state report (Wright, 2003b) and in Folsom, Shah, and Vaish (1999). The goals and implementation of small area estimation (SAE) modeling remain the same and are described in Appendix E of the 2001 state report (Wright, 2003b). A general model description is given in Section B.1 of this document. A list of measures for which small area estimates are produced is given in Section B.2. Predictors used in the 2013-2014 SAE modeling are listed and described in Section B.3.

Small area estimates obtained using the SWHB methodology are design consistent (i.e., the small area estimates for states with large sample sizes are close to the robust design-based estimates). The state small area estimates when aggregated using the appropriate population totals result in national small area estimates that are very close to the national design-based estimates. However, to ensure internal consistency, it is desirable to have national small area estimates³ exactly match the national design-based estimates. The benchmarked state-level estimates are also potentially less biased than the unbenchmarked state-level estimates. Beginning in 2002, exact benchmarking was introduced, as described in Section B.4.⁴ Tables of the estimated numbers of individuals associated with each measure are available online,⁵ and an explanation of how these counts and their respective Bayesian confidence intervals⁶ are calculated can be found in Section B.5. Section B.6 discusses the method to compare the estimates of a particular measure between two states. For all measures except major depressive episode (MDE, i.e., depression), serious mental illness (SMI), any mental illness (AMI), and past year serious thoughts of suicide, the age groups for which estimates are provided are 12 to 17, 18 to 25, 26 or older, 18 or older, and 12 or older.⁷ Estimates of underage (aged 12 to 20) alcohol use and binge alcohol use were also produced. Alcohol consumption is expected to differ significantly across the 18 to 25 age group because of the legalization of alcohol at age 21. Therefore, it was decided that it would be useful to produce small area estimates for individuals aged 12 to 20.

In Section C, the 2012, 2013, 2014, pooled 2012-2013, and pooled 2013-2014 survey sample sizes, population estimates, and response rates are included in Tables C.1 to C.14, respectively. Table C.15 lists all of the measures and the years for which small area estimates were produced going back to the 2002 NSDUH, and Table C.16 lists all of the measures by age groups for which small area estimates were produced. In addition, Table C.17 provides a summary of milestones implemented in the SAE production process from 2002 to 2014.

A.2 Summary of NSDUH Methodology

This section provides a brief overview of the NSDUH methodology, specifically the sample design. For additional details on NSDUH's methodology, see Section A.2 of the 2011-2012 state SAE methodology document.⁸

The 1999 through 2001 National Household Surveys on Drug Abuse (NHSDAs)⁹ and the 2002 through 2013 NSDUHs employed a 50-state design with an independent, multistage area probability sample for each of the 50 states and the District of Columbia. For the 50-state design, 8 states were designated as large sample states (California, Florida, Illinois, Michigan, New York, Ohio, Pennsylvania, and Texas) with target sample sizes of 3,600 per year. For the remaining 42 states and the District of Columbia, the target sample size was 900 per year. This approach ensured that there was sufficient sample in every state to support SAE while at the same time maintaining efficiency for national estimates. The design also oversampled youths and young adults, so that each state's sample was approximately equally distributed among three major age groups: 12 to 17 years, 18 to 25 years, and 26 years or older.

A coordinated design was developed for the 2014 through 2017 NSDUHs. Similar to the 1999 through 2013 surveys, the coordinated 4-year design is state-based with an independent, multistage area probability sample within each state and the District of Columbia. This design designates 12 states as large sample states. These 12 states have the following target sample sizes per year: 4,560 interviews in California; 3,300 interviews in Florida, New York, and Texas; 2,400 interviews in Illinois, Michigan, Ohio, and Pennsylvania; and 1,500 interviews in Georgia, New Jersey, North Carolina, and Virginia. Making the sample sizes more proportional to the state population sizes improves the precision of national NSDUH estimates. This change also allows for a more cost-efficient sample allocation to the largest states while slightly increasing the sample sizes in smaller states to improve the precision of state estimates (note that the target sample size per year in the small states is 960 interviews with the exception of Hawaii where the target sample size is 967 interviews). The fielded sample sizes for each state in 2014 are provided in Table C.5, and the combined 2013-2014 sample sizes are provided in Table C.9.

Starting in 2014, the allocation of the NSDUH sample is 25 percent for adolescents aged 12 to 17, 25 percent for adults aged 18 to 25, and 50 percent for adults aged 26 or older. The sample of adults aged 26 or older is further divided into three subgroups: aged 26 to 34 (15 percent), aged 35 to 49 (20 percent), and aged 50 or older (15 percent). For more information on the 2014 through the 2017 NSDUH sample design and for differences between the 2013 and 2014 surveys, refer to the 2014 NSDUH sample design report (CBHSQ, 2015b).

Nationally in 2013-2014, 287,930 addresses were screened, and 135,739 individuals responded within the screened addresses (see Table C.9). The screening response rate (SRR) for 2013-2014 combined averaged 82.9 percent, and the interview response rate (IRR) averaged 71.4 percent, for an overall response rate (ORR) of 59.2 percent (Table C.9). The ORRs for 2013-2014 ranged from 44.7 percent in New York to 74.0 percent in Utah. Estimates have been adjusted to reflect the probability of selection, unit nonresponse, poststratification to known census population estimates, item imputation, and other aspects of the estimation process. These procedures are described in detail in the 2012, 2013, and 2014 NSDUHs' methodological resource books (MRBs) (CBHSQ, 2014, 2015a, in press).

A.3 Presentation of Data

In addition to this methodology document for the 2013-2014 state SAE results, the following files are available at https://www.samhsa.gov/data/:

• 2013-2014 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 26, by Age Group): Tables of percentages and associated 95 percent Bayesian confidence intervals are included for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) use of alcohol and underage binge alcohol use. These tables are available in Excel and PDF format. In order to increase the precision of small area estimates and rankings especially for small sample states and to detect year-to-year changes more efficiently, an SAE expert panel¹⁰ recommended producing annual state estimates based on 2 consecutive years of pooled NSDUH data and to base comparisons of estimates (to measure change) on 2-year moving averages.
• 2013-2014 NSDUH National Maps of Prevalence Estimates, by State (Figures 1a to 26d): The color of each state on the U.S. maps indicates how the state ranks relative to other states for each measure. States could fall into one of five groups according to their ranking by quintiles. Because 51 states were ranked for each measure, the middle quintile was assigned to 11 states, and the remaining quintiles were assigned 10 states each. In some cases, a "quintile" could have more or fewer states than desired because two (or more) states had the same estimate (to two decimal places). When such ties occurred at the "boundary" between two quintiles, all of the states with the same estimate were conservatively assigned to the lower quintile. Those states with the highest rates for a given measure are in red, with the exception of the perceptions of risk measures, for which the lowest perceptions of great risk are in red. Those states with the lowest estimates are in white, with the exception of the perceptions of risk measures, for which the highest perceptions of great risk are in white.
  
  Note that because the average annual incidence of marijuana was so low for the 26 or older age group and had such an abbreviated range, no U.S. map was included for it.
• 2013-2014 NSDUH State Estimates Categorized into Five Groups, by Age Group: This Excel table shows the ranges of percentages for each outcome categorized into five groups (used to form the U.S. maps described above) from the lowest to highest estimate for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older. Also included are ranges for underage (12 to 20) alcohol use and underage binge alcohol use.
• 2013-2014 NSDUHs: Model-Based Estimated Totals (in Thousands) (50 States and the District of Columbia) (Tables 1 to 26): Tables showing estimated numbers (counts in thousands) and confidence intervals are included for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) alcohol use and underage binge alcohol use. These tables are available in Excel and PDF format.
• 2013-2014 NSDUH State-Specific Tables (Tables 1 to 112): Tables are provided for each individual state and the District of Columbia, as well as for the total United States and the four census regions (i.e., Northeast, Midwest, South, and West). The tables (two per area) show the percentages and the numbers of individuals (counts in thousands).
• NSDUH: Comparison of 2012-2013 and 2013-2014 Population Percentages (50 States and the District of Columbia) (Tables 1 to 26): Tables are presented that show the 2012-2013 (previously published data) and 2013-2014 NSDUH state estimates and an indication of the statistical significance of the difference or change (p value). Estimates are shown for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, adults aged 18 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) alcohol use and underage binge alcohol use. Because annual state-level estimates are based on 2 years of pooled NSDUH data, two consecutive sets of estimates have a 1-year overlap (e.g., 2012-2013 and 2013-2014). If the population totals across the 3 years (e.g., 2012, 2013, and 2014) were the same, then the null hypothesis of no difference between the log odds of the 2012-2013 and 2013-2014 prevalence rates would be equivalent to testing the null hypothesis that the difference between the 2012-2013 and 2013-2014 prevalence rates is zero, which in turn would be equivalent to testing that the difference between the 2012 and 2014 prevalence rates is zero.
• NSDUH: Comparison of 2002-2003 and 2013-2014 Population Percentages (50 States and the District of Columbia) (Tables 1 to 22): Tables are presented that show the 2002-2003 and 2013-2014 NSDUH state estimates and an indication of the statistical significance of the difference or change (p value). Estimates are shown for youths aged 12 to 17, young adults aged 18 to 25, adults aged 26 or older, and all individuals aged 12 or older. Also included are tables for underage (12 to 20) alcohol use and underage binge alcohol use. This comparison is done between the most recent estimates (in this case, 2013-2014) and the earliest comparable estimates (based on 2002-2003 NSDUH data).
• 2013-2014 NSDUH: Other Sources of State-Level Data: This document compares two outcomes (cigarette and alcohol use) from NSDUH with data from the Behavioral Risk Factor Surveillance System (BRFSS).

A.4 Confidence Intervals and Margins of Error

At the top of each of the 26 state model-based estimate tables¹¹ is the design-based national estimate along with a 95 percent design-based confidence interval, all of which are based on the survey design, the survey weights, and the reported data. The state estimates are model-based statistics (using SAE methodology) that have been adjusted (benchmarked) such that the population-weighted mean of the estimates across the 50 states and the District of Columbia equals the design-based national estimate. For more details on this benchmarking, see Section B.4. The region-level estimates are also benchmarked and are obtained by taking the population-weighted mean of the associated state-level benchmarked estimates. Associated with each state and regional estimate is a 95 percent Bayesian confidence interval. These intervals indicate the uncertainty in the estimate due to both sampling variability and model fit. For example, the state with the highest estimate of past month use of marijuana for young adults aged 18 to 25 was Colorado, with an estimate of 31.2 percent and a 95 percent confidence interval that ranged from 27.7 to 35.1 percent (Table 3 of the state model-based estimates' tables). Assuming that sampling and modeling conditions held, the Bayes posterior probability was 0.95 that the true percentage of past month marijuana use in Colorado for young adults aged 18 to 25 in 2013-2014 was between 27.7 and 35.1 percent. As noted earlier in a Section A.1 footnote, the term "prediction interval" (PI) was used in the 2004-2005 NSDUH state report and prior reports to represent uncertainty in the state and regional estimates. However, that term also is used in other applications to estimate future values of a parameter of interest. That interpretation does not apply to NSDUH state model-based estimates, so PI was replaced with "Bayesian confidence interval."

Margin of error is another term used to describe uncertainty in the estimates. For example, if is a 95 percent symmetric confidence interval for the population proportion (p) and is an estimate of p obtained from the survey data, then the margin of error of is given by or . Because is a symmetric confidence interval, will be the same as . In this case, the probability is 0.95 that the interval ± or ± will contain the true population value (p). The margin of error defined above will vary for each estimate and will be affected not only by the sample size (e.g., the larger the sample, the smaller the margin of error), but also by the sample design (e.g., telephone surveys using random digit dialing and surveys employing a stratified multistage cluster design will, more than likely, produce a different margin of error) (Scheuren, 2004).

The confidence intervals shown in NSDUH reports are asymmetric, meaning that the distance between the estimate and the lower confidence limit will not be the same as the distance between the upper confidence limit and the estimate. For example, Utah's past month marijuana use estimate of 11.6 percent for adults aged 18 to 25 years with a 95 percent confidence interval equal to (9.3, 14.3) (see Table 3 of the state model-based estimates' tables). Therefore Utah's estimate is 2.3 (i.e., 11.6 − 9.3) percentage points from the lower 95 percent confidence limit and 2.7 (i.e., 14.3 − 11.6) percentage points from the upper limit. These asymmetric confidence intervals work well for small percentages often found in NSDUH tables and reports while still being appropriate for larger percentages. Some surveys or polls provide only one margin of error for all reported percentages. This single number is usually calculated by setting the sample percentage estimate () equal to 50 percent, which will produce an upper bound or maximum margin of error. Such an approach would not be feasible in NSDUH because the estimates vary from less than 1 percent to over 75 percent; hence, applying a single margin of error to these estimates could significantly overstate or understate the actual precision levels. Therefore, given the differences mentioned above, it is more useful and informative to report the confidence interval for each estimate instead of a margin of error.

When it is indicated that a state has the highest or lowest estimate, it does not imply that the state's estimate is significantly higher or lower than the next highest or lowest state's estimate. Additionally, two significantly different state estimates (at the 5 percent level of significance) may have overlapping 95 percent confidence intervals. For details on a more accurate test to compare state estimates, see Section B.6.

A.5 Related Substance Use Measures

Small area estimates are produced for a number of related drug measures, such as marijuana use and illicit drug use. It might appear that one could draw conclusions by subtracting one from the other (e.g., subtracting the percentage who used illicit drugs other than marijuana in the past month from the percentage who used illicit drugs in the past month to find the percentage who only used marijuana in the past month). Because related measures have been estimated with different models (i.e., separate models by age group and outcome), subtracting one measure from another related measure at the state or census region level can give misleading results, perhaps even a "negative" estimate, and should be avoided. However, these comparisons can be made at the national level because these estimates are design-based estimates. For example, at the national level, subtracting cigarette use estimates from tobacco use estimates will give the estimate of individuals who did not use cigarettes, but used other forms of tobacco.

Section B: State Model-Based Estimation Methodology

B.1 General Model Description

The model can be characterized as a complex mixed¹² model (including both fixed and random effects) of the following form:

,     D

where is the probability of engaging in the behavior of interest (e.g., using marijuana in the past month) for person-k belonging to age group-a in grouped state sampling region (SSR)-j of state-i.¹³ Let denote a vector of auxiliary (predictor) variables associated with age group-a (12 to 17, 18 to 25, 26 to 34, and 35 or older) and denote the associated vector of the regression parameters. The age group-specific vectors of the auxiliary variables are defined for every block group in the nation and also include person-level demographic variables, such as race/ethnicity and gender. The vectors of state-level random effects and grouped SSR-level random effects are assumed to be mutually independent with and , where is the total number of individual age groups modeled (generally, ). For hierarchical Bayes (HB) estimation purposes, an improper uniform prior distribution is assumed for , and proper Wishart prior distributions are assumed for and . The HB solution for involves a series of complex Markov Chain Monte Carlo (MCMC) steps to generate values of the desired fixed and random effects from the underlying joint posterior distribution. The basic process is described in Folsom et al. (1999), Shah, Barnwell, Folsom, and Vaish (2000), and Wright (2003a, 2003b).

Once the required number of MCMC samples (1,250 in all) for the parameters of interest are generated and tested for convergence properties (see Raftery & Lewis, 1992), the small area estimates for each race/ethnicity × gender cell within a block group can be obtained for each age group. These block group-level small area estimates then can be aggregated using the appropriate population count projections for the desired age group(s) to form state-level small area estimates. These state-level small area estimates are benchmarked to the national design-based estimates as described in Section B.4.

B.2 Variables Modeled

The 2014 NSDUH data were pooled with the 2013 NSDUH data, and age group-specific state estimates for 25 binary (0, 1) measures were produced for the following outcomes:

1. past month use of illicit drugs,
2. past year use of marijuana,
3. past month use of marijuana,
4. perceptions of great risk from smoking marijuana once a month,
5. average annual rate of first use of marijuana,¹⁴
6. past month use of illicit drugs other than marijuana,
7. past year use of cocaine,
8. past year nonmedical use of pain relievers,
9. past month use of alcohol,
10. past month binge alcohol use,
11. perceptions of great risk from having five or more drinks of an alcoholic beverage once or twice a week,
12. past month use of tobacco products,
13. past month use of cigarettes,
14. perceptions of great risk from smoking one or more packs of cigarettes per day,
15. past year alcohol dependence or abuse,
16. past year alcohol dependence,
17. past year illicit drug dependence or abuse,
18. past year illicit drug dependence,
19. past year dependence or abuse of illicit drugs or alcohol,
20. needing but not receiving treatment for illicit drug use in the past year,
21. needing but not receiving treatment for alcohol use in the past year,
22. serious mental illness (SMI) in the past year,
23. any mental illness (AMI) in the past year,
24. serious thoughts of suicide in the past year, and
25. past year major depressive episode (MDE, i.e., depression).

Estimates of underage (aged 12 to 20) alcohol use and binge alcohol use were also produced. Comparisons between the 2012-2013 and the 2013-2014 state estimates were produced for all of these measures as well. For details on how measures such as AMI, SMI, MDE, illicit drugs, dependence or abuse, and average annual rate of first use of marijuana are defined, see "2011-2012 National Surveys on Drug Use and Health: Guide to State Tables and Summary of Small Area Estimation Methodology" at https://www.samhsa.gov/data/.

Illicit drug use includes the use of methamphetamines. NSDUH questions on methamphetamine use are asked in the stimulant module of the core section of the questionnaire in the context of questions about the nonmedical use of prescription stimulants. Beginning in 2005, new questions were added to the special drug module in the noncore section of the NSDUH questionnaire to capture information from respondents who may have used methamphetamines but did not recognize it as a prescription drug and therefore did not report use in the core stimulant module. However, the illicit drug estimates (including dependence, abuse, and treatment of illicit drugs) shown in the 2013-2014 small area estimation (SAE) documents include data from the original methamphetamine questions, but they do not include the new methamphetamine items added in 2005 and 2006 (i.e., the responses to the noncore questions). For more information on these new methamphetamine questions, see the findings from the methamphetamine analysis section of the 2005 NSDUH Methodological Resource Book (Center for Behavioral Health Statistics and Quality [CBHSQ], 2007).

B.3 Predictors Used in Mixed Logistic Regression Models

Local area data used as potential predictor variables in the mixed logistic regression models were obtained from several sources, including Nielsen Claritas, the U.S. Census Bureau, the Federal Bureau of Investigation (FBI) (Uniform Crime Reports [UCRs]), the Bureau of Labor Statistics (BLS), the Bureau of Economic Analysis (BEA), the Substance Abuse and Mental Health Services Administration (SAMHSA) (National Survey of Substance Abuse Treatment Services [N-SSATS]), and the National Center for Health Statistics (NCHS) (mortality data). Note that the predictors used to produce the 2013-2014 state small area estimates are the same as the predictors used to produce the 2012-2013 state small area estimates (however, values of the data were updated when possible). That is, no new variable selection was done for 2013-2014.

Sources and potential data items used in the modeling are provided in the following text and lists.

• Nielsen Claritas. This demographic data package contains data for 2013 with projections to 2018. The population projections are used to update these predictor variables each year. The 2013 population estimates and the 2014 population projections were used for producing the 2013-2014 state estimates. For more information on these data, see Section B.4.
• U.S. Census Bureau. The 2010 census (demographic and geographic variables) and 2012 food stamp participation estimates were used (http://www.census.gov/did/www/saipe/inputdata/cntysnap.xls). The Census Bureau's small area income and poverty estimates division obtains Food Stamp Program (now known as the Supplemental Nutrition Assistance Program [SNAP]) participation estimates from the U.S. Department of Agriculture, Food and Nutrition Service.
• American Community Survey (ACS). The 2009-2013 5-year ACS demographic and socioeconomic variables at the tract level and poverty variable at the county level were used (http://www.census.gov/acs/www/data_documentation/data_main/).
• Federal Bureau of Investigation. Uniform Crime Report (UCR) arrest totals were obtained from http://www.icpsr.umich.edu/icpsrweb/NACJD/archive.jsp. The most current data used are from 2012 for most counties, with previous years' data substituted in a few cases.
• Bureau of Labor Statistics (BLS). The 2014 county-level unemployment estimates were used (http://www.bls.gov/lau/tables.htm). The BLS uses results from the Current Population Survey (CPS) to provide county-level unemployment estimates. The CPS is a monthly survey of households conducted by the Census Bureau for the BLS.
• Bureau of Economic Analysis (BEA). The 2013 county-level per capita income estimates were used (http://bea.gov/iTable/index.cfm). These county-level per capita income estimates are produced by the Regional Income Division of the BEA.
• National Center for Health Statistics (NCHS). Mortality data using International Classification of Diseases, 10th revision (ICD-10), 2006-2011, were used. The ICD-10 death data are from the NCHS at the Centers for Disease Control and Prevention (CDC).
• SAMHSA, Center for Behavioral Health Statistics and Quality (CBHSQ, formerly the Office of Applied Studies [OAS]). Data were used from the National Survey of Substance Abuse Treatment Services (N-SSATS), formerly known as the Uniform Facility Data Set (UFDS). The 2013-2014 data on drug and alcohol treatment estimates were obtained. Maintenance of effort expenditures, block grant awards, cost of services, and total taxable resources data were also used.

The following lists provide the specific independent variables that were potential predictors in the models.

B.4 Benchmarking the Age Group-Specific Small Area Estimates

The self-calibration built into the survey-weighted hierarchical Bayes (SWHB) solution ensures that the population-weighted average of the state small area estimates will closely match the national design-based estimates. The national design-based estimates in NSDUH are based entirely on survey-weighted data using a direct estimation approach, whereas the state and census region estimates are model-based. Given the self-calibration ensured by the SWHB solution, for state reports prior to 2002, the standard Bayes prescription was followed; specifically, the posterior mean was used for the point estimate, and the tail percentiles of the posterior distribution were used for the Bayesian confidence interval limits.

Singh and Folsom (2001) extended Ghosh's (1992) results on constrained Bayes estimation to include exact benchmarking to design-based national estimates. In the simplest version of this constrained Bayes solution where only the design-based mean is imposed as a benchmarking constraint, each of the 2013-2014 state-by-age group small area estimates is adjusted by adding the common factor , where is the design-based national estimate and is the population-weighted mean of the state small area estimates for age group-a. The exactly benchmarked state-s and age group-a small area estimates then are given by . Experience with such additive adjustments suggests that the resulting exactly benchmarked state small area estimates will always be between 0 percent and 100 percent because the SWHB self-calibration ensures that the adjustment factor is small relative to the size of the state-level small area estimates.

Relative to the Bayes posterior mean, these benchmark-constrained state small area estimates are biased by the common additive adjustment factor. Therefore, the posterior mean squared error (MSE) for each benchmarked state small area estimate has the square of this adjustment factor added to its posterior variance. To achieve the desirable feature of exact benchmarking, this constrained Bayes adjustment factor was implemented for the state-by-age group small area estimates. The associated Bayesian confidence (credible) intervals can be recentered at the benchmarked small area estimates on the logit scale with the symmetric interval end points based on the posterior root mean squared errors (RMSEs). The adjusted 95 percent Bayesian confidence intervals are defined below:

,     D

where

,     D

,     D     and

.     D

The associated posterior coverage probabilities for these benchmarked intervals are very close to the prescribed 0.95 value because the state small area estimates have posterior distributions that can be approximated exceptionally well by a Gaussian distribution.

B.5 Calculation of Estimated Number of Individuals Associated with Each Outcome

Tables 1 to 26 of "2013-2014 NSDUHs: Model-Based Estimated Totals (in Thousands) (50 States and the District of Columbia)" show the estimated numbers of individuals associated with each of the 25 outcomes of interest.¹⁵ To calculate these numbers, the benchmarked small area estimates and the associated 95 percent Bayesian confidence intervals are multiplied by the average population across the 2 years (in this case, 2013 and 2014) of the state by the age group of interest.

For example, past month use of alcohol among 18 to 25 year olds in Alabama was 51.36 percent.¹⁶ The corresponding Bayesian confidence intervals ranged from 47.49 to 55.21 percent. The population count for 18 to 25 year olds averaged across 2013 and 2014 in Alabama was 535,409 (see Table C.10 in Section C of this methodology document). Hence, the estimated number of 18 to 25 year olds using alcohol in the past month in Alabama was 0.5136 × 535,409, which is 274,986.¹⁷ The associated Bayesian confidence intervals ranged from 0.4749 × 535,409 (i.e., 254,266) to 0.5521 × 535,409 (i.e., 295,599). Note that when estimates of the number of individuals are calculated for Tables 1 to 26 in "2013-2014 NSDUHs: Model-Based Estimated Totals (in Thousands) (50 States and the District of Columbia)" (follow the link in footnote 17), the unrounded percentages and population counts are used, then the numbers are reported to the nearest thousand. Hence, the number obtained by multiplying the published estimate with the published population estimate may not exactly match the counts that are published in these tables because of rounding differences.

The only exception to this calculation is the production of the estimated numbers of marijuana initiates. Those estimates cannot be directly calculated as the product of the percentage estimate of first use of marijuana and the population counts available in Section C. That is because the denominator of that percentage estimate is defined as the number of person years at risk for marijuana initiation, which is a combination of individuals who never used marijuana and one half of the individuals who initiated in the past 24 months.

B.6 Comparison of Two 2013-2014 Small Area Estimates

This section describes a method for determining whether differences between two 2013-2014 State population percentages are statistically significant. This procedure can be used for any two State population percentages representing the same age group (e.g., young adults aged 18 to 25) and time period (e.g., 2013-2014).

Let and denote the 2013-2014 age group-a specific prevalence rates for two different states, and , respectively. The null hypothesis of no difference, that is, , is equivalent to the log-odds ratio equal to zero, that is, , where is defined as , where ln denotes the natural logarithm. An estimate of is given by , where and are the 2013-2014 state estimates given in the "2013-2014 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 26, by Age Group)" (follow the link in footnote 16). To compute the variance of , that is, , let and , then , where denotes the covariance between and . This covariance is defined in terms of the associated correlation as follows:

.     D

The quantities and can be obtained by using the 95 percent Bayesian confidence intervals given in the "2013-2014 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia) (Tables 1 to 26, by Age Group)" (follow the link in footnote 16). For this purpose, let and denote the 95 percent Bayesian confidence intervals for the two states, and , respectively. Then

,     D

where .

For all practical purposes, the correlation between and is assumed to be negligible; hence, can be approximated by . The correlation is assumed to be negligible because each State was a stratum in the first level of stratification; therefore, each State sample is selected independently. However, the correlation between the two State estimates is theoretically nonzero because State estimates share common fixed-effect parameters in the SAE models. Hence, the test statistic (defined below) might result in a different conclusion in a few cases when the correlation between the State estimates is incorporated in calculating . To calculate the p value for testing the null hypothesis of no difference (), it is assumed that the posterior distribution of is normal with and . With the null value of , the Bayes p value or significance level for the null hypothesis of no difference is , where is a standard normal random variate, , and denotes the absolute value of . This Bayesian significance level (or p value) for the null value of , say , is defined following Rubin (1987) as the posterior probability for the collection of the values that are less likely or have smaller posterior density than the null (no change) value . That is, . With the posterior distribution of approximately normal, is given by the above expression.

Hence, to test whether differences between two 2013-2014 state estimates are statistically significant, the test statistic and the associated p value can be used. If p 0.05, then the two state estimates can be considered different at the 5 percent level of significance. Because age group estimates within a state are correlated, the method described here cannot be used to test whether differences between two age group estimates within a state are statistically significant.

When comparing estimates for two states, it is tempting and often convenient to look at their 95 percent Bayesian confidence intervals to decide whether the difference in the state estimates is significant. If the two Bayesian confidence intervals overlap, one would conclude that the difference is not statistically significant. If the two Bayesian confidence intervals do not overlap, it implies that the state estimates are significantly different from each other. However, the type-I error for the overlapping 95 percent Bayesian confidence intervals test may be as low as 0.6 percent (assuming that the two state estimates are uncorrelated and have the same variances) as compared with the 5 percent type-I error of the test based on the statistics defined above (Payton, Greenstone, & Schenker, 2003).

As discussed in Schenker and Gentleman (2001), the method of overlapping Bayesian confidence intervals is more conservative (i.e., it rejects the null hypothesis of no difference less often) than the standard method based on statistics when the null hypothesis is true. Even if Bayesian confidence intervals for two states overlap, the two estimates may be declared significantly different by the test based on statistics. Hence, the method of overlapping Bayesian confidence intervals is not recommended to test the difference of two state estimates. A detailed description of the method of overlapping confidence intervals and its comparison with the standard methods for testing of a hypothesis is given in Schenker and Gentleman (2001) and Payton et al. (2003).

Example. The percentages for past month alcohol use among 12 to 17 year olds in New Jersey and Oklahoma are shown in the following exhibit and also in Table 9 of the "2013-2014 NSDUH: Model-Based Prevalence Estimates (50 States and the District of Columbia)" at https://www.samhsa.gov/data/. Looking at the two 95 percent Bayesian confidence intervals, it would appear that the Oklahoma and New Jersey percentages for past month alcohol use are not statistically different at the 5 percent level of significance because the two Bayesian confidence intervals overlap:

However, in the following example, the test based on the statistic described earlier concludes that they are significantly different at the 5 percent level of significance.

Let , , , , , . Then,

,     D

,     D

,     D

,     D

,     D   and

.     D

Because the computed absolute value of is greater than or equal to 1.96 (the critical value of the statistic), then at the 5 percent level of significance, the hypothesis of no difference (Oklahoma prevalence rate = New Jersey prevalence rate) is rejected. Thus, the two state prevalence rates are statistically different. The Bayes p value or significance level for the null hypothesis of no difference is .

Section C: Sample Sizes, Response Rates, and Population Estimates