Methodology Used to Estimate the Bayes Posterior Probability of No Change
between the 2002-2003 and 2011-2012 Small Area Estimates
To determine whether the differences between the 2002-2003 and 2011-2012 National Survey on Drug Use and Health (NSDUH) State prevalence rates are statistically significant, let 
and 
denote the 2002-2003 and 2011-2012 prevalence rates, respectively, for State-s and age group-a. The difference between and is defined in terms of the log odds ratio (
) as opposed to the simple difference because the posterior distribution of is closer to Gaussian than the posterior distribution of the simple difference (
). The is defined as
, D
where ln denotes the natural logarithm. The p value given in each of the tables in this document (i.e., Tables 1 to 22) is computed to test the null hypothesis of no change (i.e., 
or equivalently, 
). An estimate of is given by
, D
where 
are the previously published 2002-2003 State estimates1 and 
are the 2011-2012 State estimates presented in Tables 1 to 22 and elsewhere at https://www.samhsa.gov/data/report/2011-2012-nsduh-state-estimates-substance-use-and-mental-disorders. To compute the variance of 
, that is, 
, let 
and 
, then
, D
where 
denotes the covariance between 
and 
. This covariance is defined in terms of the associated correlation as follows:
. D
Note that 
and 
used here to calculate are the same variances used in calculating the previously published 2002-2003 Bayesian confidence intervals and the 2011-2012 Bayesian confidence intervals given here, respectively.
The correlation between and for an outcome measure by State by age group combination was estimated by taking the average of five correlations for each outcome measure by State by age group (2002-2003 vs. 2007-2008, 2002-2003 vs. 2008-2009, two sets of 2002-2003 vs. 2009-2010,2 and 2002-2003 vs. 2010-2011). The average of these five correlations is henceforth referred to as a "generalized correlation." Each of these five sets of correlations was produced by simultaneously fitting 4 years of NSDUH data separately for each outcome measure. For example, to produce correlations between the 2002-2003 and 2007-2008 State estimates for past month marijuana use, four age groups (12 to 17, 18 to 25, 26 to 34, and 35 or older) by two time periods (2002-2003 and 2007-2008), that is, eight subpopulation-specific models, were fitted each with its own set of fixed and random effects. In this case, the general covariance matrices for the State and within-State random effects were 8 × 8 matrices corresponding to the eight element (age group × time period) vectors of random effects.
For estimating correlations between the 2002-2003 and 2011-2012 State estimates, the pooled 2002-2003 data and the pooled 2011-2012 data were not modeled simultaneously (unlike in previous years) because the 2002-2003 estimates and data were based on the 2000 census geography and the 2011-2012 estimates and data were based on the 2010 census. In addition, new variable selection was performed on the 2011-2012 data, resulting in the inclusion of fixed-effect predictors that may differ from predictors from the 2002-2003 models. For the 2002-2003 versus 2011-2012 comparisons given here and for future comparisons involving extended time period differences, such as 2002-2003 versus 2012-2013, these generalized correlations will be used. Tables showing these generalized correlations will available at https://www.samhsa.gov/data/report/2011-2012-nsduh-state-estimates-substance-use-and-mental-disorders.
Note that in tests conducted earlier using 2002-2003 versus 2007-2008 data and 2002-2003 versus 2008-2009 data, when the actual (modeled) correlation was replaced by an average correlation of those 2 years, the p values from both sets of comparisons were fairly comparable, and in most cases they had the same conclusion of whether the test between 2002-2003 and 2007-2008 or 2008-2009 was significant or not. Only a few tests changed their significance (61 out of 9,632 tests across all outcomes, States, and age groups changed their conclusion, from significant to nonsignificant and vice versa) depending on which correlation (actual or the average) was used. More details on this study will be available at https://www.samhsa.gov/data/Methodological_Reports.aspx.
To calculate the p value for testing the null hypothesis of no difference (
), it is assumed (based on some empirical studies carried out earlier) that the posterior distribution of 
is normal with 
and 
. With the null value of (), the Bayes p value or posterior probability of no difference is 
, where 
is a standard normal random variate, 
, and 
denotes the absolute value of 
.
