2022‑2023
National Surveys on Drug Use and Health:
Other Sources of State-Level Data

Introduction

A variety of surveys and data systems other than the National Survey on Drug Use and Health (NSDUH) collect data on substance use and mental health. This document briefly describes one of these other data systems that publish state estimates and presents selected comparisons with NSDUH results. The state-level survey that collects data on substance use discussed in this document is the Behavioral Risk Factor Surveillance System (BRFSS), sponsored by the Centers for Disease Control and Prevention (CDC).

Another CDC data system that provides state-level substance use estimates for most but not all states is the Youth Risk Behavior Survey (YRBS). Differences between the YRBS and NSDUH sampling designs, as well as the wider range of age groups used in NSDUH state small area estimates mean that comparisons of estimates are not straightforward. However, ignoring these differences and examining estimates at the national level, the YRBS has generally been shown to have higher estimates than NSDUH (Center for Behavioral Health Statistics and Quality [CBHSQ], 2023).¹ Note that comparisons between the state YRBS estimates and the NSDUH state small area estimates are not presented because of some of the differences discussed above.

When considering the information presented in this document, it is important to understand the methodological differences between the BRFSS and NSDUH and the impact that these differences could have on estimates of substance use and mental health. Several studies have compared NSDUH estimates with estimates from other studies and have evaluated how differences may have been affected by differences in survey methodology (Brener et al., 2006; CBHSQ, 2012; Gfroerer et al., 1997; Grucza et al., 2007; Hennessy & Ginsberg, 2001; Miller et al., 2004). These studies suggest that the goals and approaches of surveys are often different, making comparisons among them difficult. Some methodological differences that have been identified as affecting comparisons include populations covered, sampling methods, mode of data collection, survey setting, questionnaires, and estimation methods.

Because of the coronavirus disease 2019 (COVID-19) pandemic, an additional web data collection mode was introduced to the 2020 NSDUH. The 2022 and 2023 surveys continued the use of multimode data collection procedures that were first implemented in October 2020 for the 2020 NSDUH. For 2022, 42.4 percent of interviews were completed via the web, and 57.6 percent were completed in person. In 2023, 36.1 percent of the interviews were completed via the web, and 63.9 percent were completed in person.²

BRFSS is an annual state-based system of health surveys that collects information using landline and cellular telephones on health-related risk behaviors (including cigarette and alcohol use), chronic health conditions, healthcare access, and use of preventive services among the noninstitutionalized adult population aged 18 or older. During 2022 and 2023, all 50 states, the District of Columbia, Guam, Puerto Rico, and the U.S. Virgin Islands collected BRFSS data using computer-assisted telephone interviewing (CATI). Note that, in 2023, Kentucky and Pennsylvania were unable to collect enough BRFSS data to meet the minimum requirements for inclusion in the 2023 public dataset. Thus, the estimates shown in this report for Kentucky and Pennsylvania are based only on 2022 data. The 2022‑2023 BRFSS state estimates and confidence intervals presented here are design-based (direct) estimates (i.e., each respondent is weighted in a way that accounts for the survey design).³

In both BRFSS and NSDUH, data are collected on the following three substance use measures in each of the 50 states and the District of Columbia:⁴

• past month alcohol use,
• cigarette use (“past month” use for NSDUH and “current” use for BRFSS), and
• past month binge alcohol use.

Note that estimates for these measures are compared in this document. The BRFSS and NSDUH questions that were used for the three measures are shown in the next section.

Past month alcohol use is defined consistently in both BRFSS and NSDUH as having an alcoholic beverage in the past month. Similarly, past month binge alcohol use is defined consistently in the two surveys as drinking five or more drinks (for males) or four or more drinks (for females) on the same occasion (i.e., at the same time or within a couple of hours of each other) on at least 1 day in the past 30 days.

In NSDUH, past month cigarette use is defined as having smoked part or all of a cigarette during the past 30 days (i.e., the 30 days prior to the interview). In BRFSS, the cigarette use measure reported is current cigarette use, which is defined as having smoked at least 100 cigarettes during the lifetime and indicating smoking every day or some days at the time of the survey. Because of these subtle but present differences in definitions, NSDUH’s cigarette use estimates tend to be higher in that they cover two groups of people that the BRFSS estimates would not: (1) respondents who have not smoked 100 cigarettes in their lifetime but had smoked in the past month, and (2) respondents who had smoked a cigarette earlier in the month but were not smoking at the time of the survey.

This document presents findings comparing 2022‑2023 BRFSS state design-based estimates with corresponding 2022‑2023 NSDUH state small area estimates⁵ for past month alcohol use, past month binge alcohol use, and cigarette use (“past month” use for NSDUH and “current” use for BRFSS). In Tables 1, 2, and 3 (shown after this text discussion), the 2022‑2023 BRFSS state design-based estimates for adults aged 18 or older are shown alongside the 2022‑2023 NSDUH small area estimates for the same age group. The BRFSS estimates were calculated using SUDAAN^® Software for Statistical Analysis of Correlated Data (RTI International, 2013) and the publicly available BRFSS SAS^® (SAS Institute Inc., 2017) datasets. Tables 1 and 2 also include p values that indicate whether the BRFSS and NSDUH alcohol use and binge alcohol use population percentages are significantly different from each other for a given state. The statistical test used for calculating the p values is described in the Methodology for Comparing BRFSS and NSDUH Estimates section. Users are advised to use caution when interpreting these significant differences due to the methodological differences in the two surveys.

Due to definitional differences in the cigarette use measure, no formal statistical tests of differences between NSDUH and BRFSS estimates were produced.

NSDUH and BRFSS Questions

The 2022 and 2023 NSDUH questions that were used to determine past month alcohol use and past month binge alcohol use were as follows:⁶

AL01

Have you ever, even once, had a drink of any type of alcoholic beverage? Please do not include times when you only had a sip or two from a drink.

1   Yes
2   No
DK/REF⁷

ALLAST3

[IF AL01 = 1 OR ALREF = 1] How long has it been since you last drank an alcoholic beverage?

1   Within the past 30 days — that is, since [DATEFILL]
2   More than 30 days ago but within the past 12 months
3   More than 12 months ago

DK/REF
PROGRAMMER: SHOW 12 MONTH CALENDAR

AL08

[IF ALC30DAY = 1 – 30 OR ALCEST30 = (1 – 6, DK OR REF)] During the past 30 days, that is, since [DATEFILL], on how many days did you have [IF RSEX=5 THEN FILL 5 IF RSEX=9 THEN FILL 4] or more drinks on the same occasion? By ’occasion,’ we mean at the same time or within a couple of hours of each other.

# OF DAYS: _____ days in the past 30 days [RANGE: 0 - 30]
DK/REF
PROGRAMMER: SHOW 30 DAY CALENDAR

The 2022 and 2023 BRFSS questions that were used to determine past month alcohol use and past month binge alcohol use were as follows:⁸

CALC.01

During the past 30 days, how many days per week or per month did you have at least one drink of any alcoholic beverage?

INTERVIEWER NOTE: A 40-ounce beer would count as 3 drinks, or a cocktail with 2 shots would count as 2 drinks.

1 _ _ Days per week
2 _ _ Days in past 30 days
888 No drinks in past 30 days
777 Don’t know / Not sure
999 Refused

CALC.03

Considering all types of alcoholic beverages, how many times during the past 30 days did you have X [CATI X = 5 for men, X = 4 for women] or more drinks on an occasion?

_ _ Number of times
77 Don’t know / Not sure
88 No days
99 Refused

The 2002 and 2023 NSDUH questions that were used to determine past month cigarette use were as follows:

CG01

Have you ever smoked part or all of a cigarette?

1   Yes
2   No
DK/REF

CG05

[IF CG01 = 1 OR CGREF1 = 1] Now think about the past 30 days, that is, from [DATEFILL] up to and including today. During the past 30 days, have you smoked part or all of a cigarette?

1   Yes
2   No
DK/REF
PROGRAMMER: SHOW 30 DAY CALENDAR

The 2002 and 2023 BRFSS questions that were used to determine current cigarette use were as follows:

CTOB.01

Have you smoked at least 100 cigarettes in your entire life?

1   Yes
2   No
7   Don’t know / Not sure
9   Refused

INTERVIEWER NOTE: Do not include: electronic cigarettes (e-cigarettes, njoy, bluetip, JUUL), herbal cigarettes, cigars, cigarillos, little cigars, pipes, bidis, kreteks, water pipes (hookahs) or marijuana. 5 packs = 100 cigarettes.

CTOB.02

Do you now smoke cigarettes every day, some days, or not at all?

1   Every day
2   Some days
3   Not at all
7   Don’t know / Not sure
9   Refused

Methodology for Comparing BRFSS and NSDUH Estimates

The methodology used to compare BRFSS and NSDUH estimates is similar to what is described in Section B.7 of 2014-2015 National Survey on Drug Use and Health: Guide to State Tables and Summary of Small Area Estimation Methodology.⁹ Here, the null hypothesis of no difference is tested, that is, (where is the expected value¹⁰ of the BRFSS estimate and is the expected value of the NSDUH estimate), or equivalently the log-odds ratio is zero, that is, , where is defined as , and ln denotes the natural logarithm. An estimate of is given by , where and are the 2022‑2023 BRFSS state-level design-based estimates and the 2022‑2023 NSDUH state model-based estimates, respectively (as given in Tables 1 and 2). To compute the variance of , that is, , let and , then

.     D

The covariance term can be assumed to be zero because the BRFSS and NSDUH samples are independent.

The quantity can be obtained by using the 95 percent Bayesian confidence intervals in Tables 1 and 2. For this purpose, let denote the 95 percent Bayesian confidence interval¹¹ for a given state:

,     D

where .

The quantity can be obtained by using the 95 percent confidence intervals in Tables 1 and 2. For this purpose, let denote the 95 percent BRFSS confidence interval for a given state, then is given by

.     D

Now, using the first-order Taylor series approximation,¹² can be calculated from as follows:

.     D

The p value that is given in Tables 1 and 2 for testing the null hypothesis of no difference () is provided by , where is a standard normal random variate, , and denotes the absolute value of .

Results: Alcohol Use, Binge Alcohol Use, and Cigarette Use

As seen in Table 1, for past month alcohol use, the 2022‑2023 NSDUH estimates and the 2022‑2023 BRFSS estimates were statistically significantly different (i.e., at the 5 percent level of significance) for seven states (Arizona, Delaware, Hawaii, Kentucky, Pennsylvania, Texas, and Wyoming).¹³ Also, these two sets of estimates were highly correlated (correlation coefficient = 0.92).

The NSDUH estimates of past month binge alcohol use were significantly larger than the BRFSS estimates for all states (see Table 2). As noted previously, NSDUH and BRFSS used the same thresholds for binge alcohol use among males and females in the 2022 and 2023 surveys; therefore, these differences can be partly attributed to differences in data collection methodologies of BRFSS and NSDUH. First, the 2022‑2023 NSDUHs used audio computer-assisted self-interviewing (ACASI) for in-person data collection and self-administration for web data collection, whereas BRFSS used CATI. Self-administration (including ACASI for in-person data collection) can increase respondent privacy for reporting of sensitive behaviors (such as binge drinking) and therefore may yield higher prevalence estimates than interviewer-administered modes such as CATI (Kreuter et al., 2008; Lind et al., 2013; Tourangeau & Smith, 1996; Turner et al., 1998). Although the NSDUH estimates were larger, these two sets of estimates are moderately correlated (correlation coefficient = 0.75).

Table 3 shows the NSDUH estimates of past month cigarette use and the BRFSS estimates of current cigarette use. Statistical tests to examine significant differences between the NSDUH and BRFSS cigarette use population percentages are not included because the definitions are different, as discussed earlier in this document. Although the NSDUH estimates tended to be larger, these two sets of estimates were highly correlated (correlation coefficient = 0.91).

Sample Size Comparisons

The BRFSS estimates are design based, whereas the NSDUH estimates are model based. Both sets of estimates are based on 2 years of pooled data (2022‑2023). The BRFSS sample sizes for a given state were, in general, much larger than the sample sizes for NSDUH. In the 2022‑2023 NSDUHs, the 18 or older sample sizes in the states ranged from approximately 1,130 to 6,580 respondents, with a median sample size of 1,680.¹⁴ For the 2022‑2023 BRFSSs, states had larger sample sizes as compared with their counterparts in NSDUH. Overall, the BRFSS sample sizes for the states varied from a low of 4,023 respondents in Kentucky¹⁵ to a high of 52,596 in Washington, with a median sample size of 15,579.¹⁶ Sample size differences of this magnitude explain why the NSDUH Bayesian confidence intervals were generally wider than the corresponding BRFSS design-based confidence intervals.

References

Brener, N. D., Eaton, D. K., Kann, L., Grunbaum, J. A., Gross, L. A., Kyle, T. M., & Ross, J. G. (2006). The association of survey setting and mode with self-reported health risk behaviors among high school students. Public Opinion Quarterly, 70(3), 354-374. https://doi.org/10.1093/poq/nfl003 

Center for Behavioral Health Statistics and Quality. (2012). Comparing and evaluating youth substance use estimates from the National Survey on Drug Use and Health and other surveys (HHS Publication No. SMA 12-4727, Methodology Series M-9). Substance Abuse and Mental Health Services Administration. https://www.samhsa.gov/data/sites/default/files/NSDUH-M9-Youth-2012/NSDUH-M9-Youth-2012.pdf

Center for Behavioral Health Statistics and Quality. (2023). Chapter 5: Other sources of data. In 2022 National Survey on Drug Use and Health (NSDUH): Methodological summary and definitions. https://www.samhsa.gov/data/sites/default/files/reports/rpt42729/2022-nsduh-method-summary-defs/2022-nsduh-method-summary-defs-110123.pdf

Gfroerer, J., Wright, D., & Kopstein, A. (1997). Prevalence of youth substance use: The impact of methodological differences between two national surveys. Drug and Alcohol Dependence, 47(1), 19-30. https://doi.org/10.1016/s0376-8716(97)00063-x 

Grucza, R. A., Abbacchi, A. M., Przybeck, T. R., & Gfroerer, J. C. (2007). Discrepancies in estimates of prevalence and correlates of substance use and disorders between two national surveys. Addiction, 102(4), 623-629. https://doi.org/10.1111/j.1360-0443.2007.01745.x 

Hennessy, K. H., & Ginsberg, C. (2001). Substance use survey data collection methodologies: Introduction. Journal of Drug Issues, 31(3), 595-597. https://doi.org/10.1177/002204260103100301 

Kreuter, F., Presser, S., & Tourangeau, R. (2008). Social desirability bias in CAI, IVR, and web surveys: The effects of mode and question sensitivity. Public Opinion Quarterly, 72(5), 847-865. https://doi.org/10.1093/poq/nfn063 

Lind, L., Schober, M., Conrad, F., & Reichert, H. (2013). Why do survey respondents disclose more when computers ask the questions? Public Opinion Quarterly, 77(4), 888-935. https://doi.org/10.1093/poq/nft038 

Miller, J. W., Gfroerer, J. C., Brewer, R. D., Naimi, T. S., Mokdad, A., & Giles, W. H. (2004). Prevalence of adult binge drinking: A comparison of two national surveys. American Journal of Preventive Medicine, 27(3), 197-204. https://doi.org/10.1016/s0749-3797(04)00121-7 

RTI International. (2013). SUDAAN^® language manual, release 11.0.1.

SAS Institute Inc. (2017). SAS/STAT software: Release 14.1.

Tourangeau, R., & Smith, T. W. (1996). Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opinion Quarterly, 60(2), 275-304. https://doi.org/10.1086/297751 

Turner, C. F., Ku, L., Rogers, S. M., Lindberg, L. D., Pleck, J. H., & Sonenstein, F. L. (1998). Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science, 280(5365), 867-873. https://doi.org/10.1126/science.280.5365.867