Why does SAMHSA caution against comparing 2021 and later estimates with estimates from prior years?

Starting largely in 2021, NSDUH became a fully mixed-mode survey, with many responses from both web and in-person administration. Repeated analyses showed that the web responses are not comparable to the in-person responses and that the comparability is not consistent in a way that we can fully account for. Because of this, data from 2021 should not be compared to estimates from 2019 or earlier.

Estimates from 2021 should also not be compared to estimates from 2020 or even from just quarter 4 of 2020. The reasons are a bit complex. First, the 2020 NSDUH is missing two quarters of data. Tests of data from before 2020 show that estimates based on just quarters 1 and 4 are not comparable to estimates based on the entire year. This indicates that 2020 estimates should not be compared to 2021.

Moreover, while the methodology for Quarter 4 of 2020 is mostly the same as for the 2021 NSDUH, data in Quarter 4 of 2020 mostly consists of web interviews. In 2021, the proportion of in-person interviews increased over the year as county-level restrictions related to COVID-19 were gradually lifted, and field interviewers were able to follow up in more areas. These mostly single-mode estimates from quarter 4 should not be compared to the multi-mode estimates from 2021.

A full description of the analyses can be found in chapter 6 of the 2021 Methodological Summary and Definitions report.