Skip to main content

Why does SAMHSA caution against comparing 2020 estimates with estimates from other years?

Because of the coronavirus disease 2019 (COVID-19) pandemic, major changes were made to the methods used in data collection in 2020. There is no way to separate out the true changes in behavior from the changes due to the new methodology.

The main methodological changes were:

  • Almost no data collection from mid-March through September 2020,
  • Introduction of web data collection in October 2020 with very limited in-person data collection and
  • Additions to the questionnaire beginning in October 2020.

The 2020 NSDUH is missing two quarters of data. Tests of data from the years preceding 2020 show that estimates based on just quarters 1 and 4 are not comparable to estimates based on the entire year’s worth of data. This means that 2020 estimates should not be compared to previous years. Repeated analyses have shown that web responses are not comparable to in-person responses, and differences are not consistent enough to be fully accounted for with weights or other statistical measures. This means that years with an in-person response option and years without do not produce comparable estimates. For these reasons, it is not recommended to compare any estimates from 2020 to estimates from other survey years.

A full description of the analyses that were conducted can be found in chapter 6 of the 2021 Methodological Summary and Definitions report.