Atypical dengue prevalence was observed in 2020 in many dengue-endemic countries, including Brazil. Evidence suggests that the pandemic disrupted not only dengue dynamics due to changes in mobility patterns, but also several aspects of dengue surveillance, such as care seeking behavior, care availability, and monitoring systems. However, we lack a clear understanding of the overall impact on dengue in different parts of the country as well as the role of individual causal drivers. In this study, we estimated the gap between expected and observed dengue cases in 2020 using an interrupted time series design with forecasts from a neural network and a structural Bayesian time series model. We also decomposed the gap into the impacts of climate conditions, pandemic-induced changes in reporting, human susceptibility, and human mobility. We find that there is considerable variation across the country in both overall pandemic impact on dengue and the relative importance of individual drivers. Increased understanding of the causal mechanisms driving the 2020 dengue season helps mitigate some of the data gaps caused by the COVID-19 pandemic and is critical to developing effective public health interventions to control dengue in the future.