Obesity, characterized by chronic low-grade inflammation of the adipose tissue, is associated with adverse coronavirus disease 2019 (COVID-19) outcomes, yet the underlying mechanism is unknown. To explore whether severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection of adipose tissue contributes to pathogenesis, we evaluated COVID-19 autopsy cases and deeply profiled the response of adipose tissue to SARS-CoV-2 infection in vitro. In COVID-19 autopsy cases, we identified SARS-CoV-2 RNA in adipocytes with an associated inflammatory infiltrate. We identified two distinct cellular targets of infection: adipocytes and a subset of inflammatory adipose tissue-resident macrophages. Mature adipocytes were permissive to SARS-CoV-2 infection; although macrophages were abortively infected, SARS-CoV-2 initiated inflammatory responses within both the infected macrophages and bystander preadipocytes. These data suggest that SARS-CoV-2 infection of adipose tissue could contribute to COVID-19 severity through replication of virus within adipocytes and through induction of local and systemic inflammation driven by infection of adipose tissue-resident macrophages.