Accurate estimates of COVID-19 burden of infections in communities can inform public health strategy for the current pandemic. Wastewater based epidemiology (WBE) leverages sewer infrastructure to provide insights on rates of infection by measuring viral concentrations in wastewater. By accessing the sewer network at various junctures, important insights regarding COVID-19 disease activity can be gained. The analysis of sewage at the wastewater treatment plant level enables population-level surveillance of disease trends and virus mutations. At the neighborhood level, WBE can be used to describe trends in infection rates in the community thereby facilitating local efforts at targeted disease mitigation. Finally, at the building level, WBE can suggest the presence of infections and prompt individual testing. In this critical review, we describe the types of data that can be obtained through varying levels of WBE analysis, concrete plans for implementation, and public health actions that can be taken based on WBE surveillance data of infectious diseases, using recent and successful applications of WBE during the COVID-19 pandemic for illustration.