Our objective was to describe how residents of Philadelphia, Pennsylvania, coped psychologically with the first wave of COVID-19 pandemic. In a cross-sectional design, we aimed to estimate the rates and correlates of anxiety and depression, examine how specific worries correlated with general anxiety and depression, and synthesize themes of "the most difficult experiences" shared by the respondents. We collected data through an on-line survey in a convenience sample of 1,293 adult residents of Philadelphia, PA between April 17 and July 3, 2020, inquiring about symptoms of anxiety and depression (via the Hospital Anxiety and Depression Scale), specific worries, open-ended narratives of "the most difficult experiences" (coded into themes), demographics, perceived sources of support, and general health. Anxiety was evident among 30 to 40% of participants and depression-about 10%. Factor analysis revealed two distinct, yet inter-related clusters of specific worries related to mood disorders: concern about "hardships" and "fear of infection". Regression analyses revealed that anxiety, depression, and fear of infection, but not concern about hardships, worsened over the course of the epidemic. "The most difficult experiences" characterized by loss of income, poor health of self or others, uncertainty, death of a relative or a friend, and struggle accessing food were each associated with some of the measures of worries and mood disorders. Respondents who believed they could rely on support of close personal network fared better psychologically than those who reported relying primarily on government and social services organizations. Thematic analysis revealed complex perceptions of the pandemic by the participants, giving clues to both positive and negative experiences that may have affected how they coped. Despite concerns about external validity, our observations are concordant with emerging evidence of psychological toll of the COVID-19 pandemic and measures employed to mitigate risk of infection.