Neglectful mothering is one of the most common forms of childhood maltreatment, involving a severe disregard of the child's needs, yet little is known about its neural substrate. A child's needs are usually conveyed by signals of distress revealed by crying faces. We tested whether infant and adult crying faces are processed differently in two sociodemographically similar groups of Neglectful (NM) and non-neglectful Control Mothers (CM). We used functional brain imaging to analyze the BOLD response from 43 mothers (23 neglectful and 20 control) while viewing faces from infants and adults (crying and neutral). In NM as compared to CM, the BOLD responses to both infant and adult crying faces were significantly reduced in the cerebellum, lingual, fusiform, amygdala, hippocampus, parahippocampus, and inferior frontal gyrus. The reduced BOLD was also modulated by comorbid psychiatric symptoms. In the CM, frontal activation to infant versus adult crying faces was enhanced, whereas in the NM activation in the anterior cingulate cortex to infant crying was reduced compared to adult crying. The altered neural response to crying faces in NM, showing generic face and infant-specific face processing deficits, could underlie their characteristic poor social abilities as well as their poor response to infant needs, both affecting the caregiving role.