A growing body of research has examined the impact of childhood adversity on neural structure and function. Advances in our understanding of the neurodevelopmental consequences of adverse early environments require the identification of dimensions of environmental experience that influence neural development differently and mechanisms other than the frequently-invoked stress pathways. We propose a novel conceptual framework that differentiates between deprivation (absence of expected environmental inputs and complexity) and threat (presence of experiences that represent a threat to one's physical integrity) and make predictions grounded in basic neuroscience principles about their distinct effects on neural development. We review animal research on fear learning and sensory deprivation as well as human research on childhood adversity and neural development to support these predictions. We argue that these previously undifferentiated dimensions of experience exert strong and distinct influences on neural development that cannot be fully explained by prevailing models focusing only on stress pathways. Our aim is not to exhaustively review existing evidence on childhood adversity and neural development, but to provide a novel framework to guide future research.