The loss of Neanderthal groups across Western and Central Europe during Oxygen Isotope Stage (OIS) 3 has held the attention of archaeologists for decades. The role that climatic change, genetic interbreeding, and interspecies competition played in the extinction of Neanderthal groups is still debated. Hohle Fels is one of several important Middle and Upper Paleolithic sites from the Ach Valley in southwestern Germany which documents the presence of Neanderthals and modern humans in the region. Chronological and stratigraphic records indicate that these two groups occupied the site with little to no overlap or interaction. This provides the opportunity to examine the behavioural variability of Swabian Neanderthal populations without the complication of cross-cultural influence. In this study we contribute a terrestrial paleoenvironmental record derived from the small mammal material from Hohle Fels Cave to the ever-growing archaeological record of this period. By reconstructing the climate and landscape of the Ach Valley during this time we can identify the effect that the OIS 3 environment had on the presence of Neanderthals in the region. Based on indicator taxa and the habitat weighing method, the small mammal record, which includes rodents, insectivores, and bats, from Hohle Fels shows that the earliest Neanderthal occupation took place on a landscape characterized by substantial woodland and forest, rivers and ponds, as well as moist meadows and grasslands. A gradual increase in cold tundra and arctic environments is clear towards the end of the Middle Paleolithic, extending to the end of the early Aurignacian which may correlate with the onset of the Heinrich 4 event (~42,000 kya). Our taphonomic analysis indicates the material was accumulated primarily by opportunistic predators such as the great grey owl, snowy owl, and European eagle owl, and therefore reflects the diversity of landscapes present around the site in the past. Importantly, at the time Neanderthals abandoned the Ach Valley we find no indication for dramatic climatic deterioration. Rather, we find evidence of a gradual cooling of the Swabian landscape which may have pushed Neanderthal groups out of the Ach Valley prior to the arrival of modern human Aurignacian groups.