The atmospheric compositions of Venus and Earth differ significantly, with the venusian atmosphere containing about 50 times as much 36Ar as the atmosphere on Earth. The different effects of the solar wind on planet-forming materials for Earth and Venus have been proposed to account for some of this difference in atmospheric composition, but the cause of the compositional difference has not yet been fully resolved. Here we propose that the absence or presence of an ocean at the surface of a protoplanet during the giant impact phase could have determined its subsequent atmospheric amount and composition. Using numerical simulations, we demonstrate that the presence of an ocean significantly enhances the loss of atmosphere during a giant impact owing to two effects: evaporation of the ocean, and lower shock impedance of the ocean compared to the ground. Protoplanets near Earth's orbit are expected to have had oceans, whereas those near Venus' orbit are not, and we therefore suggest that remnants of the noble-gas rich proto-atmosphere survived on Venus, but not on Earth. Our proposed mechanism explains differences in the atmospheric contents of argon, krypton and xenon on Venus and Earth, but most of the neon must have escaped from both planets' atmospheres later to yield the observed ratio of neon to argon.