Explaining the evolution and maintenance of cooperation among unrelated individuals is one of the fundamental problems in biology and the social sciences. Recent findings suggest that altruistic punishment is an important mechanism maintaining cooperation among humans. We experimentally explore the boundaries of altruistic punishment to maintain cooperation by varying both the cost and the impact of punishment, using an exceptionally extensive subject pool. Our results show that cooperation is only maintained if conditions for altruistic punishment are relatively favourable: low cost for the punisher and high impact on the punished. Our results indicate that punishment is strongly governed by its cost-to-impact ratio and that its effect on cooperation can be pinned down to one single variable: the threshold level of free-riding that goes unpunished. Additionally, actual pay-offs are the lowest when altruistic punishment maintains cooperation, because the pay-off destroyed through punishment exceeds the gains from increased cooperation. Our results are consistent with the interpretation that punishment decisions come from an amalgam of emotional response and cognitive cost-impact analysis and suggest that altruistic punishment alone can hardly maintain cooperation under multi-level natural selection. Uncovering the workings of altruistic punishment as has been done here is important because it helps predicting under which conditions altruistic punishment is expected to maintain cooperation.