Abrupt changes in climate, termed Dansgaard-Oeschger and Heinrich events, have punctuated the last glacial period (approximately 100-10 kyr ago) but not the Holocene (the past 10 kyr). Here we use an intermediate-complexity climate model to investigate the stability of glacial climate, and we find that only one mode of Atlantic Ocean circulation is stable: a cold mode with deep water formation in the Atlantic Ocean south of Iceland. However, a 'warm' circulation mode similar to the present-day Atlantic Ocean is only marginally unstable, and temporary transitions to this warm mode can easily be triggered. This leads to abrupt warm events in the model which share many characteristics of the observed Dansgaard-Oeschger events. For a large freshwater input (such as a large release of icebergs), the model's deep water formation is temporarily switched off, causing no strong cooling in Greenland but warming in Antarctica, as is observed for Heinrich events. Our stability analysis provides an explanation why glacial climate is much more variable than Holocene climate.