Theoretical models of time perception suggest a simple bottom-up relationship between physiological arousal and perceived duration. Increases in physiological arousal lengthen the perceived duration of events whereas decreases in physiological arousal reduce them. Whilst this relationship has been demonstrated for highly arousing negatively valenced stimuli, it has not been demonstrated for other classes of distorting stimuli (e.g. positively valenced or low arousal stimuli). The current study tested the effect of valence (positive and negative) and arousal level (high and low) on the relationship between physiological arousal and perceived duration. Sympathetic nervous system (SNS) and parasympathetic nervous system (PSNS) activity was measured during a verbal estimation task in which participants judged the duration of high and low arousal, positive, negative and neutrally valenced IAPS images. SNS and PSNS activity were indexed by measuring Pre-Ejection Period (PEP) and High Frequency Heart-rate Variability (HF-HRV) respectively. SNS reactivity was predicative of perceived duration, but only for high arousal negatively valenced stimuli, with decreases in PEP being associated with longer duration estimates. SNS and PSNS activity was not predictive of perceived duration for the low arousal negative stimuli or the low and high arousal positive stimuli. We therefore propose a new model suggesting that emotional distortions to time result from a combination of bottom-up (physiological arousal) and top-down (threat detection) factors.