Premature ejaculation (PE) is one of the most common male sexual complaints, but its etiology is unclear. Psychological problems, such as symptoms of anxiety and depression, have traditionally been seen as causal or maintaining etiological components of PE, and previous cross-sectional studies have found weak positive associations between them. The aim of the present study was to test possible causal pathways over time between PE and symptoms of the psychological problems anxiety, depression, and sexual distress. A sample of 985 male Finnish twins and brothers of twins completed a questionnaire in 2006 and 2012. Significant bivariate correlations were found both within and across time between PE and the psychological problems. When fitting structural equation models to test hypothesized causal pathways, symptoms of anxiety and sexual distress at the first measurement time point did not predict future PE. Likewise, PE symptoms at the first measurement did not predict increments or decrements in anxiety, sexual distress, or depression later on. These null findings regarding hypothesized associations may partly be explained by the relatively long time between measurements, or that the measures possibly did not capture the aspects of anxiety that are related to PE.