The striking diversity of psychological and neurophysiological models of 'time perception' characterizes the debate on how and where in the brain time is processed. In this review, the most prominent models of time perception will be critically discussed. Some of the variation across the proposed models will be explained, namely (i) different processes and regions of the brain are involved depending on the length of the processed time interval, and (ii) different cognitive processes may be involved that are not necessarily part of a core timekeeping system but, nevertheless, influence the experience of time. These cognitive processes are distributed over the brain and are difficult to discern from timing mechanisms. Recent developments in the research on emotional influences on time perception, which succeed decades of studies on the cognition of temporal processing, will be highlighted. Empirical findings on the relationship between affect and time, together with recent conceptualizations of self- and body processes, are integrated by viewing time perception as entailing emotional and interoceptive (within the body) states. To date, specific neurophysiological mechanisms that would account for the representation of human time have not been identified. It will be argued that neural processes in the insular cortex that are related to body signals and feeling states might constitute such a neurophysiological mechanism for the encoding of duration.