The eye pupil reacts to cognitive processes, but its analysis is challenging when luminance varies or when stimulation is fast-paced. Current approaches relying on deconvolution techniques do not account for the strong low-frequency spontaneous changes in pupil size or the large interindividual variability in the shape of the responses. Here a system identification framework is proposed in which the pupil responses to different parameters are extracted by means of an autoregressive model with exogenous inputs. In an example application of this technique, pupil size was shown to respond to the luminance and arousal scores of affective pictures presented in rapid succession. This result was significant in each subject (N = 5), but the pupil response varied between individuals both in amplitude and latency, highlighting the need for determining impulse responses subjectwise. The same method was also used to account for pupil size variations caused by respiration, illustrating the possibility to model the relation between pupil size and other continuous signals. In conclusion, this new framework for the analysis of pupil size data allows us to dissociate the response of the eye pupil from intermingled sources of influence and can be used to study the relation between pupil size and other physiological signals.