Advances in brain-computer interface research have recently empowered the development of wearable sensors to record mobile electroencephalography (EEG) as an unobtrusive and easy-to-use alternative to conventional scalp EEG. One such mobile solution is to record EEG from the ear canal, which has been validated for auditory steady state responses and discrete event related potentials (ERPs). However, it is still under discussion where to place recording and reference electrodes to capture best responses to auditory stimuli. Furthermore, the technology has not yet been tested and validated for ecologically relevant auditory stimuli such as speech. In this study, Ear-EEG and conventional scalp EEG were recorded simultaneously in a discrete-tone as well as a continuous-speech design. The discrete stimuli were applied in a dichotic oddball paradigm, while continuous stimuli were presented diotically as two simultaneous talkers. Cross-correlation of stimulus envelope and Ear-EEG was assessed as a measure of ongoing neural tracking. The extracted ERPs from Ear-EEG revealed typical auditory components yet depended critically on the reference electrode chosen. Reliable neural-tracking responses were extracted from the Ear-EEG for both paradigms, albeit weaker in amplitude than from scalp EEG. In conclusion, this study shows the feasibility of extracting relevant neural features from ear-canal-recorded "Ear-EEG", which might augment future hearing technology.