Computerized detection of epileptiform transients (ETs), characterized by interictal spikes and sharp waves in the EEG, has been a research goal for the last 40 years. A reliable method for detecting ETs would assist physicians in interpretation and improve efficiency in reviewing long-term EEG recordings. Computer algorithms developed thus far for detecting ETs are not as reliable as human experts, primarily due to the large number of false-positive detections. Comparing the performance of different algorithms is difficult because each study uses individual EEG test datasets. In this article, we present EEGnet, a distributed web-based platform for the acquisition and analysis of large-scale training datasets for comparison of different EEG ET detection algorithms. This software allows EEG scorers to log in through the web, mark EEG segments of interest, and categorize segments of interest using a conventional clinical EEG user interface. This software platform was used by seven board-certified academic epileptologists to score 40 short 30-second EEG segments from 40 patients, half containing ETs and half containing artifacts and normal variants. The software performance was adequate. Interrater reliability for marking the location of paroxysmal activity was low. Interrater reliability of marking artifacts and ETs was high and moderate, respectively.