Skip to main page content
Access keys NCBI Homepage MyNCBI Homepage Main Content Main Navigation
, 7 (5), e1133

Crowdsourced Validation of a Machine-Learning Classification System for Autism and ADHD

Affiliations

Crowdsourced Validation of a Machine-Learning Classification System for Autism and ADHD

M Duda et al. Transl Psychiatry.

Abstract

Autism spectrum disorder (ASD) and attention deficit hyperactivity disorder (ADHD) together affect >10% of the children in the United States, but considerable behavioral overlaps between the two disorders can often complicate differential diagnosis. Currently, there is no screening test designed to differentiate between the two disorders, and with waiting times from initial suspicion to diagnosis upwards of a year, methods to quickly and accurately assess risk for these and other developmental disorders are desperately needed. In a previous study, we found that four machine-learning algorithms were able to accurately (area under the curve (AUC)>0.96) distinguish ASD from ADHD using only a small subset of items from the Social Responsiveness Scale (SRS). Here, we expand upon our prior work by including a novel crowdsourced data set of responses to our predefined top 15 SRS-derived questions from parents of children with ASD (n=248) or ADHD (n=174) to improve our model's capability to generalize to new, 'real-world' data. By mixing these novel survey data with our initial archival sample (n=3417) and performing repeated cross-validation with subsampling, we created a classification algorithm that performs with AUC=0.89±0.01 using only 15 questions.

Conflict of interest statement

The authors declare no conflict of interest.

Figures

Figure 1
Figure 1
Representation of our machine-learning pipeline. In Trial 1, each of the five models was trained on 100 random subsamples of the archival data and tested on the same survey sample. In Trial 2, each model was trained once on the entire survey sample and tested on 100 random subsamples of the archival data. In Trial 3, each algorithm was trained and tested on 50 rounds of twofold CV with subsampling. CV, cross-validation.
Figure 2
Figure 2
Classification performance boxplot of five algorithms trained on archival, survey and mixed data samples over 100 validation trials.
Figure 3
Figure 3
Distribution of responses to each of the 15 questions for both the archival (n=2925) and survey (n=422) sets. The coded response scale corresponds to answers of ‘Always’ (4), ‘Often’ (3), ‘Sometimes’ (2) and ‘Never’ (1) for each question. For each box, both the median (line) and the mean (diamond) of that subset is shown.
Figure 4
Figure 4
Performance of the five machine-learning models on five different training and testing set combinations across 100 trials.
Figure 5
Figure 5
ROC and PR curves and prediction score distributions for ENet (a) and LDA (b) classifiers using fivefold CV with subsampling on the mixed data set. CV, cross-validation; ENet, Elastic Net; LDA, linear discriminant analysis; PR, precision–recall; ROC, receiver operating characteristic.
Figure 6
Figure 6
Feature weights of ENet (a) and LDA (b) models over fivefold CV with subsampling on the mixed data set. Important features (those with largest absolute mean feature weight) are indicated in red, and are consistent between the two models. CV, cross-validation; ENet, Elastic Net; LDA, linear discriminant analysis.

Similar articles

See all similar articles

Cited by 4 PubMed Central articles

References

    1. Developmental Disabilities Monitoring Network Surveillance Year Principal I. Prevalence of autism spectrum disorder among children aged 8 years - autism and developmental disabilities monitoring network, 11 sites, United States, 2010. MMWR Surveil Summ 2014; 63(Suppl 2): 1–21. - PubMed
    1. Visser SN, Danielson ML, Bitsko RH, Holbrook JR, Kogan MD, Ghandour RM et al. Trends in the parent-report of health care provider-diagnosed and medicated attention-deficit/hyperactivity disorder: United States, 2003-2011. J Am Acad Child Adolesc Psychiatry 2014; 53: 34–46 e32. - PMC - PubMed
    1. Berenguer-Forner C, Miranda-Casas A, Pastor-Cerezuela G, Rosello-Miranda R. [Comorbidity of autism spectrum disorder and attention deficit with hyperactivity. A review study]. Rev Neurol 2015; 60(Suppl 1): S37–S43. - PubMed
    1. Wiggins LD, Baio J, Rice C. Examination of the time between first evaluation and first autism spectrum diagnosis in a population-based sample. J Dev Behav Pediatr 2006; 27(2 Suppl): S79–S87. - PubMed
    1. Veness C, Prior M, Eadie P, Bavin E, Reilly S. Predicting autism diagnosis by 7 years of age using parent report of infant social communication skills. J Paediatr Child Health 2014; 50: 693–700. - PubMed

MeSH terms

Feedback