The objective of this review is to report and discuss the evidence for fully automated diagnostic instruments for cutaneous melanoma tested in a real-world clinical setting directly compared with human diagnosis. A systematic review was performed and articles excluded when studies did not report sensitivity or specificity for melanoma directly compared with humans on an independent test set. Only 3 instruments have had their diagnostic accuracy compared with a human diagnosis in the clinical field with a meaningful sample size that could allow some generalization with the wider clinical arena. Two of these instruments showed a significantly inferior specificity for the diagnosis of melanoma compared with specialists. In one of these studies, the sensitivity for diagnosis, although superior to the specialist diagnosis, did not reach statistical significance. In contrast, one instrument had an equivalent specificity and trended superior but not significantly for sensitivity for the diagnosis of melanoma. Other image based nonclinic studies and studies comparing clinical management as the endpoint rather than diagnosis are also reviewed.