Morphology of anatomical structures can provide important diagnostic information regarding disease. Implicit features of morphology, such as contour smoothness or perimeter-to-area ratio, have been used in the context of computerized decision support classifiers to aid disease diagnosis. These features are usually specific to the domain and application (e.g., margin irregularity is a predictor of malignant breast lesions on DCE-MRI). In this paper we present a framework for extracting Diffeomorphic Based Similarity (DBS) features to capture subtle morphometric differences between shapes that may not be captured by implicit features. Object morphology is represented using the medial axis model and objects are compared by determining correspondences between medial axis models using a cluster-based diffeomorphic registration scheme. To visualize and classify morphometric differences, a manifold learning scheme (Graph Embedding) is employed to identify nonlinear dependencies between medial axis model similarity and calculate DBS. We evaluated our DBS on two clinical problems discriminating: (a) different Gleason grades of prostate cancer using gland morphology on a set of 102 images, and (b) benign and malignant lesions on 44 breast DCE-MRI studies. Precision-recall curves demonstrate DBS features are better able to classify shapes belonging to the same class compared to implicit features. A support vector machine (SVM) classifier is trained to distinguish between different classes utilizing DBS. SVM accuracy was 83 +/- 4.47% for distinguishing benign from malignant lesions on breast DCE-MRI and over 80% in distinguishing between intermediate Gleason grades of prostate cancer on digitized histology.