We evaluated a series of pathologically proven breast tumors using an image-retrieval technique for classifying benign and malignant lesions. A total of 263 breast tumors (129 malignant and 134 benign) were retrospectively evaluated. The physician located regions-of-interest (ROI) of ultrasonic images and texture parameters (contrast, covariance and dissimilarity) were used in the process of the content-based image-retrieval technique. The accuracy of using the retrieval technique for classifying malignancies was 92.55% (236 of 255), the sensitivity was 94.44% (119 of 126), the specificity was 90.70% (117 of 129), the positive predictive value was 90.84% (119 of 131), and negative predictive value was 94.35% (117 of 124) for the proposed computer-aided diagnostic system. This computer-aided diagnosis system can provide a useful tool and its high negative predictive value could potentially help avert benign biopsies. It is unnecessary to perform any training procedures. This computer-aided diagnosis system can provide a second opinion for a sonographic interpreter; the main advantage in this proposed system is that we do not need any training. Historical cases can be directly added into the database and training of the diagnosis system again is not needed. With the growth of the database, more and more information can be collected and used as reference cases while performing diagnoses. This increases the flexibility of our diagnostic system.