Next-generation sequencing technologies have led to profound characterization of mutation spectra for several cancer types. Hence, we sought to systematically compare genomic aberrations between primary tumors and cancer lines. For this, we compiled publically available sequencing data of 1651 genes across 905 cell lines. We used them to characterize 23 distinct primary tumor sites by a novel approach that is based on Bayesian spam-filtering techniques. Thereby, we confirmed the strong overall similarity of alterations between patient samples and cell culture. However, we also identified several suspicious mutations, which had not been associated with their cancer types before. Based on these characterizations, we developed the inferring cancer origins from mutation spectra (ICOMS) tool. On our cell line collection, the algorithm reached a prediction specificity rate of 79%, which strongly variegated between primary cancer sites. On an independent validation cohort of 431 primary tumor samples, we observed a similar accuracy of 71%. Additionally, we found that ICOMS could be employed to deduce further attributes from mutation spectra, including sub-histology and compound sensitivity. Thus, thorough classification of site-specific mutation spectra for cell lines may decipher further genome-phenotype associations in cancer.