Many drugs commonly prescribed during pregnancy lack a fetal safety recommendation - called FDA 'category C' drugs. This study aims to classify these drugs into harmful and safe categories using knowledge gained from chemoinformatics (i.e., pharmacological similarity with drugs of known fetal effect) and empirical data (i.e., derived from Electronic Health Records). Our fetal loss cohort contains 14,922 affected and 33,043 unaffected pregnancies and our congenital anomalies cohort contains 5,658 affected and 31,240 unaffected infants. We trained a random forest to classify drugs of unknown pregnancy class into harmful or safe categories, focusing on two distinct outcomes: fetal loss and congenital anomalies. Our models achieved an out-of-bag accuracy of 91% for fetal loss and 87% for congenital anomalies outperforming null models. Fifty-seven 'category C' medications were classified as harmful for fetal loss and eleven for congenital anomalies. This includes medications with documented harmful effects, including naproxen, ibuprofen and rubella live vaccine. We also identified several novel drugs, e.g., haloperidol, that increased the risk of fetal loss. Our approach provides important information on the harmfulness of 'category C' drugs. This is needed, as no FDA recommendation exists for these drugs' fetal safety.