Background Deep learning-based segmentation could facilitate rapid and reproducible T1 lesion load assessments, which is crucial for disease management in multiple sclerosis (MS). T1 unenhancing and contrast-enhancing lesions in MS are those that enhance or do not enhance after administration of a gadolinium-based contrast agent at T1-weighted MRI. Purpose To develop deep learning models for automated assessment of T1 unenhancing and contrast-enhancing lesions; to investigate if joint training improved performance; to reproduce a known ocrelizumab treatment response; and to evaluate the association of baseline T1-weighted imaging metrics with clinical outcomes in relapsing MS clinical trials. Materials and Methods Joint and individual deep learning models (U-Nets) were developed retrospectively on multimodal MRI data sets from large multicenter OPERA trials of relapsing MS (August 2011 to May 2015). The joint model included cross-network connections and a combined loss function. Models were trained on OPERA I data sets with three-fold cross-validation. OPERA II data sets were the internal test set. Dice coefficients, lesion true-positive and false-positive rates, and areas under the receiver operating characteristic curve (AUCs) were used to evaluate model performance. Association of baseline imaging metrics with clinical outcomes was assessed with Cox proportional hazards models. Results A total of 796 patients (3030 visits; mean age, 37 years ± 9; 521 women) from the OPERA II trial were evaluated. The joint model achieved a mean Dice coefficient of 0.77 and 0.74, lesion true-positive rate of 0.88 and 0.86, and lesion false-positive rate of 0.04 and 0.19 for T1 contrast-enhancing and T1 unenhancing lesion segmentation, respectively. Joint training improved performance for smaller T1 contrast-enhancing lesions (≤0.06 mL; individual training AUC: 0.75; joint training AUC: 0.87; P < .001). A significant ocrelizumab treatment effect (P < .001) was seen in reducing the mean number of T1 contrast-enhancing lesions at 24, 48, and 96 weeks (manual assessment at 24 weeks: 10 lesions in 366 patients with ocrelizumab, 141 lesions in 355 patients with interferon, 93% reduction; manual assessment at 48 weeks: six lesions in 355 patients with ocrelizumab, 150 lesions in 317 patients with interferon, 96% reduction; manual assessment at 96 weeks: five lesions in 340 patients with ocrelizumab, 157 lesions in 294 patients with interferon, 97% reduction; joint model assessment at 24 weeks: 19 lesions in 365 patients with ocrelizumab, 128 lesions in 354 patients with interferon, 86% reduction; joint model assessment at 48 weeks: 14 lesions in 355 patients with ocrelizumab, 121 lesions in 317 patients with interferon, 90% reduction; joint model assessment at 96 weeks: 10 lesions in 340 patients with ocrelizumab, 144 lesions in 294 patients with interferon, 94% reduction) and the mean number of new T1 unenhancing lesions across all follow-up examinations (manual assessment: 504 lesions in 1060 visits for ocrelizumab-treated patients, 1438 lesions in 965 visits for interferon-treated patients, 68% reduction; joint model assessment: 205 lesions in 1053 visits for ocrelizumab-treated patients, 661 lesions in 957 visits for interferon-treated patients, 78% reduction). Baseline T1 unenhancing total lesion volume was associated with clinical outcomes (manual hazard ratio [HR]: 1.12, P = .02; joint model HR: 1.11, P = .03). Conclusion Joint architecture and training improved segmentation of MRI T1 contrast-enhancing multiple sclerosis lesions, and both deep learning models had sufficiently high performance to detect an ocrelizumab treatment response consistent with manual assessments. ClinicalTrials.gov: NCT01247324 and NCT01412333 © RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Talbott in this issue.