A number of trials have evaluated the use of single-agent immune checkpoint inhibitors for the treatment of metastatic castration-resistant prostate cancer (mCRPC). The benefit appears to be limited to a small subset of patients, such as those with tumors with microsatellite instability, highlighting the importance of biomarkers to identify which patients may be more likely to respond. Given the lack of efficacy for most patients with mCRPC, our understanding of the mechanisms of primary resistance to checkpoint inhibitors and of the tumor immune microenvironment in prostate cancer is critical. Knowledge gained in these key areas will allow for the identification of novel combination therapies that will circumvent resistance mechanisms and should be tested in clinical trials. Improving our understanding of the effects of androgen deprivation therapy on immune cells and of the most favorable disease setting (e.g., biochemically recurrent vs. castration-resistant prostate cancer) may aid in the optimal use of checkpoint inhibitors in combination with other agents. If successful, this may move immune checkpoint inhibitors into the treatment armamentarium of prostate cancer management.