Advances in harnessing the immune system for cancer treatment have been spectacular in recent years as witnessed by the approval of a number of antibodies targeting the PD-1/PD-L1 immune checkpoint pathway spanning an expanding list of indications. However, it is well recognized that while these antibodies show impressive clinical activity, they suffer from shortcomings including the failure to show response in a majority of patients, their need to be administered by intravenous injection, and immune-related adverse events due to the breaking of immune self-tolerance. Small-molecule-based therapeutic approaches offer the potential to address the shortcomings of these antibody-based checkpoint inhibitors. In the first part of this review, we discuss the rationale for small-molecule-based checkpoint therapy followed by efforts on the discovery of small-molecule-based approaches targeting the PD-1/PD-L1 axis and other immune checkpoint pathways. In the latter part of the article, we describe small-molecule inhibitors simultaneously targeting two non-redundant checkpoint inhibitor pathways as an approach to improve the response rate. A brief review of the progress of an oral small-molecule checkpoint inhibitor currently in clinical development is presented at the end.