F-box protein S-phase kinase-associated protein 2 (Skp2) is a component of cullin-RING ligases, which is responsible for recruiting and ubiquitinating substrates and subsequently plays its proteolytic and non-proteolytic role. High expression of Skp2 is frequently observed in multiple aggressive tumor tissues and associated with poor prognosis. Several of the Skp2 inhibitors have been reported in the last decades; however, few of them have shown detailed structure-activity relationship (SAR) and potent bioactivity. Herein, based on the hit compound 11a from our in-house library, we optimize and synthesize a series of new 2,3-diphenylpyrazine-based inhibitors targeting the Skp2-Cks1 interaction and further systematically study the SAR. Among them, compound 14i shows potent activity against the Skp2-Cks1 interaction with an IC50 value of 2.8 μM and against PC-3 and MGC-803 cells with IC50 values of 4.8 and 7.0 μM, respectively. Most importantly, compound 14i exhibited effectively anticancer effects on PC-3 and MGC-803 xenograft mice models without obvious toxicity.