Glycogen synthase kinase 3 (GSK-3) is an attractive target for the treatment of diabetes, and paullones have been reported to be effective inhibitors of GSK-3. However, it is still a challenging task to improve selectivity among protein kinases, especially cyclin-dependent kinases (CDKs). Here we investigated the mechanism that enables paullones to selectively inhibit GSK-3 rather than cyclin-dependent kinase 5 (CDK5) using sequence alignment, molecular dynamics simulations, free-energy calculations and free-energy decomposition analysis. The results indicate that the interaction between paullones and Val135 of GSK-3 is obviously stronger than that between paullones and Cys83 of CDK5, suggesting that paullones could be utilized as potent selective inhibitors. Meanwhile, we observed that the decrease in the interaction between paullones and the Asp86 of CDK5 favors their selectivity towards GSK-3 rather than CDK5, as demonstrated using 1-azakenpaullone as an example. Although substitution at position 9 and replacement at position 2 may influence the activity of GSK-3, they only have a minor effect on the selectivity. We expect that the information obtained here could prove useful for developing specific paullone inhibitors of GSK-3.