In computational drug discovery, ranking a series of compound analogues in the order that is consistent with the experimental binding affinities remains a challenge. Many of the computational methods available for evaluating binding affinities have adopted molecular mechanics (MM)-based force fields, although they cannot completely describe protein-ligand interactions. By contrast, quantum mechanics (QM) calculations play an important role in understanding the protein-ligand interactions; however, their huge computational costs hinder their application in drug discovery. In this study, we have evaluated the ability to rank the binding affinities of tankyrase 2 ligands by combining both MM and QM calculations. Our computational approach uses the protein-ligand binding energies obtained from a cost-effective multilayer fragment molecular orbital (MFMO) method combined with the solvation energy obtained from the MM-Poisson-Boltzmann/surface area (MM-PB/SA) method to predict the binding affinity. This approach enabled us to rank tankyrase 2 inhibitor analogues, outperforming several MM-based methods, including rescoring by molecular docking and the MM-PB/SA method alone. Our results show that this computational approach using the MFMO method is a promising tool for predicting the rank order of the binding affinities of inhibitor analogues.