Acid sphingomyelinase (ASM), which regulates sphingolipid metabolism and lipid signaling, has been considered as a new potential target for the treatment of atherosclerosis. In this study, a series of benzene-heterocyclic-based ASM inhibitors were rationally designed, synthesized, and screened for the first time. As a result, some compounds showed favorable inhibitory activity against recombinant human ASM. The detailed SARs are also discussed. Compound 4i revealed good pharmacokinetic data and in vivo inhibitory activity against ASM by reducing the level of ceramide in mice plasma and liver. Pharmacodynamic studies confirmed that 4i could lessen lipid plaques in the aortic arch and aorta and reduce plasma ceramide concentration and Ox-LDL levels. Moreover, 4i was found to significantly decrease LPS-induced and Ox-LDL-induced cell inflammation by regulating the levels of ceramide and sphingomyelin. Overall, this study preliminarily demonstrates that ASM may be an effective target against atherosclerosis for the first time.