Tuberculosis is an infectious disease caused by the bacterium Mycobacterium tuberculosis (Mtb). Mtb protein tyrosine phosphatase B (mPTPB) is a virulence factor required for Mtb survival in host macrophages. Consequently, mPTPB represents an exciting target for tuberculosis treatment. Here, we identified N-phenyl oxamic acid as a highly potent and selective monoacid-based phosphotyrosine mimetic for mPTPB inhibition. SAR studies on the initial hit, compound 4 (IC50 = 257 nM), resulted in several highly potent inhibitors with IC50 values lower than 20 nM for mPTPB. Among them, compound 4t showed a Ki of 2.7 nM for mPTPB with over 4500-fold preference over 25 mammalian PTPs. Kinetic, molecular docking, and site-directed mutagenesis analyses confirmed these compounds as active site-directed reversible inhibitors of mPTPB. These inhibitors can reverse the altered host cell immune responses induced by the bacterial phosphatase. Furthermore, the inhibitors possess molecular weights <400 Da, log D7.4 < 2.5, topological polar surface area < 75, ligand efficiency > 0.43, and good aqueous solubility and metabolic stability, thus offering excellent starting points for further therapeutic development.