Assessment of permeability is a critical step in the drug development process for selection of drug candidates with favorable ADME properties. We have developed a novel physics-based method for fast computational modeling of passive permeation of diverse classes of molecules across lipid membranes. The method is based on heterogeneous solubility-diffusion theory and operates with all-atom 3D structures of solutes and the anisotropic solvent model of the lipid bilayer characterized by transbilayer profiles of dielectric and hydrogen bonding capacity parameters. The optimal translocation pathway of a solute is determined by moving an ensemble of representative conformations of the molecule through the dioleoyl-phosphatidylcholine (DOPC) bilayer and optimizing their rotational orientations in every point of the transmembrane trajectory. The method calculates (1) the membrane-bound state of the solute molecule; (2) free energy profile of the solute along the permeation pathway; and (3) the permeability coefficient obtained by integration over the transbilayer energy profile and assuming a constant size-dependent diffusivity along the membrane normal. The accuracy of the predictions was evaluated against experimental permeability coefficients measured in pure lipid membranes (for 78 compounds, R2 was 0.88 and rmse was 1.15 log units), PAMPA-DS (for 280 compounds, R2 was 0.75 and rmse was 1.59 log units), BBB (for 182 compounds, R2 was 0.69 and rmse was 0.87 log units), and Caco-2/MDCK assays (for 165 compounds, R2 was 0.52 and rmse was 0.89 log units).