Fast evolution of modern society stimulates intense development of new materials with novel functionalities in energy and environmental applications. Due to rapid progress of computer science, computational design of materials with target properties has recently attracted a lot of interest. Accurate and efficient calculation of fundamental thermodynamic properties, including redox potentials, acidity constants, and solvation free energies, is of great importance for selection and design of desirable materials. Free energy calculation based on ab initio molecular dynamics (AIMD) can predict these properties with high accuracy at complex environments, however, they are being impeded by high computational costs. To address this issue, this work develops an automated scheme that combines iterative training of machine learning potentials (MLPs) and free energy calculation and demonstrates that these thermodynamic properties can be computed by ML accelerated MD with ab initio accuracy and a much longer time scale at cheaper costs, improving poor statistics and convergence of numerical integration by AIMD. Our automated scheme lays the foundation for computational chemistry-assisted materials design.