We have constructed a general thermodynamic model of chemical potentials and applied ab initio electronic structure and molecular dynamics simulations, as well as kinetic Monte Carlo analysis, to probe the dynamical, reactive, and kinetic aspects of metal single-atom catalysts (SACs) on oxide support. We choose Au single atoms (SAs) supported on ceria as a typical example to demonstrate how our model can guide the rational design of highly stable and reactive SACs. It is shown that, under realistic conditions, various factors such as temperature, pressure, particle size, and the reducibility of the support can strongly affect both the stability and the reactivity of SACs by altering the relative chemical potentials between SAs and metal nanoparticles (NPs). The Au SAs at step sites of ceria support are rather stable, even at temperatures as high as 700 K, and exhibit around 10 orders of magnitude more reactivity for CO oxidation than the terrace sites. Remarkably, under reaction conditions, Au SAs can be dynamically created at the interface of small-size Au NPs on ceria support even without step sites, which accounts for the puzzling significant size effect in gold catalysis. Our work underscores an unrecognized critical role of Au SAs in gold nanocatalysis and provides a general methodology for designing the metal SACs on oxide supports.