Atomic clusters of metals are an emerging class of extremely interesting materials occupying the intermediate size regime between atoms and nanoparticles. Here we report the nonlinear optical (NLO) characteristics of ultrasmall, atomically precise clusters of gold, which are smaller than the critical size for electronic energy quantization (∼2 nm). Our studies reveal remarkable features of the distinct evolution of the optical nonlinearity as the clusters progress in size from the nonplasmonic regime to the plasmonic regime. We ascertain that the smallest atomic clusters do not show saturable absorption at the surface plasmon wavelength of larger gold nanocrystals (>2 nm). Consequently, the third-order optical nonlinearity in these ultrasmall gold clusters exhibits a significantly lower threshold for optical power limiting. This limiting efficiency, which is superior to that of plasmonic nanocrystals, is highly beneficial for optical limiting applications.