There has been a recent burst of activity in the atmosphere/ocean sciences community in utilizing stable linear Langevin stochastic models for the unresolved degree of freedom in stochastic climate prediction. Here several idealized models for stochastic climate modeling are introduced and analyzed through unambiguous mathematical theory. This analysis demonstrates the potential need for more sophisticated models beyond stable linear Langevin equations. The new phenomena include the emergence of both unstable linear Langevin stochastic models for the climate mean and the need to incorporate both suitable nonlinear effects and multiplicative noise in stochastic models under appropriate circumstances. The strategy for stochastic climate modeling that emerges from this analysis is illustrated on an idealized example involving truncated barotropic flow on a beta-plane with topography and a mean flow. In this example, the effect of the original 57 degrees of freedom is well represented by a theoretically predicted stochastic model with only 3 degrees of freedom.