We consider evolutionary game dynamics in a finite population of size N. When mutations are rare, the population is monomorphic most of the time. Occasionally a mutation arises. It can either reach fixation or go extinct. The evolutionary dynamics of the process under small mutation rates can be approximated by an embedded Markov chain on the pure states. Here we analyze how small the mutation rate should be to make the embedded Markov chain a good approximation by calculating the difference between the real stationary distribution and the approximated one. While for a coexistence game, where the best reply to any strategy is the opposite strategy, it is necessary that the mutation rate μ is less than N (-1/2)exp[-N] to ensure that the approximation is good, for all other games, it is sufficient if the mutation rate is smaller than (N ln N)(-1). Our results also hold for a wide class of imitation processes under arbitrary selection intensity.