We describe analytical and numerical models to study gamete encounters in two and three dimensions between differently sized gametes without assuming pre-existing mating types. Our results indicate that an isogamous population can be successfully invaded by a gametangium if it produces gametes of a different size. The existence of a low adaptive peak for isogamy and a much higher adaptive peak for anisogamy suggests that stochastic forces may be initially important in driving isogamy through the fitness saddle to anisogamy.