To model biochemical systems in which both noise in the chemical reaction process and spatial movement of molecules is important, both the reaction-diffusion master equation (RDME) and Smoluchowski diffusion-limited reaction (SDLR) partial differential equation (PDE) models have been used. In previous work we showed that the solution to the RDME may be interpreted as an asymptotic approximation in the reaction radius to the solution of the SDLR PDE [S. A. Isaacson, SIAM J. Appl. Math. 70, 77 (2009)]. The approximation was shown to be divergent in the limit that the lattice spacing in the RDME approached zero. In this work we expand upon these results for the special case of the two-molecule annihilation reaction, A+B-->Ø. We first introduce a third stochastic reaction-diffusion PDE model that incorporates a pseudopotential based bimolecular reaction mechanism. The solution to the pseudopotential model is then shown to be an asymptotic approximation to the solution of the SDLR PDE for small reaction radii. We next illustrate how the RDME may be obtained by a formal discretization of the pseudopotential model, motivating why the RDME is itself an asymptotic approximation of the SDLR PDE. Finally, we give a more detailed numerical analysis of the difference between solutions to the RDME and SDLR PDE models as a function of both the reaction-radius and the lattice spacing (in the RDME).