We developed an efficient computation scheme for the phase-field simulation of grain growth, which allows unlimited number of the orientation variables and high computational efficiency independent of them. Large-scale phase-field simulations of the ideal grain growth in two-dimensions (2D) and three-dimensions (3D) were carried out with holding the coalescence-free condition, where a few tens of thousands grains evolved into a few thousand grains. By checking the validity of the von Neumann-Mullins law for individual grains, it could be shown that the present simulations were correctly carried out under the conditions of the ideal grain growth. The steady-state grain size distribution in 2D appeared as a symmetrical shape with a plateau slightly inclined to the small grain side, which was quite different from the Hillert 2D distribution. The existence of the plateau stems from the wide separation of the peaks in the size distributions of the grains with five, six, and seven sides. The steady-state grain size distribution in 3D simulation of the ideal grain growth appeared to be very close to the Hillert 3D distribution, independent of the initial average grain size and size distribution. The mean-field assumption, the Lifshitz-Slyozov stability condition, and all resulting predictions in the Hillert 3D theory were in excellent agreement with the present 3D simulation. Thus the Hillert theory can be regarded as an accurate description for the 3D ideal grain growth. The dependence of the growth rate in 3D simulations on the grain topology were discussed. The large-scale phase-field simulation confirms the 3D growth law obtained from the Surface Evolver simulations in smaller scales.