In the past few decades, gate insulators with a high dielectric constant (high-k dielectric) enabling a physically thick but dielectrically thin insulating layer, have been used to replace traditional SiOx insulator and to ensure continuous downscaling of Si-based transistor technology. However, due to the non-silicon derivative natures of the high-k metal oxides, transport properties in these dielectrics are still limited by various structural defects on the hetero-interfaces and inside the dielectrics. Here, we show that another insulating silicon compound, amorphous silicon nitride (a-Si3N4), is a promising candidate of effective electrical insulator for use as a high-k dielectric. We have examined a-Si3N4 deposited using the plasma-assisted atomic beam deposition (PA-ABD) technique in an ultra-high vacuum (UHV) environment and demonstrated the absence of defect-related luminescence; it was also found that the electronic structure across the a-Si3N4/Si heterojunction approaches the intrinsic limit, which exhibits large band gap energy and valence band offset. We demonstrate that charge transport properties in the metal/a-Si3N4/Si (MNS) structures approach defect-free limits with a large breakdown field and a low leakage current. Using PA-ABD, our results suggest a general strategy to markedly improve the performance of gate dielectric using a nearly defect-free insulator.