The multipolar Hamiltonian of quantum electrodynamics is extensively employed in chemical and optical physics to treat rigorously the interaction of electromagnetic fields with matter. It is also widely used to evaluate intermolecular interactions. The multipolar version of the Hamiltonian is commonly obtained by carrying out a unitary transformation of the Coulomb gauge Hamiltonian that goes by the name of Power-Zienau-Woolley (PZW). Not only does the formulation provide excellent agreement with experiment, and versatility in its predictive ability, but also superior physical insight. Recently, the foundations and validity of the PZW Hamiltonian have been questioned, raising a concern over issues of gauge transformation and invariance, and whether observable quantities obtained from unitarily equivalent Hamiltonians are identical. Here, an in-depth analysis of theoretical foundations clarifies the issues and enables misconceptions to be identified. Claims of non-physicality are refuted: the PZW transformation and ensuing Hamiltonian are shown to rest on solid physical principles and secure theoretical ground.