A huge effort is underway to develop semiconductor nanostructures as low-noise hosts for qubits. The main source of dephasing of an electron spin qubit in a GaAs-based system is the nuclear spin bath. A hole spin may circumvent the nuclear spin noise. In principle, the nuclear spins can be switched off for a pure heavy-hole spin. In practice, it is unknown to what extent this ideal limit can be achieved. A major hindrance is that p-type devices are often far too noisy. We investigate here a single hole spin in an InGaAs quantum dot embedded in a new generation of low-noise p-type device. We measure the hole Zeeman energy in a transverse magnetic field with 10 neV resolution by dark-state spectroscopy as we create a large transverse nuclear spin polarization. The hole hyperfine interaction is highly anisotropic: the transverse coupling is <1% of the longitudinal coupling. For unpolarized, randomly fluctuating nuclei, the ideal heavy-hole limit is achieved down to nanoelectronvolt energies; equivalently dephasing times up to a microsecond. The combination of large and strong optical dipole makes the single hole spin in a GaAs-based device an attractive quantum platform.