This study investigates X-ray intensity and dispersion around handheld X-ray fluorescence (XRF) instruments during the measurement of a range of sample matrices to establish radiation exposure risk during operation. Four handheld XRF instruments representing three manufacturers were used on four smooth, flat-lying materials of contrasting matrix composition. Dose rates were measured at 10, 20, 30, and 40 cm intervals every 30° around the instrument at 0 and 45° from the horizontal, as well as vertically from the instrument screen. The analysis of polyethylene recorded dose rates 156 times higher (on average) than steel measurements and 34 times higher than both quartz sand and quartz sandstone. A worst-case exposure scenario was assumed where a user analyses a polyethylene material at arms reach for 1 h each working day for one year. This scenario resulted in an effective body dose of 73.5 μSv, equivalent to three to four chest X-rays (20 μSv) a year, 20 times lower than the average annual background radiation exposure in Australia and well below the annual exposure limit of 1 mSv for non-radiation workers. This study finds the advantages of using handheld XRF spectrometers far outweighs the risk of low radiation exposure linked to X-ray scattering from samples.