Extrapolating from a brief survey of the literature, we outline a vision for the future development of time-resolved electron probe instruments that could offer levels of performance and flexibility that push the limits of physical possibility. This includes a discussion of the electron beam parameters (brightness and emittance) that limit performance, the identification of a dimensionless invariant figure of merit for pulsed electron guns (the number of electrons per lateral coherence area, per pulse), and calculations of how this figure of merit determines the trade-off of spatial against temporal resolution for different imaging modes. Modern photonics' ability to control its fundamental particles at the quantum level, while enjoying extreme flexibility and a very large variety of operating modes, is held up as an example and a goal. We argue that this goal may be approached by combining ideas already in the literature, suggesting the need for large-scale collaborative development of next-generation time-resolved instruments.