Memristive devices are promising candidates for the next generation non-volatile memory and neuromorphic computing. It has been widely accepted that the motion of oxygen anions leads to the resistance changes for valence-change-memory (VCM) type of materials. Only very recently it was speculated that metal cations could also play an important role, but no direct physical characterizations have been reported yet. Here we report a Ta/HfO2/Pt memristor with fast switching speed, record high endurance (120 billion cycles) and reliable retention. We programmed the device to 24 discrete resistance levels, and also demonstrated over a million (2(20)) epochs of potentiation and depression, suggesting that our devices can be used for both multi-level non-volatile memory and neuromorphic computing applications. More importantly, we directly observed a sub-10 nm Ta-rich and O-deficient conduction channel within the HfO2 layer that is responsible for the switching. This work deepens our understanding of the resistance switching mechanism behind oxide-based memristive devices and paves the way for further device performance optimization for a broad spectrum of applications.