Artificial synaptic devices implemented by emerging post-CMOS non-volatile memory technologies such as Resistive RAM (RRAM) have made great progress recently. However, it is still a big challenge to fabricate stable and controllable multilevel RRAM. Benefitting from the control of electron spin instead of electron charge, spintronic devices, e.g., magnetic tunnel junction (MTJ) as a binary device, have been explored for neuromorphic computing with low power dissipation. In this paper, a compound spintronic device consisting of multiple vertically stacked MTJs is proposed to jointly behave as a synaptic device, termed as compound spintronic synapse (CSS). Based on our theoretical and experimental work, it has been demonstrated that the proposed compound spintronic device can achieve designable and stable multiple resistance states by interfacial and materials engineering of its components. Additionally, a compound spintronic neuron (CSN) circuit based on the proposed compound spintronic device is presented, enabling a multi-step transfer function. Then, an All Spin Artificial Neural Network (ASANN) is constructed with the CSS and CSN circuit. By conducting system-level simulations on the MNIST database for handwritten digital recognition, the performance of such ASANN has been investigated. Moreover, the impact of the resolution of both the CSS and CSN and device variation on the system performance are discussed in this work.