A pair of orthogonal tilting prisms has been explored in our previous work to perform the orientation and position tracking function with tracking accuracy better than submicroradian order. Crucial to the function implementation, however, is the real-time nonlinear control of the tilting angles of double prisms for tracking a given target trajectory. In previous papers [Proc. SPIE5892, 1-5 (2005).PSISDG0277-786X; Appl. Opt.45, 8063 (2006).PSISDG0277-786X; Proc. SPIE6709, 41 (2007).PSISDG0277-786X; Appl. Opt.51, 356 (2011).10.1364/AO.51.000356APOPAI1559-128X; Appl. Opt.53, 3712 (2014).10.1364/AO.53.003712APOPAI1559-128X], a new driving method by a cam-based mechanism, which can transfer the control problem to the design of corresponding cam configuration, is investigated. The design process of a cam-based mechanism is explained from the mapping relation between the tilting angles of a prism and the configuration curve of a corresponding cam. Based on the designed cam-based mechanism, a tracking error less than 0.375% is depicted between the tracking trajectory and the original one. Moreover, the dynamic characteristic of the tracking mechanism is discussed in detail as well as the impacts of different tilting speeds on the tracking trajectory. The proposed tracking mechanism of a tilting double-prism scanner can create a new avenue for passively tracking a given target.