Organ motion is a major problem for any dynamic radiotherapy delivery technique, and is particularly so for spot scanned proton therapy. On the other hand, the use of narrow, magnetically deflected proton pencil beams is potentially an ideal delivery technique for tracking tumour motion on-line. At PSI, our new Gantry is equipped with a Beams Eye View (BEV) imaging system which will be able to acquire 2D x-ray images in fluoroscopy mode during treatment delivery. However, besides precisely tracking motion from BEVs, it is also essential to obtain information on the 3D motion vector throughout the whole region of interest, and any sparsely acquired surrogate motion is generally not sufficient to describe the deformable behaviour of the whole volume in three dimensions. In this study, we propose a method by which 3D deformable motions can be estimated from surrogate motions obtained using this monoscopic imaging system. The method assumes that example motions over a number of breathing cycles can be acquired before treatment for each patient using 4DMRI. In this study, for each of 11 different subjects, 100 continuous breathing cycles have been extracted from extended 4DMRI studies in the liver and then subject specific motion models have been built using principle component analysis (PCA). To simulate treatment conditions, a different set of 30 continuous breathing cycles from the same subjects have then been used to generate a set of simulated 4DCT data sets (so-called 4DCT(MRI) data sets), from which time-resolved digitally reconstructed radiographs (DRRs) were calculated using the BEV geometry for three treatment fields respectively. From these DRRs, surrogate motions from fiducial markers or the diaphragm have been used as a predictor to estimate 3D motions in the liver region for each subject. The prediction results have been directly compared to the 'ground truth' motions extracted from the same 30 breath cycles of the originating 4DMRI data set. Averaged over all 11 subjects, and for three field directions, for 99% of predicted positions, median (max) error magnitudes of better than 2.63(5.67) mm can be achieved when fiducial markers was chosen as predictor. Furthermore, three single fields, 4D dose calculations have been performed as a verification tool to evaluate the prediction performance of such a model in the context of scanned proton beam therapy. These show a high similarity between plans considering either PCA predicted motion or ground truth motion, where absolute dose differences of more than 5% (V(dosediff = 5%)) occur for the worst field scenarios in only 3.61% (median) or 15.13% (max) of dose calculation points in the irradiated volume. The magnitude of these dose differences were insignificantly dependent on whether surrogate motions were tracked by monoscopic or stereoscopic imaging systems, or whether fiducial markers or diaphragm were chosen as surrogate. This study has demonstrated that on-line deformable motion reconstruction from sparse surrogate motions is feasible, even when using only a monoscopic imaging system. In addition, it has also been found that diaphragm motion can be considered as a good predictor for respiratory deformable liver motion prediction, implying that fiducial markers might not be compulsory if used in conjunction with a patient specific PCA based model.