We present a new method for reconstruction of 4-D cone beam computed tomography from an undersampled set of X-ray projections. The novelty of the proposed method lies in utilizing optical flow based registration to facilitate that each temporal phase is reconstructed from the full set of acquired projections. The reconstruction of each phase thus exhibits limited aliasing despite significant intra-phase undersampling. The method is fully self-contained. Initially an approximate 4-D volume is reconstructed and an inter-phase registration based hereon. A subsequent reconstruction pass integrates the optical flow estimation in a cost function formulation in which the X-ray projections from all temporal phases are considered for the reconstruction of each individual phase. Quantitative and qualitative evaluations were performed through reconstruction of both a numerical phantom and a clinical dataset. The obtained reconstructions are compared to the state-of-the-art alternatives of total variation regularization and prior image constrained compressed sensing. Our studies show that the proposed method is the better overall "compromise" in the depiction of both moving and stationary anatomical structures.