The requirement of a large number of samplings limits the performance of ghost imaging for moving objects. Conventionally, tracking and imaging of the moving objects are done independently; thus, sequential clear images of the moving target during its evolution are required. In this Letter, we propose to obtain the displacement of the object via cross correlation between sequential unclear rough images. Then, a high-quality image of the moving object can be reconstructed gradually during its evolution. Our method works well for translating and rotating objects.