Neonatal images have low spatial resolution and insufficient tissue contrast. Generally, interpolation methods are used to upsample neonatal images to a higher resolution for more effective image analysis. However, the resulting images are often blurry and are susceptible to partial volume effect. In this paper, we propose an algorithm that utilizes longitudinal prior information for effective super-resolution reconstruction of neonatal images. We use a non-local approach to learn the spatial relationships of brain structures in high-resolution longitudinal images and apply this information to the super-resolution reconstruction of the neonatal image. In other words, the recurring patterns throughout the longitudinal scans are leveraged for reconstructing the neonatal image with high resolution. To solve this otherwise ill-posed inverse problem, low-rank and total-variation regularizations are enforced. Experiments performed on both T1- and T2-weighted MR images of 28 neonates demonstrate that the proposed method is capable of recovering more structural details and outperforms methods such as nearest neighbor interpolation, spline-based interpolation, non-local means upsampling, and both low-rank and total variation based super-resolution.