Neuroimaging biomarkers play a prominent role for disease diagnosis or tracking neurodegenerative processes. Multiple methods have been proposed by the community to extract robust disease specific markers from various imaging modalities. Evaluating the accuracy and robustness of developed methods is difficult due to the lack of a biologically realistic ground truth. We propose a proof-of-concept method for a patient- and disease-specific brain neurodegeneration simulator. The proposed scheme, based on longitudinal multi-modal data, has been applied to a population of normal controls and patients diagnosed with Alzheimer's disease or frontotemporal dementia. We simulated follow-up images from baseline scans and compared them to real repeat images. Additionally, simulated maps of volume change are generated, which can be compared to maps estimated from real longitudinal data. The results indicate that the proposed simulator reproduces realistic patient-specific patterns of longitudinal brain change for the given populations.