Functional mapping has emerged as a next-generation statistical tool for mapping quantitative trait loci (QTL) that affect complex dynamic traits. In this article, we incorporated the idea of nonlinear mixed-effect (NLME) models into the mixture-based framework of functional mapping, aimed to generalize the spectrum of applications for functional mapping. NLME-based functional mapping, implemented with the linearization algorithm based on the first-order Taylor expansion, can provide reasonable estimates of QTL genotypic-specific curve parameters (fixed effect) and the between-individual variation of these parameters (random effect). Results from simulation studies suggest that the NLME-based model is more general than traditional functional mapping. The new model can be useful for the identification of the ontogenetic patterns of QTL genetic effects during time course.