Gestational age (GA) is thought to affect height growth in small-for-gestational age (SGA) children. However, the GA-specific trajectories in body mass index (BMI) and early appearances of adiposity rebound (AR) have not been fully investigated in a cohort of Japanese SGA children. A longitudinal cohort study was conducted with 1063 SGA children born in Kobe, Japan, with sufficient records from birth to 3 years of age. Subjects were divided into subgroups based on GA: 39-41 weeks GA (n = 723), 37-38 weeks GA (n = 256), 34-36 weeks GA (n = 62), and <34 weeks GA (n = 22). Height and BMI were assessed at 4 months, 9 months, 1.5 years, and 3 years of age. The catch-up rate for height was GA-dependent. Most children with 39-41 weeks GA (91%) caught up by 4 months of age; however, lower GA was associated with a slower elevation in the catch-up rate. The BMI trajectory during the first 3 years was also GA-dependent, with a change in GA dependency at a boundary of 37 weeks GA. Approximately 7% of SGA children had already developed AR before 3 years of age. In conclusion, growth patterns during infancy and early childhood in SGA children differ depending on GA.