Kernel weight is an important factor determining grain yield and nutritional quality in sorghum, yet the developmental processes underlying the genotypic differences in potential kernel weight remain unclear. The aim of this study was to determine the stage in development at which genetic effects on potential kernel weight were realized, and to investigate the developmental mechanisms by which potential kernel weight is controlled in sorghum. Kernel development was studied in two field experiments with five genotypes known to differ in kernel weight at maturity. Pre-fertilization floret and ovary development was examined and post-fertilization kernel-filling characteristics were analysed. Large kernels had a higher rate of kernel filling and contained more endosperm cells and starch granules than normal-sized kernels. Genotypic differences in kernel development appeared before stamen primordia initiation in the developing florets, with sessile spikelets of large-seeded genotypes having larger floret apical meristems than normal-seeded genotypes. At anthesis, the ovaries for large-sized kernels were larger in volume, with more cells per layer and more vascular bundles in the ovary wall. Across experiments and genotypes, there was a significant positive correlation between kernel dry weight at maturity and ovary volume at anthesis. Genotypic effects on meristem size, ovary volume, and kernel weight were all consistent with additive genetic control, suggesting that they were causally related. The pre-fertilization genetic control of kernel weight probably operated through the developing pericarp, which is derived from the ovary wall and potentially constrains kernel expansion.