The success of germination, growth and final yield of every crop depends to a large extent on the quality of the seeds used to grow the crop. Seed quality is defined as the viability and vigor attribute of a seed that enables the emergence and establishment of normal seedlings under a wide range of environments. We attempt to dissect the mechanisms involved in the acquisition of seed quality, through a combined approach of physiology and genetics. To achieve this goal we explored the genetic variation found in a RIL population of Solanum lycopersicum (cv. Moneymaker) x Solanum pimpinellifolium through extensive phenotyping of seed and seedling traits under both normal and nutrient stress conditions and root system architecture (RSA) traits under optimal conditions. We have identified 62 major QTLs on 21 different positions for seed, seedling and RSA traits in this population. We identified QTLs that were common across both conditions, as well as specific to stress conditions. Most of the QTLs identified for seedling traits co-located with seed size and seed weight QTLs and the positive alleles were mostly contributed by the S. lycopersicum parent. Co-location of QTLs for different traits might suggest that the same locus has pleiotropic effects on multiple traits due to a common mechanistic basis. We show that seed weight has a strong effect on seedling vigor and these results are of great importance for the isolation of the corresponding genes and elucidation of the underlying mechanisms.