Polymers are known to effectively improve the toughness of inorganic matrices; however, the mechanism at the molecular level is still unclear. In this study, we used molecular dynamics simulations to unravel the effects and mechanisms of different molecular chain lengths of polyacrylic acid (PAA) on toughening calcium silicate hydrate (CSH), which is the basic building block of cement-based materials. Our simulation results indicate that an optimal molecular chain length of polymers contributes to the largest toughening effect on the matrix, leading to up to 60.98% increase in fracture energy. During the uniaxial tensile tests along the x-axis and z-axis direction, the configuration evolution of the PAA molecule determines the toughening effect. As the polymer unfolds and its size matches the defects of CSH, the stress distribution of the system becomes more homogeneous, which favors an increase in toughness. Furthermore, based on our simulation results and a mathematical model, we propose a theory of "strain rate/optimal chain length". This theory suggests that the optimal toughening effect can be achieved when the molecular chain length of the organic component is 1.3-1.5 times the largest defect size of the inorganic matrix. This work provides molecular-scale insights into the toughening mechanisms of an organic/inorganic system and may have practical implications for improving the toughness of cement-based materials.