Atomic descriptions of peptide self-assembly are crucial to an understanding of disease-related peptide aggregation and the design of peptide-assembled materials. Obtaining these descriptions through computer simulation is challenging because current force fields, which were not designed for this process and are often unable to describe correctly peptide self-assembly behavior and the sequence dependence. Here, we developed a framework using dipeptide aggregation as a model system to improve force fields for simulations of self-assembly. Aggregation-related structural properties were designed and used to guide the optimization of peptide-peptide and peptide-solvent interactions. With this framework, we developed a self-assembly force field, termed PACE-ASM, by reoptimizing a hybrid-resolution force field that was originally developed for folding simulation. With its applicability in folding simulations, the new PACE was used to simulate the self-assembly of two disease-related short peptides, Aβ16-21 and PHF6, into β-sheet-rich cross-β amyloids. These simulations reproduced the crystal structures of Aβ16-21 and PHF6 amyloids at near-atomic resolution and captured the difference in packing orientations between the two sequences, a task which is challenging even with all-atom force fields. Apart from cross-β amyloids, the self-assembly of emerging helix-rich cross-α amyloids by another peptide PSMα3 can also be correctly described with the new PACE, manifesting the versatility of the force field. We demonstrated that the ability of the PACE-ASM to model peptide self-assembly is based largely on its improved description of peptide-peptide and peptide-solvent interactions. This was achieved with our optimization framework that can readily identify and address the deficiency in describing these interactions.