20S proteasome plays a critical role in the regulation of several important cellular processes and has drawn extensive interest in the field of anti-tumor research. Peptide aldehydes can inhibit the 20S proteasome activity by covalently binding to the active site of the beta subunits. In this work, covalent docking in conjunction with molecular dynamics (MD) simulation was used to explore the binding mode of MG132. Two conformations with the lowest docking energy were selected as the representative binding modes. One of the conformations was confirmed as a more reasonable binding mode by molecular dynamics simulations. The binding mode analysis revealed that a space demanding aromatic group with a short linker at the P4 site of the peptide aldehyde inhibitor would form favorable hydrophobic contacts with the neighboring subunit. A bulky substituent at the P2 position would also increase the binding stability by reducing water accessibility of the covalent bond. This study contributed to our understanding of the mechanism and structure-activity relationship of the peptide aldehyde inhibitors and may provide useful information for rational drug design.