We study analytically, numerically, and experimentally the nonlinear symmetry breaking induced by broken reflection symmetry in an optical fiber system. In particular, we investigate the modulation instability regime and reveal the key role of the third-order dispersion on the asymmetry in the spectrum of the dissipative structures. Our theory explains early observations, and the predictions are in excellent agreement with our experimental findings.