Polarization singularities have attracted considerable interest in photonic crystals (PhCs) due to their high quality factor and selective response to circularly polarized light. In this work, we propose a strategy to manipulate the evolution of polarization singularities within degenerate bands in the magneto-optical PhC slab, by breaking geometric symmetry through unit structures and time-reversal symmetry through magnetic field. For C6 symmetry, the magnetic field lifts the degeneracy of bound states in the continuum (BICs), resulting in chiral BICs with opposite chirality. Reducing the symmetry to C3, an at-Γ parabolic point and six off-Γ circularly polarized points (C-points) are observed. Applying the magnetic field, the parabolic point splits into four new C-points that subsequently undergo pairwise merging and annihilation with the pre-existing ones as the field strength increases, illustrating rich topological dynamics. Upon reduction to C2 symmetry, an at-Γ BIC and two off-Γ Dirac points are observed. Under the external magnetic field, the Dirac points evolve into two couples of C-points, where the surrounding highly circularly polarized states are simultaneously controlled by the field strength. In addition, flexible and on-demand modulation of the polarization singularities can be achieved by rotating the unit structure to break mirror symmetry and changing the magnetic field. Our work establishes the tuning approach of polarization singularities based on multiple symmetry-breaking, which enables effective chiroptical manipulation.