Exploring quantum spin liquid (QSL) state has both fundamental scientific value and realistic application potential. Recently, α-RuCl3 was experimentally observed to hold in-plane zigzag antiferromagnetic (AFM) order at low temperature, which was further proposed to be proximate to a Kitaev QSL ground state. We have studied the magnetic properties of α-RuCl3 in the framework of electronic structure calculation based on density functional theory (DFT) with Hubbard U correction (DFT+U) and spin-orbit coupling. When the intra-orbital Hubbard interaction U and the inter-orbital Hund's coupling J adopt the commonly accepted values of U = 2.0 eV and J = 0.4 eV, the zigzag AFM order indeed owns the minimum energy, consistent with the experimental observation. More importantly, we find that compared with the ferromagnetic order in the previous theoretical studies, there exist a series of magnetic configurations energetically even closer to the zigzag AFM ground state. The further calculations and analysis indicate that these low-energy magnetic states are closely related to the electronic correlation effect of Ru 4d orbitals. By decreasing U and increasing J with just about 0.2 eV, they become energetically degenerate with the zigzag AFM order, inducing strong magnetic frustration and then yielding a state without long-range magnetic order but with nonzero local moments. Considering the facts that theoretically the pressure usually reduces the intra-orbital Hubbard interaction and meanwhile enhances the inter-orbital Hund's coupling, while experimentally the pressure drives α-RuCl3 into a quantum disordered phase, our results provide a perspective to understand the exotic magnetic behaviors of α-RuCl3.