Perfect randomness is of great significance in various quantum key distribution (QKD) protocols. In this Letter, we investigate the effect of weak randomness on the state preparation in reference-frame-independent QKD (RFI-QKD), which may be implemented with imperfect random numbers or quantum-state encoding devices. In the scenario of weak randomness, the maximal amount of information the eavesdropper can acquire should be carefully evaluated. With practical experimental parameters, we demonstrate that even a small proportion of weak randomness will impact the security of RFI-QKD seriously. Furthermore, we briefly study the side effect of weak randomness on RFI measurement-device-independent QKD (RFI-MDI-QKD), and simulation results show that weak randomness damages the performance of RFI-MDI-QKD more critically than that of RFI-QKD.