The phase measurement deflectometry considering the refraction effect is presented to measure the mirror surface in this paper. In the context of the conventional phase measurement deflectometry, the biplanar structure of the system constructed by spatial multiplexing of a screen or a half mirror with two screens is a compromise of traditional display technology, while they suffer from complex calibration process and low accuracy. To improve the system compactness and efficiency, a novel measurement model consisting of a transparent screen and an ordinary screen is used to determine the incident light. To compensate for the measurement errors caused by transparent screen refraction, the refraction of the transparent screen is characterized by two physical parameters, which can be calibrated thanks to the multi-stereo vision technique. Then, the improved mirror calibration method with the refraction model is proposed to determine the posed relationship of the system. After that, the three-dimensional (3D) information of mirror surface is restored by the radial basis function interpolation with the optimized refraction parameters and posed relationships from the gradient data which is transformed from the normal information. Higher measurement efficiency, higher measurement accuracy and more compactness of the proposed measurement method are verified by the experimental results.