Learning-based phase imaging balances high fidelity and speed. However, supervised training requires unmistakable and large-scale datasets, which are often hard or impossible to obtain. Here, we propose an architecture for real-time phase imaging based on physics-enhanced network and equivariance (PEPI). The measurement consistency and equivariant consistency of physical diffraction images are used to optimize the network parameters and invert the process from a single diffraction pattern. In addition, we propose a regularization method based total variation kernel (TV-K) function constraint to output more texture details and high-frequency information. The results show that PEPI can produce the object phase quickly and accurately, and the proposed learning strategy performs closely to the fully supervised method in the evaluation function. Moreover, the PEPI solution can handle high-frequency details better than the fully supervised method. The reconstruction results validate the robustness and generalization ability of the proposed method. Specially, our results show that PEPI leads to considerable performance improvement on the imaging inverse problem, thereby paving the way for high-precision unsupervised phase imaging.