One of the most puzzling problems of high temperature cuprate superconductor is the pseudogap phase (PG) at temperatures above the superconducting transition temperature in the underdoped regime. The PG phase is found by the angle-resolved photoemission spectra (ARPES) to have a gap at some regions in momentum space and a fraction of Fermi surface remained, known as Fermi arcs. The arc turns into a d-wave SC gap with a node below the SC transition temperature. Here, by studying a strongly correlated model at low temperatures, we obtained a phase characterized by two kinds of pairing order parameters with the total momentum of the Cooper pair to be zero and finite. The finite momentum pairing is accompanied with a spatial modulation of pairing order, i.e. a pair density wave (PDW). These PDW phases are intertwined with modulations of charge density and intra-unit cell form factors. The coexistence of the two different pairing orders provides the unique two-gaps like spectra observed by ARPES for superconducting cuprates. As temperature raises, the zero-momentum pairing order vanishes while the finite momentum pairing orders are kept, thus Fermi arcs are realized. The calculated quasiparticle spectra have the similar doping and temperature dependence as reported by ARPES and scanning tunneling spectroscopy (STS). The consequence of breaking symmetry between x and y due to the unidirectional PDW and the possibility to probe such a PDW state in the PG phase is discussed.