The first Kelvin relation, which states the Peltier coefficient should be equal to the product of temperature and Seebeck coefficient, is a fundamental principle in thermoelectricity. It has been regarded as an important application and direct experimental verification of the Onsager reciprocal relation (ORR) that is a cornerstone of irreversible thermodynamics. However, some critical questions still remain: (a) why Kelvin's proof-which omits all irreversibility within a thermoelectric transport process-can reach the correct result, (b) how to properly select the generalized-force-flux pairs for deriving the first Kelvin relation from the ORR, and (c) whether the first Kelvin relation is restricted by the requirement of the linear transport regime. The aim of the present work is to answer these questions based on the fundamental thermodynamic principles. Since the thermoelectric effects are reversible, we can redefine the Seebeck and Peltier coefficients using the quantities in reversible processes with no time derivative involved; these are renamed "reversible Seebeck and Peltier coefficients." The relation between them (called "the reversible reciprocal relation of thermoelectricity") is derived from the Maxwell relations, which can be reduced to the conventional Kelvin relation, when the local equilibrium assumption (LEA) is adopted. In this sense, the validity of the first Kelvin relation is guaranteed by the reversible thermodynamic principles and the LEA, without the requirement of the linear transport process. Additionally, the generalized force-flux pairs to obtain the first Kelvin relation from the ORR can be proper both mathematically and thermodynamically, only when they correspond to the conjugate-variable pairs of which Maxwell relations can yield the reversible reciprocal relation. The present theoretical framework can be further extended to other coupled phenomena.