Patients with bone and cartilage defects due to infection, tumors, and trauma are quite common. Repairing bone and cartilage defects is thus a major problem for clinicians. Autologous and artificial bone transplantations are associated with many challenges, such as limited materials and immune rejection. Bone and cartilage regeneration has become a popular research topic. Inorganic polyphosphate (polyP) is a widely occurring biopolymer with high-energy phosphoanhydride bonds that exists in organisms from bacteria to mammals. Much data indicate that polyP acts as a regulator of gene expression in bone and cartilage tissues and exerts morphogenetic effects on cells involved in bone and cartilage formation. Exposure of these cells to polyP leads to the increase of cytokines that promote the differentiation of mesenchymal stem cells into osteoblasts, accelerates the osteoblast mineralization process, and inhibits the differentiation of osteoclast precursors to functionally active osteoclasts. PolyP-based materials have been widely reported in in vivo and in vitro studies. This paper reviews the current cellular mechanisms and material applications of polyP in bone and cartilage regeneration.