The extracellular matrix (ECM) is one of the major components of tumors that plays multiple crucial roles, including mechanical support, modulation of the microenvironment, and a source of signaling molecules. The quantity and cross-linking status of ECM components are major factors determining tissue stiffness. During tumorigenesis, the interplay between cancer cells and the tumor microenvironment (TME) often results in the stiffness of the ECM, leading to aberrant mechanotransduction and further malignant transformation. Therefore, a comprehensive understanding of ECM dysregulation in the TME would contribute to the discovery of promising therapeutic targets for cancer treatment. Herein, we summarized the knowledge concerning the following: (1) major ECM constituents and their functions in both normal and malignant conditions; (2) the interplay between cancer cells and the ECM in the TME; (3) key receptors for mechanotransduction and their alteration during carcinogenesis; and (4) the current therapeutic strategies targeting aberrant ECM for cancer treatment.