Objectives: Financial difficulties experienced by cancer patients may affect their health-related quality of life (HRQoL). This study assessed the direct economic burden that out-of-pocket (OOP) payments cause and explored how they and financial difficulties are associated with HRQoL. Methods: This is a cross-sectional registry and survey study of 1978 cancer patients having either prostate (630), breast (840) or colorectal cancer (508) treated in Finland. The patients were divided into five groups according to the stage of their disease: primary treatment, rehabilitation, remission, metastatic disease and palliative care. The cost data and OOP payments were retrieved from primary and secondary healthcare registries, the Social Insurance Institution of Finland, and a patient questionnaire. HRQoL was measured by 15D, EQ-5D-3L and by EORTC-QLQ-C30. Financial difficulties were evaluated based on patients' self-assessment in the EORTC-QLQ-C30 four-level question about financial difficulties. A path analysis was used to explore the relationship between clinical and demographic factors, HRQoL, OOP payments and financial difficulties. Results: The highest OOP payments were caused by outpatient medication. Total costs and OOP payments were highest in the palliative care group in which the OOP payments consisted mostly of outpatient medication and public sector specialist care. Private sector health care was an important item of OOP payments in the early stages of cancer. Financial difficulties increased together with OOP payments. HRQoL deteriorated the more a person had financial difficulties. In the path analysis, financial difficulties had a major negative direct and total effect on the HRQoL. Factors that attenuated financial difficulties were age, cohabiting and higher education and factors that increased them were OOP payments, total costs of healthcare use, and unemployment. Conclusions: High OOP payments are related to financial difficulties, which have a negative effect on HRQoL. Outpatient medication was a major driver of OOP payments. Among palliative patients, the economic burden was highest and associated with impaired HRQoL.