Vacancy-ordered double perovskites (A2BX6), being one of the environmentally friendly and stable alternatives to lead halide perovskites, have garnered considerable research attention in the scientific community. However, their thermal transport has not been explored much, despite their potential applications. Here, we explore Cs2BI6 (B = Pt, Pd, Te, Sn) as potential thermoelectric materials using state-of-the-art first-principles-based methodologies, viz., density functional theory combined with many-body perturbation theory (G0W0) and spin-orbit coupling. The absence of polyhedral connectivity in vacancy-ordered perovskites gives rise to additional degrees of freedom, leading to lattice anharmonicity. The presence of anharmonic lattice dynamics leads to strong electron-phonon coupling, which is well-captured by the Fröhlich mesoscopic model. The lattice anharmonicity is further studied using ab initio molecular dynamics and the electron localization function. The maximum anharmonicity is observed in Cs2PtI6, followed by Cs2PdI6, Cs2TeI6, and Cs2SnI6. Also, the computed average thermoelectric figure of merit (zT) for Cs2PtI6, Cs2PdI6, Cs2TeI6, and Cs2SnI6 is 0.88, 0.85, 0.95, and 0.78, respectively, which reveals their promising renewable energy applications.