The magnetic skyrmion is a nanoscale topological object characterized by the winding of magnetic moments, appearing in magnetic materials with broken inversion symmetry. Because of its low current threshold for driving the skyrmion motion, they have been intensely studied toward novel storage applications by using electron-beam, X-ray, and visible light microscopies. Here, we demonstrate another imaging method for skyrmions by using spin-caloritronic phenomena, that is, the spin Seebeck and anomalous Nernst effects, as a probe of magnetic texture. We scanned a focused heating spot on a Hall-cross shaped MgO/CoFeB/Ta/W multilayer film and mapped the magnitude as well as the direction of the resultant thermoelectric current due to the spin-caloritronic phenomena. Our experimental and calculation reveal that the characteristic patterns in the thermoelectric signal distribution reflect the skyrmions' magnetic texture. The thermoelectric microscopy will be a complementary and useful imaging technique for the development of skyrmion devices owing to the unique symmetry of the spin-caloritronic phenomena.