We report the optical properties of layered calcium cobaltate, Ca3Co4O9, which is regarded as a promising candidate for use as a thermoelectric material. The optical conductivity shows three broad peaks related to the inter-band transition below 4 eV, which are quite similar to those in the spectra of Na x CoO2. This similarity implies that the CoO2 layer, which is an essential unit for both Ca3Co4O9 and Na x CoO2, is dominant in the energy band structure below 4 eV. In addition, we estimate the effective carrier number per Co site and find similarity between the CoO2 layers of Ca3Co4O9 and Na0.75CoO2, which is consistent with the similarity in their Seebeck coefficients. To discuss the contribution of the rocksalt-type Ca2CoO3 layer in Ca3Co4O9, we propose the concept of optical sheet conductivity in the layered materials and estimate its value in the Ca2CoO3 layer. A comparison with the spin-polarized band calculation of the LDA + Hubbard U formalism with U = 5 eV suggests that the Ca2CoO3 layer has the inter-band transition of 2.6 eV in the spin-down band structure. Evaluation of the valences of Co 3d orbitals indicates the existence of charge transfer from the Ca2CoO3 layer to the CoO2 layer and mixing of Co(3+) and Co(4+) in the CoO2 layer, which may be the origin of the large thermoelectric effect.