We developed a new high pressure cell which allows us to perform accurate dielectric measurements above 3 GPa. The developed cell is based on Bridgman type opposed anvil cells with electrodes immersed into holes in the anvils in order to make the electrode area constant under compression. As a feasibility experiment, dielectric measurements for H2O were conducted up to 6.6 GPa and obtained dielectric constants are well consistent with previous studies.