Geometry, an ancient yet vibrant branch of mathematics, has important and far-reaching impacts on various disciplines such as art, science, and engineering. Here, we introduce an emerging concept dubbed "geometric deep optical sensing" that is based on a number of recent demonstrations in advanced optical sensing and imaging, in which a reconfigurable sensor (or an array thereof) can directly decipher the rich information of an unknown incident light beam, including its intensity, spectrum, polarization, spatial features, and possibly angular momentum. We present the physical, mathematical, and engineering foundations of this concept, with particular emphases on the roles of classical and quantum geometry and deep neural networks. Furthermore, we discuss the new opportunities that this emerging scheme can enable and the challenges associated with future developments.