We present a comprehensive, theoretical treatment of the orientational response to external stress of active, contractile cells embedded in a gel-like elastic medium. The theory includes both the forces that arise from the deformation of the matrix as well as forces due to the internal regulation of the stress fibers and focal adhesions of the cell. We calculate the time-dependent response of both the magnitude and the direction of the elastic dipole that characterizes the active forces exerted by the cell, for various situations. For static or quasistatic external stress, cells orient parallel to the stress while for high frequency dynamic external stress, cells orient nearly perpendicular. Both numerical and analytical calculations of these effects are presented. In addition we predict the relaxation time for the cellular response for both slowly and rapidly varying external stresses; several characteristic scaling regimes for the relaxation time as a function of applied frequency are predicted. We also treat the case of cells for which the regulation of the stress fibers and focal adhesions is controlled by strain (instead of stress) and show that the predicted dependence of the cellular orientation on the Poisson ratio of the matrix can differentiate strain vs stress regulation of cellular response.