In an epidemic, individuals can widely differ in the way they spread the infection depending on their age or on the number of days they have been infected for. In the absence of pharmaceutical interventions such as a vaccine or treatment, non-pharmaceutical interventions (e.g. physical or social distancing) are essential to mitigate the pandemic. We develop an original approach to identify the optimal age-stratified control strategy to implement as a function of the time since the onset of the epidemic. This is based on a model with a double continuous structure in terms of host age and time since infection. By applying optimal control theory to this model, we identify a solution that minimizes deaths and costs associated with the implementation of the control strategy itself. We also implement this strategy for three countries with contrasted age distributions (Burkina-Faso, France, and Vietnam). Overall, the optimal strategy varies throughout the epidemic, with a more intense control early on, and depending on host age, with a stronger control for the older population, except in the scenario where the cost associated with the control is low. In the latter scenario, we find strong differences across countries because the control extends to the younger population for France and Vietnam 2 to 3 months after the onset of the epidemic, but not for Burkina Faso. Finally, we show that the optimal control strategy strongly outperforms a constant uniform control exerted over the whole population or over its younger fraction. This improved understanding of the effect of age-based control interventions opens new perspectives for the field, especially for age-based contact tracing.