Artificial neural networks (ANN's) are more and more widely used in energy management processes. ANN's can be very useful in optimizing the energy demand of buildings, especially of those of high thermal inertia. These include the so-called solar buildings. For those buildings, a controller able to forecast not only the energy demand but also the weather conditions can lead to energy savings while maintaining thermal comfort. In this paper, such an ANN controller is proposed. It consists of a meteorological module, forecasting the ambient temperature and solar irradiance, the heating energy switch predictor module and the indoor temperature-defining module. The performance of the controller has been tested both experimentally and in a building thermal simulation environment. The results showed that the use of the proposed controller can lead to 7.5% annual energy savings in the case of a highly insulated passive solar test cell.