The criteria for entering medicines in repertory rubrics are unclear and partly incorrect. A new repertory should be based on clear and objective criteria. Retrospective and prospective assessment of medicines and symptoms by the Dutch Committee for Methods and Validation gives an indication of the validity of existing repertory entries. Reliance on the experience of one expert is the cause of part of the shortcomings of the repertory. This experience is highly influenced by chance. Another part is due to the use of absolute rather than relative occurrence of symptoms. Yet another part is caused by not comparing prevalence in the population 'cured' by a medicine with the remainder of the population. In many cases we need better definitions of symptoms. A clear protocol and prospective research could overcome many shortcomings of the repertory. Statistics help to get more objective criteria, but we still need to reach consensus about how to handle probabilities of outcomes of our assessments.