Evaluative composite measurement scales (CMS) are increasingly used in medicine to measure complex constructs in the absence of a reference criterion or 'gold standard'. However, the level of measurement provided by these instruments has usually been given little attention. This paper explores the influence of the characteristics of the item set and weighting on the level of measurement of an evaluative CMS. The approach is illustrated with an application to the scoring system of the Nottingham Health Profile. Simulations are provided to indicate when the composite score of item responses, on an ordinal or pass-fail scale, could be considered to achieve the properties of an interval scale.