Background: Projects are the main financing mechanism within the EU community action programme for public health. This article reports the process of establishing and validating evaluation criteria for projects submitted for funding within this programme, to ensure that projects selected for funding conform with quality standards.
Methods: An evaluation checklist was developed, drawing on project management and health promotion literature, to Score 3 aspects of project quality: policy and contextual relevance (five criteria, scores 0-8), technical quality (five criteria, scored 0-6) and management quality (six criteria, scored 0-5). Teams of three people made consensus ratings with the checklist on each of 215 eligible applications submitted in response to Calls for Proposals in 2007 and 151 submitted in 2008. Construct validity, internal consistency and predictive validity were assessed on the grouped consensus ratings using psychometric test statistical methods.
Results: Principal component analyses on both the 2007 and 2008 data gave a three component solution, which largely coincides with the dimensions of contextual relevance, technical quality and management quality. Reliability analyses show high Cronbach α's (>0.86) for each of the three scales. Discriminant analyses indicate that all three of the dimensions contributed to the decision to fund a project. Over the 2 years, innovation, content specification, EU added value and geographical coverage contributing most consistently to the success of an application.
Conclusion: The study shows the successful development and validation of criteria to evaluate EU health project grant proposals.