From randomized controlled trials to evidence grading schemes: current state of evidence-based practice in social sciences

Abstract

With the advance of web search and navigation technology, enormous amount of information, non-information, and misinformation may be obtained in milliseconds in response to questions about 'what works' in social sciences. Today, policy makers in non-medical public service arenas are under increasing pressure to make sound decisions based on scientific evidence. Some of these decisions are a matter of legal requirement. This paper shows how such movements are closely aligned with the evolution of organizations that develop and apply evidence standards and evidence grading schemes within the social science communities. The current state of evidence-based practice in social sciences is examined by reviewing the latest development of randomized trials and evidence grading schemes in the fields of education, criminal justice, and social welfare. Studies conducted under the auspices of the Campbell Collaboration and What Works Clearinghouse are used to illustrate ingredients of evidence grading schemes, graphic display of results of systematic reviews, and discrepancies of evidence derived from randomized trials and non-experimental trials. Furthermore, it is argued that the use of evidence on 'what works' depends on the potential users' awareness, understanding of the evidence, as well as their capacity and willingness to use it. Awareness and understanding depends on the world wide web and its augmentations, while capacity and willingness depends more on incentives to use good evidence and on political and ethical values. Implications for the future development of evidence grading organizations are discussed.