Problem: Many assessments in medical education involve measuring proficiency in a content area. Thus, proper content development (blueprinting) of tests in this field is of primary importance. Prior efforts to conduct content review as part of assessment development have been time- and resource-intensive, relying on practice analysis and then on linking methods. This monograph explores a "rapid, cost-effective" approach to blueprinting that allows efficient assessment development with rigor. Our investigation seeks to explore an efficient and effective alternate method for creating a content design (blueprint) for medical credentialing and evaluation examinations by focusing directly on assessment requirements.
Approach: We employed a two-phase process to propose a rapid blueprinting method. Phase 1 involved a 1-day direct meeting of content experts/practitioners. Phase 2 involved a corroboration survey sent to a wider group of content experts/practitioners. The rapid blueprinting method was applied to developing eleven blueprints (five for medical specialty certification; five for health professions certification; and one for in-training assessment).
Outcomes: The methods we used resulted in effective, well-balanced, operational examinations that successfully implemented the resulting blueprints in item writing assignments and test development. Assessments resulting from the use of the rapid blueprinting method also generated psychometrically sound inferences from the scores. For example, the assessments resulting from this methodology of test construction had KR-20 reliability coefficients ranging from .87 to .92.
Next steps: This approach leveraged the effectiveness and feasibility of the rapid blueprinting method and demonstrated successful examination designs (blueprints) that are cost- and time-effective. The rapid blueprinting method may be explored for further implementation in local assessment settings beyond medical credentialing examinations.
Keywords: Assessment development; Blueprinting; Content validity; Test design.
© The Author(s) under exclusive licence to International Association of Medical Science Educators 2024. Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.