This research applied generalizability theory to assess the effect of varying the number of cases and items nested within cases on generalizability of scores on Part II of the National Board Dental Examinations (NBDE Part II). In this research, sources of error were defined. Measurement conditions were classified. Error variances and generalizability coefficients for different conditions were computed. The data analyzed were the item responses of 1,535 candidates enrolled in accredited dental education programs who all took the same test form in 2007. Results showed that using more cases of fewer items might lead to a greater increase in generalizability than using more items per case. Other practical considerations such as time and cost constraints must be taken into account when applying the results of this research in other testing situations.