The Early Conception Factor (ECF) test is a commercially available qualitative assay that reportedly detects a pregnancy-associated glycoprotein present in bovine serum within 48 h after conception. One concern with previous assessments of this test is that animals with viable embryos early during pregnancy that subsequently undergo embryonic loss before pregnancy diagnosis increase the rate of false-positive results and bias the assessment. To preclude this possibility, noninseminated Holstein cows (n = 9) and heifers (n = 8) were evaluated as an unequivocal source of nonpregnant animals, and Holstein cows (n = 17) and heifers (n = 1) inseminated at estrus and in which at least one embryo of transferable quality was recovered at a nonsurgical flush 6 d after artificial insemination were evaluated as an unequivocal source of pregnant animals. Blood samples were collected from all animals 6 d after estrus, which was immediately before embryo collection in pregnant animals. Each serum sample was evaluated using two ECF test cassettes (tests 1 and 2), and the result of each test cassette was interpreted by two independent readers (readers 1 and 2). Test sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were 86, 4, 49, 23, and 46%, respectively. Although the observed agreement between readers (91% for test 1; 89% for test 2) and between tests for the same serum sample (94% for reader 1; 91% for reader 2) was high, the overall rates of false-positive and false-negative ECF test results were 96 and 14%, respectively. We conclude that the ECF test is an unreliable method for determining pregnancy status of dairy cattle on day 6 after estrus.