Affinity binding is the principle used in a large number of bio-assays. Aside from specific bindings, non-specific bindings usually deteriorate assays by giving false positive signals and restrict the detection limit. Currently, the assay specificity is mainly dependent on the effectiveness of a suitable surface chemistry. We report an approach to discriminate specific and non-specific bindings with dielectrophoretic (DEP) forces for on-chip magnetic bio-assays. Conjugated to the analytes, magnetic particles were used as the agents for DEP force generation. Due to a weaker binding strength, the non-specifically bound particles were removed while specific bindings remained intact. Analytical and finite element calculations were also performed to study all relevant forces. Furthermore, the removal of magnetic particles was also assessed by measuring the magnetic signal using magnetoresistive sensors. This technique can not only be used to improve the specificity of the on-chip bio-assays but also be developed as a tool of force spectroscopy for the study of bio-molecular binding physics.