Functional brain network (FBN) is becoming an increasingly important measurement for exploring cerebral mechanisms and mining informative biomarkers that assist diagnosis of some neurodegenerative disorders. Despite its effectiveness to discover valuable hidden patterns in the human brain, the estimated FBNs are often heavily influenced by the quality of the observed data (e.g., blood oxygen level dependent signal series). In practice, a preprocessing pipeline is usually employed for improving data quality. With this in mind, some data points (volumes or time course in the time series) are still not clean enough, due to artifacts including spurious resting-state processes (head movement, mind-wandering). Therefore, not all volumes in the fMRI time series can contribute to the subsequent FBN estimation. To address this issue, we propose a novel FBN estimation method by introducing a latent variable as an indicator of the data quality, and develop an alternating optimization algorithm for jointly scrubbing the data and estimating FBN simultaneously. To further illustrate the effectiveness of the proposed method, we conduct experiments on two public datasets to identify subjects with mild cognitive impairment from normal controls based on the estimated FBNs, and achieve improved accuracies than the baseline methods.