Exploration of the dynamics of functional brain connectivity based on the correlation coefficients of functional magnetic resonance imaging (fMRI) data is important for understanding the brain mechanisms. Because fMRI data are time-varying in nature, the functional connectivity shows substantial fluctuations and dynamic characteristics. However, an effective method for estimating time-varying functional connectivity is lacking, which is mainly due to the difficulty in choosing an appropriate window to localize the time-varying correlation coefficients (TVCC). This paper introduces a novel method for adaptively estimating the TVCC of non-stationary signals and studies its application to infer dynamic functional connectivity of fMRI data in a visual task. The proposed method employs a sliding window having a certain bandwidth to estimate the TVCC locally and the window bandwidths are selected adaptively by a local plug-in rule to minimize the mean squared error. The results show that the functional connectivity changes in the visual task are transient, which suggests that simply assuming sustained connectivity changes during task period might not be sufficient to capture dynamic connectivity changes induced by tasks.