A fluctuation analysis technique using analytic signals is proposed. Analytic signals are suitable to characterize a single mode with time-dependent amplitude and frequency, such as an MHD mode observed in fusion plasmas since the technique can evaluate amplitude and frequency at a specific moment without limitations of temporal and frequency resolutions, which is problematic in Fourier-based analyses. Moreover, a concept of instantaneous phase difference is newly introduced, and error of the evaluated phase difference and its error reduction techniques using conditional/ensemble averaging are discussed. These techniques are applied to experimental data of the beam emission spectroscopic measurement in the Heliotron J device, which demonstrates that the technique can describe nonlinear evolution of MHD instabilities. This technique is widely applicable to other diagnostics having necessity to evaluate phase difference.