Deep-fUS: A Deep Learning Platform for Functional Ultrasound Imaging of the Brain Using Sparse Data

Abstract

Functional ultrasound (fUS) is a rapidly emerging modality that enables whole-brain imaging of neural activity in awake and mobile rodents. To achieve sufficient blood flow sensitivity in the brain microvasculature, fUS relies on long ultrasound data acquisitions at high frame rates, posing high demands on the sampling and processing hardware. Here we develop an image reconstruction method based on deep learning that significantly reduces the amount of data necessary while retaining imaging performance. We trained convolutional neural networks to learn the power Doppler reconstruction function from sparse sequences of ultrasound data with compression factors of up to 95%. High-quality images from in vivo acquisitions in rats were used for training and performance evaluation. We demonstrate that time series of power Doppler images can be reconstructed with sufficient accuracy to detect the small changes in cerebral blood volume (~10%) characteristic of task-evoked cortical activation, even though the network was not formally trained to reconstruct such image series. The proposed platform may facilitate the development of this neuroimaging modality in any setting where dedicated hardware is not available or in clinical scanners.

Fig. 1.

a, In the state-of-the-art processing, a power Doppler image is created from a sequence of 250 compound ultrasound frames. In each pixel, the temporal signal s_Dopp sampled in the Doppler time t_Dopp, is passed through a bank of filters F to remove the tissue clutter component s_clutter. The retained blood signal s_blood is squared and time-integrated to compute the power Doppler pixel value proportional to cerebral blood volume. b, The Deep-fUS 3D-Res-UNet architecture uses a modified U-Net network consisting of residual blocks arranged in a 5-layer encoder followed by a decoder. An input 3-D convolutional layer extracts spatiotemporal features from the 3-D input structure. The input data is an under-sampled compound sequence created by selecting the first k frames of N_x × N_y pixels (selected frames displayed with a green border). The network outputs N_x × N_y power Doppler images. c, Residual blocks composed of two cascaded Conv/ReLU/Dropout layers implement a shortcut connection between the input and output. d, Representative transfer functions of the input 3-D convolutional filters learned by the network. These were computed by performing a fast Fourier transform of the filter kernels averaged in the 3 × 3 spatial domain. The cutoff frequencies (−3 dB) for the two filters are 95 Hz (left) and 58 Hz (right).

Fig. 2.

a, Representative power Doppler image of a coronal slice of the rat brain reconstructed by Deep-fUS (3D-Res-UNet) from under-sampled sequences with compression factor of (CF) 75%, 85%, and 95% (Top) and absolute error images calculated against the state-of-the-art (SoA) image (Bottom). b, SoA image reconstructed by the conventional processing using 250 complex compound frames. c, Power Doppler images reconstructed with the conventional processing using under-sampled compound data (Top) and respective absolute error images (Bottom). Scale bar in a, b, c: 1 mm.

Fig. 3.

a, Scatter plots of the power Doppler pixel amplitudes and linear regression analysis (y = b₁x + b₂). b-d, Structural similarity index metric (SSIM), normalized mean squared error (NMSE), and peak signal-to-noise ratio (PSNR) of power Doppler images reconstructed by Deep-fUS (3D-Res-UNet; blue) and by the conventional approach (red). The quantitative metrics were calculated against the respective SoA reference images. Results are reported as mean (solid line) and standard deviation (shaded area) calculated over the test set.

Fig. 4.

a, Representative power Doppler image of a sagittal slice of the rat brain reconstructed by Deep-fUS (3D-Res-UNet) from under-sampled sequences with compression factor (CF) 75%, 85%, and 95% (Top) and absolute error images calculated against the state-of-the-art (SoA) image (Bottom). b, SoA image reconstructed by the conventional processing. c, Power Doppler images reconstructed with the conventional processing using under-sampled compound data (Top) and respective absolute error images (Bottom). Scale bar in a, b, c: 1 mm.

Fig. 5.

a, Time series of power Doppler images were recorded continuously during a visual stimulation task. The resulting cerebral blood volume (CBV) signals were correlated with the stimulus pattern. The visual stimulus consisted of 6 light stimuli, each with an ON time of 30 s, distributed in a pseudo-random fashion. b, State-of-the-art (SoA) activation map computed using power Doppler images reconstructed by the conventional approach using 250 complex compound frames. Statistically significant pixels (P < 0.01; Bonferroni corrected) are shown in the heat map. The white contour displays the slice at bregma −7.0 mm from the Paxinos brain atlas. The activation map shows significant bilateral activation of the rat primary and secondary visual cortices (V1/2) and superior colliculus (SC). c, Activation maps computed using power Doppler images reconstructed by Deep-fUS with compression factor (CF) between 75% and 95%. d, Activation maps computed using power Doppler images reconstructed by conventional processing with CF between 75% and 95%. e, Relative CBV signals in the statistically significant pixels of the SoA map in V1/2 for the SoA data (magenta), and Deep-fUS (blue) and conventional processing (red) with CF of 95%. Results are reported as mean (solid line) and standard deviation (shaded area). The stimulus pattern is displayed in gray. f, Relative CBV signals in the statistically significant pixels of the SoA map in SC for the SoA data (magenta), and Deep-fUS (blue) and conventional processing (red) with CF of 95%.

Fig. 6.

Representative power Doppler test images (top) and activation maps (middle) computed with spatially under-sampled sequences with spatial sampling ratio m = 1/2 (a) and m = 1/4 (b). To equalize the compression factor (CF) to 95%, k = 50 and k = 100 compound frames were used in the two cases. The spatial sampling maps are displayed in the bottom plots. The black and white pixels show discarded and retained pixels, respectively. The maps are shown in a sub-grid of 12 × 12 pixels.

Fig. 7.

a, A series of 1000 power Doppler images was filtered based on a structural similarity index metric (SSIM) filter. Black dots display the discarded images in the series. b, We defined a threshold (red) to remove the images with an SSIM value lower than 3 standard deviations from the baseline. c, d, Representative power Doppler coronal images in a case of significant degradation in the conventional reconstruction (c) that was completely resolved with under-sampled processing (d). Scale bar: 1 mm.