Fluorescence imaging uses laser light to obtain bright, detailed images of cells and even subcellular structures. However, if you want to observe what a living cell does, such as dividing into two cells, the laser can fry it and kill it. One answer is to use less light so that the cell is not damaged and can continue with its various cellular processes. But at such low light levels there isn’t much signal that a microscope can detect. It’s a faint, blurry mess.
In new work published in the June issue of Nature Methods, a team of microscopists and computer scientists used a type of artificial intelligence called a neural network to obtain clearer images of cells at work, even in extremely low, cell-friendly light levels.1
The team, led by Hari Shroff, PhD, principal investigator at the National Institute of Biomedical Imaging and Bioengineering, and Jiji Chen, PhD, at the trans-NIH Advanced Imaging and Microscopy Center, calls the process “image restoration.” The method addresses the two phenomena that cause blurry images in low light: low signal-to-noise ratio (SNR) and low resolution (blur). To address the problem, they “trained” a neural network to “denoise” noisy images and “deblur” blurry images.
So what exactly is training a neural network? It involves showing a computer program many pairs of matching images. The pairs consist of a clear, sharp image of, say, a cell’s mitochondria, and the blurry, unrecognizable version of the same mitochondria. The neural network is shown many of these matching sets and therefore “learns” to predict what a blurred image would look like if it were enhanced. Therefore, the neural network becomes capable of taking blurry images created with low light levels and turning them into the sharper, more detailed images that scientists need to study what happens in a cell.
To work on denoising and blurring 3D fluorescence microscopy images, Shroff, Chen and their colleagues collaborated with a company, SVision (now part of Leica), to refine a particular type of neural network called residual channel attention network or RCAN.
In particular, the researchers focused on restoring “super-resolution” volumes of images, so called because they reveal extremely detailed images of the tiny parts that make up a cell. Images are displayed as a 3D block that can be viewed from all angles while rotating.
The team obtained thousands of volumes of images using microscopes in their lab and other NIH labs. When they obtained images taken with very low illumination light, the cells were not damaged, but the images were very noisy and unusable: low SNR. Using the RCAN method, noise was removed from the images to create a sharp, accurate and usable 3D image.
“We were able to ‘overcome’ the limitations of the microscope by using artificial intelligence to ‘predict’ the high-SNR image from the low-SNR image,” Shroff explained. “Photodamage in super-resolution images is a major problem, so the fact that we were able to avoid it is significant.” In some cases, the researchers were able to improve the spatial resolution several times over the noisy data presented to the 3D RCAN.
Another goal of the study was to determine how messy an image researchers could present to the RCAN network, challenging it to convert a very low-resolution image into a usable image. In an “extreme blur” exercise, the research team found that at large levels of experimental blur, the RCAN was no longer able to decipher what it was looking at and turn it into a usable image.
“One thing I’m particularly proud of is that we pushed this technique until it ‘broke,'” Shroff explained. “We characterized the SNR regime on a continuum, showing the point at which the RCAN failed, and also determined how blurry an image can be before the RCAN cannot reverse the blur. We hope this will help others set limits on the performance of their own image restoration efforts, as well as spur further development in this exciting field.”
This research was supported by the intramural research programs of the National Institute of Biomedical Imaging and Bioengineering and the National Heart, Lung, and Blood Institute within the National Institutes of Health, and grants from the National Institute of General Medical Sciences, the NIH Office of Data Science Strategy, and the National Science Foundation.
This Science Highlight describes a basic research finding. Basic research increases our understanding of human behavior and biology, which is critical to promoting new and better ways to prevent, diagnose, and treat diseases. Science is an unpredictable and incremental process: each research advance builds on past discoveries, often in unexpected ways. Most clinical advances would not be possible without knowledge of fundamental basic research.
