Confocal microscopes are invaluable in biomedical research as they can produce three-dimensional images of thick samples. However, these images are blurry in the third dimension and get worse the thicker the sample. Now, a team led by NIBIB scientists has significantly improved confocal imaging to construct higher-resolution 3D images of fine structures in living samples.
The work was led by NIH principal investigator Hari Shroff, PhD, in collaboration with NIH scientist Yicong Wu, PhD, and visiting graduate student Xiaofei Han at the National Institute of Biomedical Imaging and Bioengineering. Other collaborators included Patrick La Riviere of the University of Chicago, Arpita Upadhyaya and Sougata Roy of the University of Maryland, Daniel Colón-Ramos of Yale University, and industrial partners at Applied Scientific Instrumentation.
The team combined innovative hardware (multiview microscopes) and cutting-edge software (deep learning techniques) to improve confocal microscopy on single cells, live worm embryos, and tissues from flies, mice, and worms.
“It was exciting to see several different computational tools that we have developed together over the last few years come together in this paper,” said La Riviere, who has collaborated with Shroff since 2014, “some have been redeployed after having been successful in our previous computational microscopes and others have been used for the first time against these large, sparse samples.”
The major hardware advance created by Shroff and his colleagues was the construction of a linear scanning confocal microscope that views the sample from three directions, called triple-view linear scanning confocal microscopy. The researchers then used software to merge the three views into a single high-resolution 3D image, thereby improving overall resolution and performance on thick samples.
The “line scanning” part of the new confocal microscope refers to highly focused lines that sequentially display the sample. Line scanning also allowed the researchers to achieve even higher resolution by isolating fluorescence in the vicinity of each line and using computer programs to stitch together the resulting “super-resolution” images of each view.
The team used a type of artificial intelligence (AI) called deep learning to compensate for trade-offs made when using low light levels. Although lower light levels reduce damage to samples due to focused laser illumination (thereby allowing longer imaging times), lower light levels are less effective at resolving certain features. The team’s approach produces higher quality images by training deep learning algorithms to take multiple blurry, low-resolution images and transform them into a high-resolution image.
Using deep learning techniques, they were able to obtain high-resolution images of a wide range of specimens and biological processes, including the development of an intestinal worm (an organism commonly used to study developmental processes). The technique allowed imaging of the roundworm embryo as it progressed through development to eventually hatch as a newborn worm, which included visualization of the embryos’ new nerve cells as they began to “shrink” within the eggshell.
These remarkable images also allowed for more accurate counts of cell numbers in adult worms and worm embryos; visualization of fine signaling processes in fruit fly wing development; and observations of nanoscale dynamics of proteins within immune cells.
In their quest to find ways AI could help with their work, the team also discovered that with enough training data they could teach the deep learning program to take a single confocal view (from a single direction) and predict what the sharpest volume constructed from the three views should be.
Colón-Ramos summarizes the importance of the efforts of this long-standing collaboration. “We are excited to use these new, integrated approaches to delve deeper into the biological foundations of developing organisms and understand them at all scales and in living animals, from the organization of molecules and organelles within cells to the organization of cells within tissues.”
Financial support: internal laboratories and external grants, NIH; Marine Biological Laboratory in Woods Hole, MA; and Gordon and Betty Moore Foundation. The work was published in Nature.1.
1. Wu, Y., Han, X., Su, Y. mysuch. Super-resolution multiview confocal microscopy. Nature (2021). https://doi.org/10.1038/s41586-021-04110-0
