research highlights MICROSCOPY Two approaches apply deep learning to improve single-molecule localization microscopy. n single-molecule localization microscopy Approaches that generate complete methods such as photoactivated images from relatively sparse input data can Ilocalization microscopy (PALM) and yield artifacts. Zimmer’s team addressed this stochastic optical reconstruction microscopy issue by developing an algorithm that can (STORM), samples are imaged over multiple identify and reduce artifacts by comparing rounds; in each round a random subset of the generated image with the wide-field fluorophores is activated and imaged at image. This approach was inspired by diffraction-limited resolution. The precise NanoJ-SQUIRREL, developed by Ricardo positions of these individual emitters are Henriques’s lab. A visual representation of the Deep-STORM determined, and after multiple rounds Zimmer notes that a major focus was network architecture. Reproduced with permission a composite super-resolution image is identifying the best way to train the neural from Nehme et al. (2018), The Optical Society. generated from the localized fluorophores. network. For this, they developed a data- Localization microscopy tends to augmentation strategy that allowed them to require thousands of rounds of imaging to effectively increase the number of training generate a high-resolution image, because or blurred images,” explains Zimmer. The images without more experimental data. in each round, sparse emission is preferred. researchers used their approach to generate Still, he recalls, “it was somewhat surprising Sparse emission minimizes the likelihood high-quality images of microtubules, nuclear to see that ANNA-PALM only needs to of simultaneous emission from closely pores and mitochondria, and found that be a trained on a few super-resolution positioned fluorophores, which confounds they were able to obtain super-resolution images—in some cases just one.” He explains their precise localization. The end result images of more than a thousand cells in that ANNA-PALM will improve with time is long acquisition times, which hinder around three hours—an astonishing feat for if trained on more data. Shechtman was throughput and most live-cell applications. the field. also surprised by the training of the neural Many existing methods address these Another method, Deep-STORM, was network; he notes that small numbers of challenges, including some that allow developed by Yoav Shechtman, Tomer experimental images could train the network accurate localization in images where Michaeli and their joint student Elias successfully, and that “while training fluorophore emission is dense. Although Nehme at the Technion – Israel Institute of the net on experimental measurements these work well in some cases, they can Technology. In Deep-STORM, no a priori produced the best results, training the net limit image quality and resolution. For this knowledge regarding the underlying object on simulated data, of which we could easily reason, two groups independently developed is used. Instead, the artificial neural network generate huge amounts, already yielded approaches to improve the acquisition speed ‘learns’ to extract information directly from excellent images.” of PALM/STORM while maintaining image images of dense blinking emitters, after Although these methods represent early resolution. In both cases, the researchers being trained on correct emitter positions. days in the application of deep learning used deep learning to generate super- This allows the trained model to infer to super-resolution microscopy, they are resolution images from a relatively small correct emitter positions in images where poised to have an important impact on the number of frames of localization microscopy emission is dense and output a super- field and herald a bright future for this area. data. Deep learning is a type of machine resolution image of a structure rapidly. Shechtman says that user-friendly versions learning that uses artificial neural networks Here, the ability to image densely labeled of these tools are an important future goal, to learn a mapping between input and samples corresponds to a reduced total and notes that his group is developing a output data. Once trained, these models can acquisition time. Using their approach, the stand-alone version of Deep-STORM. predict outputs from supplied input data. researchers were able to outperform existing Zimmer says that his team is currently One of the two methods, artificial neural algorithms for image generation from developing tools to facilitate training for use network accelerated PALM (ANNA-PALM), densely labeled frames of synthetic data with ANNA-PALM. ❐ was developed by Christophe Zimmer and images of microtubules. and his student Wei Ouyang at the Institut Although the two approaches differ Rita Strack Pasteur. In ANNA-PALM, an artificial in their neural networks and training neural network is trained on localizations approach, they both can be used to generate Published online: 31 May 2018 from a small number of frames matched super-resolution images that are appropriate https://doi.org/10.1038/s41592-018-0028-9 with dense localization data obtained from for quantitative analysis. One important long-duration acquisitions of the same distinction between the output from both Research papers Nehme, E. et al. Deep-STORM: super-resolution structures. The neural network can then ANNA-PALM and Deep-STORM and that single-molecule microscopy by deep learning. Optic a produce accurate super-resolution images of traditional PALM/STORM is that the 18, 2334–2536 (2018). from images generated from a smaller neural networks produce super-resolution Ouyang, W. et al. Deep learning massively accelerates number of frames. “This strategy resembles images that are not composed of super-resolution localization microscopy. Nat. Biotechnol. 36, 460–468 (2018). how humans recognize objects in noisy a compilation of emitter positions. Nature Methods | VOL 15 | JUNE 2018 | 403–409 | www.nature.com/naturemethods © 2018 Nature America Inc., part of Springer Nature. All rights reserved.
Nature Methods – Springer Journals
Published: May 31, 2018
It’s your single place to instantly
discover and read the research
that matters to you.
Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.
All for just $49/month
Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly
Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.
Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.
Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.
All the latest content is available, no embargo periods.
“Hi guys, I cannot tell you how much I love this resource. Incredible. I really believe you've hit the nail on the head with this site in regards to solving the research-purchase issue.”Daniel C.
“Whoa! It’s like Spotify but for academic articles.”@Phil_Robichaud
“I must say, @deepdyve is a fabulous solution to the independent researcher's problem of #access to #information.”@deepthiw
“My last article couldn't be possible without the platform @deepdyve that makes journal papers cheaper.”@JoseServera