Article https://doi.org/10.1038/s41467-023-41496 -z
Deep learning at the edge enables real-time
streaming ptychographic imaging
Anakha V. Babu
1,4,5
, Tao Zhou
1,5
, Saugat Kandel
1
,TekinBicer
1
,
Zhengchun Liu
1
,WilliamJudge
2
,DanielJ.Ching
1
,YiJiang
1
, Sinisa Veseli
1
,
Steven Henke
1
,RyanChard
1
, Yudong Yao
1
, Ekaterina Sirazitdinova
3
,
Geetika Gupta
3
,MartinV.Holt
1
, Ian T. Foster
1
, Antonino Miceli
1
&
Mathew J. Cherukara
1
Coherent imaging techniques provide an unparalleled multi-scale view of
materials across scientific and technological fields, from structural materials to
quantumdevices,fromintegratedcircuits to biological cells. Driven by the
construction of brighter sources and high-rate detectors, coherent imaging
methods like ptychography are poised to revolutionize nanoscale materials
characterization. However, these advancements are accompanied by sig-
nificant increase in data and compute needs, which precludes real-time ima-
ging, feedback and decision-making capabilities with conventional
approaches. Here, we demonstrate a workflow that leverages artificial intelli-
gence at the edge and high-performance computing to enable real-time
inversion on X-ray ptychography data streamed directly from a detector at up
to 2 kHz. The proposed AI-enabled workflow eliminates the oversampling
constraints, allowing low-dose imaging using orders of magnitude less data
than required by traditional methods.
Ptychography is a high-resolution coherent imaging technique that is
widely used in X-ray, optical, and electron microscopy. In particular,
X-ray ptychography has the unique potential for non-destructive
nanoscale imaging of centimeter-sized objects
1,2
with little sample
preparation, having provided unprecedented insight into countless
material systems including integrated circuits
3
and biological
specimens
4
. Strain information can be obtained additionally when
combined with X-ray diffraction
5,6
. In the optical regime, the compre-
hensive depth information of ptychography has allowed 3D imaging of
large and thick samples with micrometer resolution
7
, while novel var-
iations in the Fourier domain have enabled imaging in gigapixel scale
8
with single shot exposure
9
.Lastbutnotleast,recentdevelopmentsin
electron ptychography have witnessed the accomplishment of a
record-breaking deep sub-angstrom resolution
10,11
.
Ptychographic imaging is performed by scanning a coherent
beam across the sample with a certain degree of spatial overlap and
recording the resulting far-field diffraction patterns. Subsequently,
an image of the sample is recovered by computationally inverting
these measured patterns. The inversion of ptychographic data (or
phase retrieval) provides a solution to the phase problem, where
only the amplitude information about the wave exiting the samp le,
and not its phase, is retained in the measured intensities. Currently,
real-time imaging with ptychography is limited by the use of con-
ventional phase retrieval methods. These methods do not produce
live results until after acquiring a few tens if not hundreds of dif-
fraction patterns, despite substantial advancement in the recon-
struction algorithms
12
. Moreover, the spatial overlap required for
the numerical convergence limits the sample volume that can be
imaged in a given amount of time and can cause extra damage in
dose-sensitive specimens.
State-of-the-art ptychography instruments bring about new and
prohibitive computational challenges with their drastically increased
Received: 13 January 2023
Accepted: 6 September 2023
Check for updates
1
Argonne National Laboratory, 9700 S Cass Ave, Lemont, IL, USA.
2
Department of Chemistry, University of Illinois, Chicago, IL, USA.
3
NVIDIA Corporation,
Santa Clara, CA, USA.
4
Present address: KLA Corporation, Ann Arbor, MI, USA.
5
These authors contributed equally: Anakha V. Babu, Tao Zhou.
e-mail: amiceli@anl.gov; mcherukara@anl.gov
Nature Communications | (2023) 14:7059 1
1234567890():,;
1234567890():,;
资源评论
