TY - JOUR
T1 - Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification
AU - Chang, Julie
AU - Sitzmann, Vincent
AU - Dun, Xiong
AU - Heidrich, Wolfgang
AU - Wetzstein, Gordon
N1 - KAUST Repository Item: Exported on 2021-02-19
Acknowledgements: The authors would like to thank Evan Peng for recommending and coordinating fabrication options for the phase mask. The authors also recognize Kevin Ting for rapid prototyping of the rotation mount and Matthew O’Toole, Donald Dansereau, and Joseph Goodman for valuable discussions. J.C. was supported by a National Science Foundation Graduate Research Fellowship. V.S. was supported by a Stanford Graduate Fellowship in Science and Engineering. G.W. was supported by a Sloan Research Fellowship and an NSF CAREER award (IIS 1553333). This project was generously supported by the KAUST Office of Sponsored Research through the Visual Computing Center CCF grant.
PY - 2018/8/17
Y1 - 2018/8/17
N2 - Convolutional neural networks (CNNs) excel in a wide variety of computer vision applications, but their high performance also comes at a high computational cost. Despite efforts to increase efficiency both algorithmically and with specialized hardware, it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Here we explore a complementary strategy that incorporates a layer of optical computing prior to electronic computing, improving performance on image classification tasks while adding minimal electronic computational cost or processing time. We propose a design for an optical convolutional layer based on an optimized diffractive optical element and test our design in two simulations: a learned optical correlator and an optoelectronic two-layer CNN. We demonstrate in simulation and with an optical prototype that the classification accuracies of our optical systems rival those of the analogous electronic implementations, while providing substantial savings on computational cost.
AB - Convolutional neural networks (CNNs) excel in a wide variety of computer vision applications, but their high performance also comes at a high computational cost. Despite efforts to increase efficiency both algorithmically and with specialized hardware, it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Here we explore a complementary strategy that incorporates a layer of optical computing prior to electronic computing, improving performance on image classification tasks while adding minimal electronic computational cost or processing time. We propose a design for an optical convolutional layer based on an optimized diffractive optical element and test our design in two simulations: a learned optical correlator and an optoelectronic two-layer CNN. We demonstrate in simulation and with an optical prototype that the classification accuracies of our optical systems rival those of the analogous electronic implementations, while providing substantial savings on computational cost.
UR - http://hdl.handle.net/10754/628483
UR - https://www.nature.com/articles/s41598-018-30619-y
UR - http://www.scopus.com/inward/record.url?scp=85051767455&partnerID=8YFLogxK
U2 - 10.1038/s41598-018-30619-y
DO - 10.1038/s41598-018-30619-y
M3 - Article
C2 - 30120316
AN - SCOPUS:85051767455
VL - 8
JO - Scientific Reports
JF - Scientific Reports
SN - 2045-2322
IS - 1
ER -