Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification

Julie Chang, Vincent Sitzmann, Xiong Dun, Wolfgang Heidrich, Gordon Wetzstein

Research output: Contribution to journalArticlepeer-review

102 Scopus citations

Abstract

Convolutional neural networks (CNNs) excel in a wide variety of computer vision applications, but their high performance also comes at a high computational cost. Despite efforts to increase efficiency both algorithmically and with specialized hardware, it remains difficult to deploy CNNs in embedded systems due to tight power budgets. Here we explore a complementary strategy that incorporates a layer of optical computing prior to electronic computing, improving performance on image classification tasks while adding minimal electronic computational cost or processing time. We propose a design for an optical convolutional layer based on an optimized diffractive optical element and test our design in two simulations: a learned optical correlator and an optoelectronic two-layer CNN. We demonstrate in simulation and with an optical prototype that the classification accuracies of our optical systems rival those of the analogous electronic implementations, while providing substantial savings on computational cost.
Original languageEnglish (US)
JournalScientific Reports
Volume8
Issue number1
DOIs
StatePublished - Aug 17 2018

Fingerprint

Dive into the research topics of 'Hybrid optical-electronic convolutional neural networks with optimized diffractive optics for image classification'. Together they form a unique fingerprint.

Cite this