In-sensor computing with halide perovskite-based optoelectronic reservoir networks

The bigger picture

Detecting and classifying data, for example, from video cameras, is a key capability for many applications using artificial intelligence, including robotics, self-driving cars, image detection, and biometrics. However, the impressive progress in the capabilities of artificial intelligence comes at the cost of rapidly increasing energy consumption. A large contributor to this energy consumption is the transfer of data from sensors to processors in order to detect or classify the input data. In this work, we demonstrate a microscale halide perovskite semiconductor device that simultaneously senses and processes information. The information can be provided as electrical or optical input, and we show that the classification accuracy is highest if the two inputs are combined. This resembles how the brain merges information from, for example, sight and touch to gain a better understanding of the world.

Summary: Physical reservoir computing can provide efficient neuromorphic in- and near-sensor computing applications. Typically, reservoir networks are designed to process light or voltage inputs. Here, we demonstrate a multimodal optoelectronic reservoir network based on halide perovskite semiconductor devices capable of processing voltage and light inputs, which is also scalable for constructing high-density sensor arrays. The devices consist of micrometer-sized, asymmetric crossbars covered with a methylammonium lead iodide (MAPbI3) perovskite film. Using 4-bit inputs and linear readout layers for classification, we demonstrate multimodal networks capable of processing both voltage and light inputs. The networks reach mean accuracies up to 95.3% ± 0.1% and 87.8% ± 0.1% for image and video classification, respectively. The networks significantly outperformed linear classifier references by 3.1% for images and 14.6% for video. We show that longer retention times benefit classification accuracy for single-mode networks and give guidelines for choosing optimal experimental parameters.