2.1. Brain-inspired visual processing system based on ion-modulated memtransistor.
According to the experiments conducted by Treichler,[32] 83% of the information received by human come from vision, so the visual system plays a crucial role in the perception of the human brain. Figure 1 a shows conventional image recognition systems, which mainly include the sensor array, memory module, computing unit (e.g., CPU) and post-processing unit (e.g., GPU). First, raw image data are received by the sensor chip, and then the sensor data get stored in the memory devices for subsequent processing. Due to the more-or-less influences caused by the environmental change, the raw data generally carry some noise, so the denoising algorithm is deployed in the computing unit. Massive data are transferred between computing units and memory devices before the algorithm converges. After that, the pre-processed images are sent to the post-processing units, which perform the final classification based on the neural network algorithm. Similar to the data transfer between pre-processing units and memory devices, the neural network parameters are mainly stored in the memory module. Plenty of data need to be transferred frequently between the post-processing units and memory module when implementing the inference.
Given the large amount of data handling, the delay and power consumption get much increased for the conventional image recognition system. Introducing some denoising algorithms also increases the computational complexity of the whole system. For human visual systems, the identification of noisy images is rapid and accurate. Figure 1b presents the human visual system. External information is delivered to the retina of the human eye, then the rod cells and cone cells in the retina integrate the original information. This integrated information is transmitted through the optic nerve to the visual center of the brain. After computing in the network of neuronal cells located in the cerebral cortex, the concrete images were finally identified.
Compared with traditional hardware visual recognition systems, human visual systems are more tolerant of errors, while processing complexity and energy consumption are both at ultra-low levels. Inspired by that, we propose an artificial visual system based on reconfigurable ion-modulated memtransistors (Figure 1c) by limiting the range of signal strength applied to the gate, which enables the device to act as the key parts for different modules. These different modules mainly include 1) filtration units that simulate the function of cells on the retina which execute the information pre-processing; 2) accelerating the in-memory computing unit of the matrix-vector multiplication (MVM) used for image classification inference. And 3) nonlinear activation of neurons after receiving synaptic weighted calculations. By elaborately utilizing ion dynamics with different temporal scales which can be controlled inside the device, image recognition can be implemented more efficiently and reliably.