Chinese researchers have demonstrated a new paradigm for deep learning-based computational imaging (CI) that takes both physical model and training data into consideration. This enables them to obtain state-of-the-art results in single-pixel imaging (SPI).
This research was led by SITU Guohai from the Shanghai Institute of Optics and Fine Mechanics (SIOM) of the Chinese Academy of Sciences (CAS). Results were published in Photonics Research on December 13, 2021.
CI techniques mainly focuses on the design of encoding methods and decoding algorithms. Typically, traditional strategy uses expert knowledge to design the encoding system (forward physical model), and then exploit the model driven optimization algorithms for decoding. In recent years, data-driven strategy brings milestone progress for CI. Such strategy intelligently designs the encoding mode and reverse decoding model by using training data, which greatly improve the imaging quality and efficiency. However, its practical applications are restricted by the difficulties in data acquisition, generalization and interpretability. Although the research group previously proposed a model-driven learning framework to eliminate the requirement of training data, it did not make full use of the prior information in big data.
The team proposed a novel paradigm for CI, which makes fully use of the training data and the physical model. They demonstrated in SPI when the sampling ratio is as low as 6.25%. In their scheme shown in Fig. 1, the training data is used to optimize the encoding patterns and the decoding network (Fig. 1 a), while the physical model is utilized to eliminate the artifacts in the predictions due to the generalization issue of the train network model (Fig. 1 b).
Their proposed method (Fine tune) can obtain the best imaging fidelity among typical SPI algorithms including linear correlation (DGI) and compressive sensing (TV and GISC) when the sampling ratio is as low as 6.25%, as shown in Fig. 2 (a-c). They also built a SPI-LiDAR system and verified the proposed method in outdoor experiment. A tower located at about 570 m away from the LiDAR system was successfully reconstructed by the proposed method, suggesting its great potential for remote sensing.
This work was supported by Chinesisch-Deutsche Zentrum für Wissenschaftsforderung; National Natural Science Foundation of China; Chinese Academy of Sciences Key Project.

Fig.1. Schematic diagram of the physics enhanced deep learning approach for SPI. (a) The data-driven learning framework. (b) The SPI system. (c) The model-driven fine-tuning process. (Image by SIOM)

Fig. 2. A comparison with typical SPI algorithm. (a-c) are simulation, laboratory and outdoor results, respectively. (Image by SIOM)

Article website:
https://doi.org/10.1364/PRJ.440123