This project proposes a lightweight light estimation method based on penumbra effects, which is an improvement and optimization of DeepLight: light source estimation for augmented reality using deep learning. Since the original method only estimates the direction of a single main light source, the rendering effect is relatively simple. Considering that the soft shadow effect in reality is formed by the joint action of multiple light sources from different directions, we extended the single main light source estimation to multiple parallel light source estimation and introduced the concept of light source color temperature to make the rendering effect more realistic and natural.

The model training uses the RGB-D synthetic dataset generated by Blender, where the alpha channel of the image is used to store the depth information from the object to the camera. To enhance the realism of the synthetic data, we performed a series of post-processing optimizations on the rendered images.