Project Description
This project was done as my final/capstone project during my undergraduate at McGill University. Under the supervision of Professor Derek Nowrouzezahrai, I looked at how ray tracing could be improved using recent advancements in machine learning.
Abstract
High quality images and videos, such as those seen in media like film and advertisements, use a technique called ray tracing to produce highly detailed images. This technique takes significant time to generate images due to the millions of individual light rays that must be simulated and complex computations that must be performed.
The focus of this project is to improve the speed at which high quality images can be rendered by preferentially simulating light rays that have a significant contribution to the image. The goal is to do this using machine learning. A neural network will learn how the light is distributed in a scene and can then be used to select optimal paths to trace.
In this work, a neural network architecture known as a Real Non-Volume Preserving (RealNVP) network is used. This network can be trained to replicate an arbitrary complex multi-dimensional probability distribution, and subsequently generate new points from this distribution.
The network was implemented using the PyTorch python library. The training data was generated using the PBRT-V3 renderer. Once the network was trained, it was used to generate new samples that would hopefully contribute to the output image more reliably than randomly tracing rays. These new samples were loaded into PBRT-V3 and used to render images.
When tracing an equal number of rays, it was possible to observe an improvement in the quality of the images rendered using paths generated by the trained neural network compared to images rendered using randomly generated paths.