NVIDIA has developed an AI model capable of changing the viewing angle of a video in real time

NVIDIA and the University of Maryland have unveiled a new AI model, QUEEN (QUantized Efficient ENcoding), that can quickly and efficiently reconstruct dynamic 3D scenes, including streaming video with a free viewpoint.

QUEEN’s unique feature is its ability to compress data without losing quality, making it ideal for use in sports, medicine, industry, and media. The model uses 3D Gaussian Splatting, is able to train in less than 5 seconds, and reproduce scenes at a speed of about 350 frames per second, with a model size of 0.7 MB per frame.

This is achieved by efficiently encoding residual attributes between consecutive frames and using a quantum-fluid structure to compress data. The model will be presented at the NeurIPS 2024 conference in Vancouver, and its code is planned to be published in the open access. The QUEEN development promises to revolutionize 3D content creation, offering the ability to interactively view sporting events, educational materials, and other dynamic scenes.