Sony Working On Real-Time Image Upscaling Patent

A new patent filing suggests that Sony is developing a machine learning technique for real-time image upscaling for the PlayStation 5 and VR, which can improve the resolution of games without sacrificing performance. Much like Nvidia’s DLSS, this machine learning process allows computers to better run games by rendering the original image at a lower resolution, making it easier for the GPU to process, and using AI to upscale the image to give the appearance it is being run at a higher resolution. This means visually intensive games could be run on higher settings on a 4K screen without taking hits to FPS as the game is actually being run in 1080p, giving a very similar appearance to actual 4K resolution.

“Most recently I’ve spearheaded work in using Neural Rendering to enhance traditional rendering methods, focusing on using implicit neural representations and how to make them run efficiently,” said Andy Bigos, Senior Principal Officer at Sony. “This includes creating custom high-performance interference using compute shaders. The target for this work is PlayStation 5.”

Sony’s patent was filed in April and was published just last month. It outlines a method in which machine learning can create “masks” of a certain image to hide or fill in gaps left by missing or corrupted data. In essence, this would allow for real-time upscaling of images without sacrifices to performance as this technique would be less taxing on the hardware. The patent mentions virtual reality (VR) several times, meaning the technique could be essential to delivering a higher quality image that a superior VR headset needs. Powering a VR headset requires intensive processing power, and it was notable that the PSVR headset delivered a lower resolution.

The new patent describes the machine learning technique as using “much less computing power” and “without sacrificing image quality where it’s needed most to maintain user comfort and immersion”. This might mean the next generation of PSVR headsets could run VR games with much better visual quality and performance. As the 2020s deepen, we could see more 8K screens come into the mainstream too and this technology could conceivably be used to upscale images to 8K also.

(All information was provided by TheGamer)