IT Home On October 18, according to tech media TechSpot, Sony recently filed a patent that describes a scenario where AI predicts the next frame before GPU rendering, reducing game lag and avoiding additional delay caused by frame generation technology.
According to the original report from IT Home, this technology can address the flaw in frame generation, so the display no longer has to wait for two frames to be rendered before an intermediate frame is inserted.Instead, directly use the generated image to predict the next frame，No need to wait for two frames of video before interpolation, helps the game maintain a more stable frame output rhythm.
At the same time, Sony used a jointly trained codec to convert the rendered image into an embedded representation, giving the image a compact numerical expression, so that the decoder can use these cues to generate predicted frames. During game operation, this system works in parallel with the graphics rendering pipeline. When the hardware renders the next frame, the AI processes the previous frame and sends the prediction results to the display buffer. Therefore, before the next frame is rendered, the predicted image may be displayed first.
In addition, the system can utilize the motion vector and the player’s recent controller inputs. This allows the history of the player’s actions to serve as the basis for predicting the next frame’s movements. The training process of the system uses continuous video frames. The neural network initially uses randomly initialized weights, taking a single frame as input, and attempts to predict the next frame. Subsequently, the system compares the predicted results with the actual subsequent frame and calculates a loss value to measure the prediction error.
However, whether this technology can be effectively used depends on whether AI can quickly generate accurate images, thereby filling the gaps in frame output when rendering speed decreases.