Sony's New AI Patent Predicts Button Presses via Finger-Camera Tech

Sony's latest patent filing hints at a significant leap forward in reducing latency in future gaming hardware, leveraging an AI model supported by additional sensors. The introduction of PlayStation Spectral Super Resolution (PSSR) with the PlayStation 5 Pro marked Sony's foray into upscaling technology, enhancing smaller resolutions to 4K. However, advanced graphics technologies such as frame generation can introduce latency, which might detract from the game's responsiveness.

Competitors like AMD and Nvidia have introduced solutions like Radeon Anti-Lag and Nvidia Reflex to combat this issue. Now, Sony is poised to join the fray with its innovative approach, as detailed in the patent WO2025010132, dubbed "TIMED INPUT/ACTION RELEASE." This patent aims to streamline the "timed release of user commands" by predicting the user's next button press.

According to the patent, latency between a user's input and the system's processing can lead to delayed command execution and unintended in-game consequences. Sony's proposed solution involves a multi-faceted approach: an AI model that predicts the next input and an external sensor, such as a camera monitoring the controller, to anticipate which button will be pressed next. The patent suggests that "the method may include providing camera input as an input to a machine learning (ML) model," with the camera input indicating the upcoming user command.

Additionally, the patent explores the use of controller buttons as sensors, aligning with Sony's history of innovation with analog buttons. While the exact implementation might differ from what's described in the patent, it's evident that Sony is committed to reducing latency without sacrificing responsiveness, particularly in light of popular rendering technologies like FSR 3 and DLSS 3, which introduce additional frame latency.

This technology could revolutionize gameplay in genres like twitch shooters, where high frame rates and low latency are crucial. However, it remains to be seen whether this patent will translate into actual hardware advancements.