Sony Working On Personalised VR Controls Using “Custom Cues”

Rundown:

• Sony Interactive Entertainment has recently published a patent for a system that improves personalised controls and communication in virtual reality (VR) environments.
• The system suggests storing a player’s virtual reality (VR) profile, which includes instructions to modify virtual display elements in response to custom cues.
• During a gameplay session, communication may be monitored to detect the specific cues and the virtual elements modified accordingly.
• Cues can be predetermined by the player and associated with custom instructions that modify virtual display elements according to the player’s preferences.
• The system can detect cues based on sensor information captured by the video game device or touchpad and generate typed output or voiceover audio in real-time.
• The technology aims to enhance the accessibility and usability of virtual reality (VR) systems, particularly for new or inexperienced players and those with disabilities.

Earlier today, we came across a recently published patent titled “PERSONALIZED VR CONTROLS AND COMMUNICATIONS,” filed in October 2021 under the name of SONY INTERACTIVE ENTERTAINMENT LLC. The patent, published earlier this week, describes a system for improving personalised controls and communication in virtual reality (VR) environments. It suggests storing a player’s virtual reality (VR) profile, which includes instructions to modify virtual display elements in response to specific cues.

During an interactive session in a virtual environment, the communication may be monitored to detect these cues, and the virtual elements can be modified accordingly based on the player’s profile. This technology aims to improve the player’s experience in virtual environments, especially for those who may find navigating complex controls or actions challenging or time-consuming.

“Systems and methods for personalized controls and communications in virtual environments are provided. A virtual reality (VR) profile may be stored in memory for a user. Such VR profile may specify a cue associated with custom instructions executable to modify one or more virtual display elements. An interactive session associated with a virtual environment in which the user is participating via a user device may be monitored based on the VR profile stored for the user,” reads the abstract for the patent.

“The cue specified by the VR profile may be detected as being present in the monitored communication session. The virtual elements may be modified within a presentation of the virtual environment provided to the user device in accordance with the executable instructions associated with the cue specified by the VR profile of the user.”

Sony Interactive Entertainment aims to address the challenge of navigating complex controls and actions within virtual environments, which can be particularly difficult and time-consuming for new or inexperienced players, especially for players with disabilities. By providing personalised controls and communications, the patent seeks to improve the player’s experience in virtual environments and reduce the barriers to entry.

By detecting and responding to specific cues associated with custom instructions stored in the player’s virtual reality (VR) profile, the system can modify virtual display elements according to the player’s preferences, making the experience more intuitive and easier to navigate. The system aims to enhance the accessibility and usability of virtual reality systems, making them more inclusive and user-friendly.

In the patent context, “cues” are signals or indicators within the communication that is being monitored during the gameplay session in the virtual environment. These cues are predetermined by the player and associated with custom instructions that modify the virtual display elements according to the player’s preferences.

For instance, a cue could be a voice command, a particular gesture or movement, or even a specific phrase entered by the player. Once the specified cue is detected within the communication, the system can execute the associated instructions and modify the virtual display elements accordingly.

According to the patent’s claims, the cue can be a real-world movement by the player and the detection is based on sensor information captured by the video game device. The real-world movement may be a gesture in physical space, and the sensor information is captured by a camera associated with the player’s device. It may also be a drawing gesture on the touchpad of the player’s device, and the touchpad captures the sensor information.

The system allows the player to personalise the cues associated with virtual display elements. For instance, a hand gesture in the shape of an upward heart may trigger a heart eyes emoji, while a downward heart-shaped hand gesture may generate a heart emoji, facilitating communication between players. This implies that the system can accurately detect subtle variations in the cues to execute the corresponding actions effectively.

In addition, the system identifies the virtual elements within the virtual environment based on eye-tracking data indicating that the player is looking at the virtual display elements when the specified cue is detected. It also modifies the virtual display elements by generating typed output when the specified cue is associated with the input interfaces.

The patent also specifies that the system may receive texts from other devices connected to the same gameplay session. When the system modifies the virtual display elements, it can remove the text from the virtual environment, meaning it won’t be displayed in the virtual world. Furthermore, the system can generate voiceover audio based on the text in real-time and provide the audio with the modified presentation of the virtual environment to the player’s device.

While Sony Interactive Entertainment has not yet revealed when it plans to incorporate the recently patented technology into its current and upcoming video game platforms, it’s important to remember that a patent does not guarantee implementation or development. However, this publication shows the company’s commitment to improving the gameplay experience and making video games more accessible to everyone.

Last month, we discovered a similar patent detailing an eye-tracking technology allowing players to make choices in multiplayer narrative video games without using buttons or joysticks. Both patents appear to have the same purpose of making video games more accessible. While it’s uncertain if these conceptions will become a reality, it’s admirable to see Sony Interactive Entertainment’s dedication to inclusivity in the video game industry.

What do you think about this? Do tell us your opinions in the comments below!

Similar Reads: Sony Patent Prevents Players Speaking Over Each Other In-Game