Facebook’s new AI turns real-life characters into controllable game players

Through its new AI, Facebook is trying to make itself more interactive for users by enabling real-life people to be turned into playable video game characters that can be controlled by the user.

Facebook’s AI Research team recently created an AI called ‘Vid2Play’ that extracts playable game characters from videos of real life people. The system can analyze random videos of people doing certain actions and then recreate that character and action in any environment and let the user control them with a joystick.

The team made use of two neural networks called ‘Pose2Pose’ and ‘Pose2Frame’. A video is first fed into a Pose2Pose neural network designed for particular kinds of actions such as dancing, tennis or fencing. The system then figures out where the person is compared to the background, and then isolates them and their poses, reported Engadget.

Mark Zuckerberg aims to create machine that read users’ thoughts

Next, the Pose2Frame takes the person, their shadow and any object that they are holding and inserts them into a new scene with minimal artifacts. The person can then control their movements based on poses from the video with the help of a joystick or a keyboard, as per Venture Beat.

Facebook’s Vid2Play could possibly make gaming more personal, and will let users insert their own character, or favorite YouTube personality into games. “[It] addresses a computational problem not previously fully met, together paving the way for the generation of video games with realistic graphics,” the team wrote. “In addition, controllable characters extracted from YouTube-like videos can find their place in the virtual worlds and augmented realities.”