In a blog post from Facebook Reality Labs (FRL), the company revealed their Pittsburgh branch’s tests to generate lifelike virtual avatars that could provide the foundation for the personal avatar of the future.
“Most of us, myself included, don’t live in the places where we grew up,” said Yaser Sheikh, the Director of Research at FRL in Pittsburgh. “I’ve spent my life moving from city to city, and each time, I’ve left relationships that are important to me.”
Facebook Reality Labs’ Codec Avatar capture tool. Image source: Facebook
The project, dubbed “Codec Avatars”, uses what FRL calls “groundbreaking 3D capture technology and AI system” in order to help users create avatars of themselves quickly, and then use tracking on virtual reality (VR) headsets to have users be able to look at other realistic avatars – thereby shortening the psychological distance between users. At the moment, the volumetric captures of avatars takes around 15 minutes, and require a large number of cameras to create the most photorealistic avatars possible. The lab plans to use these captures to train AI systems so consumers could then quickly and easily build a Codec Avatar from just a few photos or videos.
“Codec Avatars is an active research project today, but it could radically change the way we connect through VR headsets and AR glasses tomorrow,” the blog post explains. “It’s not just about cutting-edge graphics or advanced motion tracking. It’s about making it as natural and effortless to interact with people in virtual reality as it is with someone right in front of you. The challenge lies in creating authentic interactions in artificial environments.”
While the Codec Avatars technology is still “years away” from being reaching consumer headsets, FRL is continuously working to make the project more efficiently and safely, especially considering recent concern around deepfakes.
“We have the resources to drive new ideas,” says Sheikh. “And when you add the chance to bring together diverse expertise to fully tackle these massive design challenges, it fuels a pace of innovation unlike anything I have seen before.”
Before and after image from Codec Avatars. Image source: Facebook
