Capturing 3D objects for use in extended reality (XR) can now be done in a snap. A global team, led by Min H. Kim, associate professor of computer science at the Korea Advanced Institute of Science and Technology (KAIST), developed a new method of replicating physical objects in virtual reality (VR) using only consumer-grade cameras, be it a conventional camera or the one the comes with your smart phone. To convert these 2D stills into 3D objects, the team developed an algorithm that takes hundreds of still images of an object (taken with the flash on), and generates the 3D model from them. This method does not need any input geometry from the object to capture it in 3D, significantly lowering many barriers to entry for the modeling process.
“Traditionally, [3D object capture] has been either done manually by 3D artists, which is a labor-intensive task, or by using specialized, expensive hardware,” said Kim. “Our method is straightforward, cheaper and efficient, and reproduces realistic 3D objects by just taking photos from a single camera with a built-in flash.”
Kim and his collaborators used a Nikon D7000 and the built-in camera of the LG Nexus 5X (a low-end Android smartphone), as reported by Science Daily and CACM. Kim and his team recently took their work to SIGGRAPH Asia 2018, an annual conference in Tokyo that features cutting edge researchers in science, art, gaming, animation, and much more. They posted a video of their results to the KAIST VCLAB YouTube Channel.
“Many traditional methods using a single camera can capture only the 3D geometry of objects, but not the complex reflectance of real-world objects,” he also noted. “Using only 3D geometry cannot reproduce the realistic appearance of the object in the AR/VR environment. Our technique can capture high-quality 3D geometry as well as its material appearance so that the objects can be realistically rendered in any virtual environment.”
While this method is incredibly ground breaking, it does still have some limitations. It can only be done correctly with still images: trying to get the required visual information from video results in 3D objects that do not look like as intended. Even more limiting, it only works for still objects. Moving items, such as human beings, are beyond the scope of this technique. For what the technology can do, though, this is amazing, and will allow all kinds of new objects to be recreated in VR.
