Kyle Jackson, CEO of Talespin, details the virtual human platform and how it aims to improve soft skills training by merging VR and AI. The following is a transcript of our interview with him.

3E: What are you bringing to VRX today?

KJ: We’re here at VRX today showing off our virtual human platform. The demo here on the show floor today is around having to let a teammate go as a supervisor. So the situation was was picked specifically because it’s one that we felt if we could pass the bar on emotional realism for this scenario, if we could evoke the emotions that you would really feel in that situation, that we convinced everybody this is a technology that can be used for much broader applications. This particular scenario, you’re letting a peer go that happens to be your your elder, who’s been the company a very long time, and has unfortunately had the third strike. The range of emotions that are expressed in the demo are very wide. You can literally get him to pound his fists and be upset, and yell at you, and he breaks down and cries, or he can have a smooth process, and actually thank you for allowing him to have served the company.

So, we had a lot of different emotional states on that journey, and we are reinforcing what is the proper course of action as you hit a certain emotional state. So, the whole demo even though it is an HR use case is really around emotion, and those soft skill interactions that drive that scenario to either be a success or a failure.

3E: How did you create the virtual human platform?

KJ: The process of creating this platform is actually bringing together the best of traditional motion capture, as well as things like audio based lip-sync, and moving towards actual persona AI. So right now, the character in the experience is actually generating his his actual animations based on the audio file. So not even necessarily using performance data. Then we’re layering on from a library of different ways for him to express himself. That is being done by artists as the scenario is being built out, but we’re also building a language to actually do that programmatically as the context of the sentence is understood. The animations themselves are based off a combination of motion capture data and key-framed animation. A lot of the blend shapes up until you get to the actual amount those are driven by a library of motion capture the mouth and all the neck and other animations that go along with that are being generated on the fly based on phonemes.

3E: What’s the future of virtual human?

KJ: Right now, it’s something that we’re rolling out the first use cases so that we can actually measure the effectiveness and put the data behind the power of using simulation for soft skills and human interaction. We are building the platform to be something that we’d be able to put in the hands of our clients – or, even looking more towards Q3/Q4, would be co-authoring of persona, and actually having no authorship of content. So, meaning that the persona itself embodies your customer or your colleague – with some bounds, not general AI, but with some bounds – based on the fact that you’re gonna go into situation to either give critical feedback, or hire or fire, or try to sell them something. The content authorship will become more at the point of experience.

3E: What trends would you like to see happen in the XR field?

KJ: As far as trends, what we’re seeing in VR and AR this year versus last year, most customers or most enterprises have actually gone through a POC. They’ve seen the actual effects that can be realized through these mediums, and they’ve got convictions. So based on the convictions they have from either POCs or demos that they’ve gotten over the last year, they’re now starting to look for more industry-wide solutions, so looking for things that they can license that are directly applicable to them in their use cases versus having to take the burden of authorship on. You’ll start seeing some very successful players focused on this particular industry, and they go deep in that subject matter, and then licensed it broadly across the industry. That again is something that’s distinctly different from last year, where everybody’s focused on tools or a first experience. We’re seeing demand for licensed solutions for the first time.

We talked more about virtual human among other initiatives, and how it helps to accomplish the goal of making XR as a whole practical, approachable, and inviting to newcomers.