For most people, virtual reality remains a novelty, and the experience of seeing your movements recreated via a digital body can be a strange one. Designing those bodies, and bridging the gap between real and virtual, poses unique challenges for creatives who are tasked with building a believable connection between people and their avatars.
CR met with Framestore Creative Director Gavin Fox – who’s worked on several virtual reality experiences for theme parks – to discuss some of the tricks of the trade, and why putting people into a digital costume gives them the freedom to play.
Creative Review: How do you design an authentic body for someone in VR, and make that experience engaging?
Gavin Fox: We’ve made a fair amount of VR projects, especially where you have embodiments. One of the projects I’m finishing up at the moment isn’t just with the user, but it’s also the body of their friends in the experiences. Often the first way to think about this is scanning people’s faces and making them look as much as they can like your friends. We chose not to go down that route in various instances, because unless you get it looking absolutely perfect, it looks strange – especially as you know that person so well.
An alternate way is to turn them into characters, so perhaps they’re wearing clothing like a crash helmet so you don’t see their face. The important thing is trying to simulate their size and their presence in the space. That can be a lot more automated, using the position of the headset and the hands and feet. We’ve done a lot of work with the thousands of different variations of what the right body size will be at different heights. Once you’ve got people’s height and scale, then the next thing that comes with VR is having their motion and gait, and the nuance of how they move. Everybody moves slightly differently and people recognise that instinctively. Getting those right is probably more important right now than trying to get the look of someone. That’s too far away, unless you want to spend a week recreating someone’s digital avatar.
Top image: VW Touareg Hyper Reality Test Drive, which takes users on a stunt track through alien worlds; above: A Moon for All Mankind, a lunar gravity VR simulation created in partnership with NASA Johnson Space Center
CR: How do you capture that bodily movement?
GF: It’s motion tracking in the room, or edge tracking. The headset itself has tracking so it knows where the head is. But the more tracking points we can get on a person the better. The most basic one is tracking the headset and giving them some kind of controller – if they have controllers we know where the hands are. We use a system called inverse kinematics that works out everything else for you. For some experiences we put more tracking elements on a person, on their feet or wrists … the more tracking elements we can get on, the more accurate we can make it. It’s about tracking every movement we can in the room, and relaying that back as quickly as possible.
CR: Is it hard to adjust people to a VR environment where your feet are moving, but your body isn’t?
GF: Sometimes in VR you don’t have a body at all, you’re just floating, but you can see your friends. It’s complicated, because if you don’t get it right you look down and see a weird, glitchy puppet that looks horrible, so it can be better to see a shadow beneath you. We try to put markers and trackers on the feet, so you can see them and know where they’re positioned in space. As far as locomotion around the room, the best way is to have a big space. I work on theme park attractions, and if we’re having an experience where you’re moving around, there’s room set aside that’s built and designed to be a VR space, with physical walls you can touch. You can redirect motion, which is where you trick people.
A VR experience we did a few years back had people walking across a rope bridge on top of the Dolomites. You can feel the bridge and the rocks, but when you get to the end it crashes away. Unbeknownst to people when that rockslide came down, we actually flipped the space 180 degrees and they were walking back down the same corridor which they thought was one long corridor. You can disorientate people with things like vibrating floors, or tracking someone’s movements around a corridor that’s a subtle curve, but in the VR space it’s a straight corridor. Your mind quickly adjusts to it, so you feel the walls but don’t notice you’re walking in a circle. It’s a massive developing area. Things like moving conveyor belts work, but it doesn’t feel as realistic as actually walking along. It’s just like pushing forward a joystick on a games console, and knowing it’s your thumb moving the character forward, but after a while you forget that and that’s how your mind works. It’s the same with these systems, it’s feels more natural but it’s still a control system.
CR: What’s the biggest challenge designers have with putting people’s bodies into these environments?
GF: The big challenge is trying to accurately recreate what someone looks like. Especially if it has to be done quickly. Accurately recreating someone’s face, and how it moves, and the way their eyes look, is a huge task. We’re doing it for celebrities and actors that we use in VR experiences, but for another user that’s hard to do quickly. The resolution of the headset is another thing that’s constantly evolving, and it’s about trying to get to the point where you can see more detail of the person – the pores of their skin and the way their hair moves. For that we need high res headsets.
Interstellar cinematic VR experience, which took users into the cockpit of spaceship
CR: And have you done any research into what that relationship is between being in the real world and seeing a recreation of yourself?
GF: It’s about testing ideas. Most of the time when you’re making a new digital avatar, you’re trying to cover it up – put clothing on it, so the person feels like a character. There’s pros and cons to that. The big pro is that the bulk of what we’re doing is telling a story, and putting people into a space they wouldn’t normally be in. By putting people into a costume they become an actor, and know they’re a character, which gives more freedom to perform and do what you wouldn’t normally do. When you see yourself in a video game, your avatar doesn’t look like you – it’s not the real world. So giving you space gloves or motorbike gloves and leathers, what it’s going to be, helps you get into the mindset of the task you’re being given.
CR: Do you think because people are used to playing video games, and seeing themselves as a different person, it doesn’t feel strange?
GF: I think so. The vast majority of people have never tried VR before. The public are still new to it as an experience, but nobody seems to flinch at characters not looking like them. If it’s a character, it’s more successful. If we try to make you look like a normal person, and you have the wrong skin tone, clothing and hair, that’s going to feel odd. The more characterful and in-world we can make it, the more it fits the narrative and isn’t questioned as much.
CR: Are designers still going to try and cross the uncanny valley?
GF: Everyone’s going to strive for photo-real, and that will be the goal because we can already do the characterful version. The more VR technology develops, the more people will be exposed to it and the quality of the experience needs to rapidly increase as the audience gets used to it. Most people come off a VR attraction, and they don’t talk about how awesome the creatures looked, they’ll be talking about moving their hands and seeing them move in VR.
It’s like when people stopped running out of the cinema because they thought the train was going to run them over, they started thinking about a story. It wasn’t good enough to just have moving pictures. For me a lot of VR experiences feel like tech demos, and I’m always striving to get a lot more and think about it as a mature medium. We need storytelling and believable characters, and a good music score – and all those things you expect from other mediums. It’s not just about the technology, the creative needs to increase exponentially as the audience gets used to this type of medium as well.

CR: Is there a potential dark side to this? There’s been musicians, like Tupac, recreated as holograms, and actors’ bodies that have been digitally scanned. Is there a disturbing aspect, for people’s digital beings?
GF: When people don’t know what’s being given to them, the possibilities for propaganda are endless. The technology is there to put a different voice into someone’s mouth. If we get to the stage in the future where we do have much more realistic VR environments that can be more powerful, and believable. There’s a few people trying to regulate it, but there need to be rules about how you do these things and manage them. There are actors archiving scans of themselves to be used in future productions – it’s happening right now, and it’s quite common to have scans of actors used in movies for stunts, but it could be used in other ways in the future. It’s something that needs to be watched.
CR: Historically speaking, advertising and gaming have both had issues with representation. Is VR going to fall into the same trap?
GF: That’s a huge topic in the gaming industry, so with VR it’s in the front of people’s minds already. It’s one of the driving factors for putting people in clothing or gloves, to avoid it. But if anyone’s making a system, then they’re already talking about how you choose the correct gender, body type and skin colour. It becomes expensive though to build those systems that let you create different versions of humans, so sometimes you go down the other route because its unachievable with the budgets you have. However, there are many VR experiences where you can choose your own avatar, and design it yourself.
framestore.comThe post Designing the digital self appeared first on Creative Review.