VR. It’s the next big thing. But don’t forget about AR and MR. Well, FMX certainly hasn’t, as day 2 exemplified with a raft of great presentations about the latest developments, including from Magic Leap and Microsoft HoloLens. Plus, day 2 also saw a celebration of the 20th anniversary of Pixar’s Toy Story. Here’s our run down.
Magic Leap
A lot of people seem to be excited about Magic Leap. Is it AR? Is it light fields? The possibility that all will be revealed at FMX meant that a good crowd was in to see Magic Leap motion capture lead John Root’s talk – but a big reveal was not forthcoming. However, Root’s talk – on the future of immersive entertainment – was still enlightening.
Root detailed the advent of VR in LA, where literally a small meetup imagined by a 19 year old film student has turned into a major regular mini-convention. Each time Root had a take-away from the number of people and the enthusiasm displayed at these meetups:
1. VR are generally good people
2. People will go out of their way to be at VR meetups
3. People are tenacious, persistent and temperature resistant (one meetup was held in a hangar with no air conditioning)
4. People are thirsty for VR knowledge
5. There’s no apparent end to the number of people interested in VR
The future of immersive media is, according to Root, a time when AR and VR will merge into a single device, the advent of light field displays, sound field audio – and everything being lightweight. It wouldn’t be unreasonable to suggest, then, that Magic Leap is aiming towards all of these things.
And as excited as Root was about the future of the tech, he said there were some possible accompanying problems – ‘the George Orwell future’, he posited. Could AR and VR bring with it ‘spam’ like contact from advertisers, and even a time when people might publicly – and graphically – tag you with unwanted notifications or messages or marks.
Another danger is the physical effects. Could someone riding a VR rollercoaster at home fall and break their arm? Sometimes VR experiences can be that disorienting, Root noted. He says AR and VR has the possibility to impact what the human body current does with convergence and accommodation of the eyes – these have been in sync since we were born. But when you put on a VR headset, the experience is actually against what your body is used to and can be a strain on your mind and your eyes.
The solution, Root suggests is several-fold:
– Keep frame rate high
– Latency low
– Respect the human brain
– Understand the dangers
– Teach others to do the same.
Panel: Beyond the hype – what it really means to produce VR content
A who’s who of VFX, games, interactive entertainment and VR/AR/MR were part of this panel which looked at the nature of VR storytelling and how experience in games and VFX might shape it. The panel were:
– Sebastian Sylwan (chair – currently working with VR studio Felix & Paul)
– Rob Bredow (Advanced development group at Lucasfilm)
– Alex Laurant (HoloLens Experiences, Microsoft Studios),
– Kim Libreri (CTO, Epic Games)
– Paul Raphaël (Felix & Paul)
– Brad Herman (DreamLab, DreamWorks)
Here were a couple of key observations.
Kim Libreri from Epic noted that as an ex-filmmaker it was challenging for him at first to contemplate the linear timeline of VR – you can’t do cuts or compress time in the normal way.
Similarly, Brad Herman from DreamLab said that in their content they realized you’re not a camera – you’re a person. “People make editorial choices on the run. We’ve even tried pausing the character and make them point a way – we have to force things along.”
Rob Bredow from Lucasfilm indicated that in their VR work they’ve had to work out exactly what it is they’re doing – is a video game? Is it a narrative film? Something else? And users will have different reactions and experiences, and “you might fine tune it and all of a sudden they miss (or can no longer miss) what you were intending to show.”
Alex Laurant suggested that the user’s attention is the most precious commodity – you have to persuade the audience to take actions that feel natural and intuitive. He noted, too, that audio designers are going to enjoy a renaissance of recognition, since the VR experience is so much to do with audio.
Paul Raphaël said one challenge was making a VR experience for the ‘average’ person. “You have to go with average size, height – this limits you – if you want a piece with spectator standing up – you might feel like floating or cutting off at knees. This is one of the reasons all of our experiences have been sitting down ones so far.”
One particularly interesting moment of the talk arose from an audience question – is VR/AR/MR a fad? Libreri said he didn’t think so, and that next year will be a big year for VR. He agreed, though, that VR was, right now, a little bit of a clunky experience. Bredow added that wearing the things doesn’t look cool – yet. And Sebastian Sylwan concluded by relating his own experience showing a VR film to his wife. “I’ve been involved in making movies for 20 years and my family was not generally interested, but I brought Gear VR home with something on it and put it on my wife’s face and she immediately said, ‘I want to show it to my grandfather.’”
Hololens – Microsoft Studios
A later session featured Alex Laurant from HoloLens Experiences at Microsoft Studios giving more detail on the actual HoloLens (but maybe not that much detail). Essentially, the HoloLens is about mixed reality holography – hardware-wise it’s an untethered holograph computer and software-rise it’s powered by Windows 10.
The MR side of HoloLens means that you see the real world with holograms and graphics that can be ‘stuck’ to parts of your environment. This is enabled via various sensors and a depth camera, with inputs to be provided from your gaze, gestures and voice.
The HoloLens is not full 360 – it has a field of view that’s not completely around you, but Microsoft has been designing for that, and Laurant said they can enable vision and gaze in a natural way that makes you forget about the FOV.
VR pioneer Mark Bolas
We recently featured ICT’s director of the Mixed Reality Lab Mark Bolas at fxphd (link) and his seminal work in immersive experiences and VR. In this FMX chat with New York University’s Ken Perlin, Bolas went through a history of his lab’s research, demo’ing a ton of great videos and anecdotes about VR.
Bolas has worked in VR for so long – including at a time in the 90s that it almost completely went away. But, he says, it never went away – it just evolved to only be a narrow field of view experience on monitors. In that way, one of Bolas’ stand-out points was that the games industry may be a good model to base VR work from – it’s all about worrying about the the first person. And, he suggests, it has the VFX and CG expertise in it already.
The pioneer described that anything is possible with VR – intimate experiences, flying with jet packs, exploring synthezied worlds. Bolas was quick to point out, too, that he has had some reservations about VR, both in terms of the nausea it can create (he said that a flying simulator he was part of resulted in research that showed what angle of flight and horizon line was guaranteed to make you barf). “I went through a crisis, too, and decided it was the Manhattan project and we’ve done a terrible thing. But I got over that. I want my jetpack dammit.”
20 years of Toy Story
Continuing the anniversary theme of this year’s FMX, Pixar presented on the advent of Toy Story – released in 1995. Christophe Hery from Pixar chaired the event which featured Bill Reeves (the supervising technical director), Ralph Eggleston (production designer) and Eben Ostby (who is not director of technical artists at Pixar).
The presentation ran through the history with which many people would be familiar – the spin-off of Pixar from Lucasfilm’s Computer Division, and the role of Steve Jobs in funding Pixar and doing a deal with Disney to make Toy Story possible. Those years also involved Pixar demonstrating its storytelling ability and technical wares with several short films. Luxo, Jnr, for example, arose from an idea in the Pixar animation group to teach John Lasseter how to model a Luxo lamp, take advantage of a shading algorithm and then produce a short. It was shown at SIGGRAPH in 1986 and generated enormous buzz.
With Toy Story, Pixar had to suddenly catapult itself into feature film making – and have actual departments like art and story and track all the work. They used Power Animator on SGI Indigos for animation then but really only saw final frames when things had been rendered. We saw an early test of Woody and Buzz that showed Woody more as a ventriloquist puppet and Buzz as ‘Lunar Larry’.
We heard about ‘Black Friday’ when Disney saw the film and basically shut it down. An intense couple of weeks then saw the story department revise the film – including making Woody less of an ass-hole essentially – and things went back on track.
In the original Toy Story budget, the need for 8 animators had been forecast. In the end they needed 33. Today they typically have 70 animators per show. TDs at the time were budgeted as 16. They needed 43. And today they need 150. Production staff then were budgeted at 6; they needed 23 and today have 36. The total crew size on Toy Story was 147 – today at Pixar it’s 400 per film.
Lots of great videos, tests, glitches and final shots were shown. Perhaps the most interesting was a Tom Hanks / Woody test which incorporated dialogue from the ‘Don’t eat the car!’ scene from Turner and Hooch.
