Sony Pictures Imageworks (SPI) is a remarkable, Academy Award–winning visual effects and animation studio. Over the last 25 years, the artists, engineers and producers there have created extraordinary and visually stunning images for more than 100 live-action and animated productions, from Contact and Castaway to Spiderman, Smurfs and Suicide Squad. Their contribution has been enormous, not only to visual effects but also as a leader in technology, helping develop some of the world’s best software (Arnold) and driving the OpenSource initiatives (such as Alembic) that have helped solve industry-wide challenges and deliver spectacular visual effects.
In an industry when all too often companies struggle to survive, SPI is an evergreen. SPI produces a vast array of types of visual stories, from their dedicated team, initially at their Sony Pictures Studio home in Culver City, and increasingly from an international collective of artists in Canada and elsewhere.
fxguide spoke to five of the current visual effects supervisors as part of a special celebration at Siggraph last month in LA.
SPI was founded in 1992 with just five employees, with the focus on providing visual effects for live action films. One of those original leads is still a senior Visual effects Supervisor at SPI, Jerome Chen. Their first office was sharing the space with TriStar Pictures production executives in what was then called the TriStar building (now called Jimmy Stewart). Their first office conference room becomes SPI’s first data centre.
The first thing that Jerome Chen, Senior VFX Supervisor Sony Pictures Imageworks, worked on at SPI 25 years ago was So I Married an Axe Murderer with Mike Myers. “I was brought on a modeller and I made a digital pickaxe – but it ended up being cut from the movie! So the first thing I ever did was never used,” he fondly recalled.
The team’s first major film was for Director Wolfgang Petersen, placing Clint Eastwood as a Presidental secret service agent in In the Line of Fire. Chen recalls that at that time everything had to be invented, “we actually didn’t have any pipeline to use, for even something as simple as getting the film scanned. So we had to create technology to scan the film, how to scale it down, the colourspace – I learnt a tremendous amount on that film.” The film was supervised by John Nelson (who would go on to win an Oscar for Gladiator and be nominated multiple times for films such as I,Robot and Iron Man).
Back in the mid nineties, SPI actually did their visual effects in 8 bit Log rather than 32 bit float. “[In the Line of Fire] was composited using Wavefront composer (or video composer at that point) and it had not been used much for film. By today’s standards it is an archaic piece of software, but back then it was the backbone of what we were doing,” explains Chen.
In 1994 the world was stunned to see a bus jump a gap in a highway off ramp for the film Speed. The shot was accomplished with a live-action stunt bus flying off a ramp. Imageworks artists digitally removed sections of the roadway to create a gap and replaced it with a matte painting to make it appear as if the bus was jumping over a gap in an unfinished section of freeway.
Five time Academy Award winner Ken Ralston (Star Wars Ep VI: Return of the Jedi, Cocoon, Who Framed Roger Rabbit, Death Becomes Her, Forrest Gump) joined SPI in 1995 and continues to day as Senior VFX Supervisor at Sony Pictures Imageworks. The first thing that Ralston worked on was actually the John Travolta film Phenomenon (1996). As he had been hired to help build SPI into a world class facility, he deliberately picked a smaller effects film. “I didn’t really know what things were like at Sony (SPI) or how it worked, so I supervised Phenomenon, just to do something smaller and see how it went through the facility (pipeline) and it was very educational doing that,” he recalls, “and it was a lot of fun too.”
Ralston brought with him and attracted a wave of brilliant young artists to SPI. By 1995 SPI had become one of the world’s leading visual effects companies with a particularly strong digital agenda. SPI was formed at a time when visual effects were much rarer and vastly more capital intensive to produce. Few if any ‘standardised’ approaches existed. To set the context, Discreet Logic Flame was only released in 1992, or rather shown for the first time under it’s original name ‘Flash’, developed by Australian Gary Tregaskis at NAB in April. SPI would be an early adopter of Flame. They set up a High Speed Compositing unit inside SPI and produced stunning early work such as for the film Contact (1997). Ralston continued his longstanding relationship with filmmaker Robert Zemeckis, and was the Visual Effects Supervisor on Contact. It was their first project together at Imageworks, but it would by no means be their last.
Contact begins with an opening cosmic pull-back shot taking up 4,738 frames – the longest visual effects shot ever at that point in time. Other incredible shots in Contact included invisible effects for the mirror sequence, the transporter and the final beach scene.
Contact was “pretty intense” recalled Ralston. “I went to the premiere of Contact and then that night I went back to SPI to finish working on the film, so that was not the most fun I ever had,” he joked. The beach scene was both complex and one of the last pieces to be finished, so the premiere actually contained Temp shots. Ralston being the natural leader so many in our industry know him to be, actually lifted several bottles of Dom Pérignon from the after party, and took it back to the compositor who, of course, missed the premiere and the party.
Sue Rowe, the newest VFX Supervisor Sony Pictures Imageworks, recalls getting the Cinefex magazine featuring Contact and reading it as a young artist to work out how they did the famous (seemingly) one take mirror shot, years before she would join Ralston at SPI. “It was a time in the industry, everything was so very much “how do we do this?” – and it was all pretty exciting… we avidly just tried to work out how other companies were doing things,” commented Rowe. “I remember being amazed by that, and I am just so glad to be at SPI now.” Chen recalls very clearly Robert Zemeckis pitching the idea of the mirror shot and “we were all like ‘what?’ Nobody understood at first, we were ‘is it all in the mirror?’ but no, he had it all very clear in his mind,” Chen recalls. Ken Ralston shared that this iconic shot of visual effects was actually not initially designed this way. “There was a Rolling Stones video at that time, with one of the first uses of what would become “bullet time” in the Matrix and it was very cool. We were going to do a version of that, downstairs where she found her father in a frozen position, with popcorn frozen flying in the air, but it was too big, in the sense it was drawing attention to the effects shot, so Bob came up with that beautiful subtle shot,” recalls Ralston.
In 1996 with production beginning on Starship Troopers and Contact, SPI moved into their then state-of-the-art 150,000 square-foot visual effects headquarters in LA. Starship Troopers was released in 1998 and won Sony Pictures Imageworks its first Academy Award nomination for ‘Best Visual Effects’ – the film combined models, miniatures and digital effects. For a long time a giant model ship from Troopers sat in the lobby of SPI.
Robert Zemeckis and Ken Ralston again collaborated on Cast Away(2000). Not everything SPI did is seen by the audience or even makes the ‘making of’ reels. For example, for the award winning Cast Away, most audience members would have been aware of the visual effects in the plane crash, some may have been aware of the blue screen work on the island to make FedEx executive Chuck Noland (Tom Hanks) seem impossible, but few would have known about the goats. Ralston explains that the production shot on a ‘near’ deserted island near Fiji “sounds exciting doesn’t it? But it was so painful (physically) – you have no idea! You get to do a lot of exciting work – boy I haven’t thought about this in a while – anyway we had to paint these damn goats out so it would really be deserted, and they were all over on rocks – everywhere.”
With most visual effects, the Supervisor in conjunction with the Director needs to walk a line between the incredible, but realised as believable. Alice in Wonderland was a huge hit for SPI, and the Red Queen in particular managed to be both believable and yet completely extraordinary, while adding a third dimension, that of avoiding being grotesque.
Ken Ralston very much enjoyed working with Tim Burton on the film, and he was given a lot of creative freedom by the director. For the Red Queen, some of her complex shots (such as the one to the right) were done some four months after principle photography as second unit. For the actress there was no one else on camera. Ralston was keen on set to support the acting choices of Helena Bonham Carter, even if that meant complex unscript additions such as the Queen tasting a tart. The cone choice meant solving her real hand connecting with her oversized head, which made an already hard shot, incredibly complex. “The stuff that works the best sometimes is brute force work, compositing it was just brute force, just going in and beating yourself up to make it work,” explained Ralston. “And all the planning in the world can not foresee that kind of stuff.”
Mark Breakspear, VFX Supervisor SPI, has found that Directors vary greatly in their understanding of visual effects. “There are advantages and downsides to both sides. It is nice working on a set when a director knows visual effects as the crew is normally… low on the hierarchy on set. So if the director knows VFX it gives you a boost on set. But when you are with a director who does not know about VFX, it can be a challenge sometimes to get the time you need to shoot say elements… what it has taught me as a supervisor is that in both cases I try and help them reach an equilibrium where they can get what they want without having to be more involved or less involved than they are comfortable with.” Sue Rowe, VFX Supervisor agrees, she has worked with both. For example on Maze Runner, with Director Wes Ball, sometimes he would just describe what he wanted no matter what, but sometimes, as he has an animation/vfx background and is a nice guy, he would be concerned he was creating extra complex work for say the roto team, “and I’d have to say ‘don’t worry about it, we can handle it’,” commented Rowe.
For Sue Rowe, the technical strength of SPI was a major attraction in working at Imagework as “they have a strong tradition of people investing in ‘how do we do this and how do we do it well’,” she explained, “and not all companies today have that.”
The technical milestones stretch back to the start of the company. For example, in 1996 in James and the Giant Peach, SPI delivered 45 shots, including a flock of digital seagulls, a CG whale and one of the first flowing CG water shots.
In the film Stuart Little, Imageworks solved hair, cloth and water sims for the show. Stuart Little was released in 1999 and also nominated for an Academy Award for Best Visual Effects featured the first-ever CG-animated character to star in a live action feature. For Chen Stuart Little is still, today, one of his favorite films that he worked on, although he jokes about suffering PTSD when watching clips, and recalling just how hard it was to pull off. “We had no guidelines for doing his fur and cloth. His hair took a long time, and our pipeline to this day has grown from that initial work,” he says, adding that he would not be surprised if some of the brilliant code from the hair solvers was not still used in some form at SPI today.
Michael Ford, VFX Supervisor Sony Pictures Imageworks, was an animation lead of the sequel Stuart Little 2 in 2002. “If you go back and look at Stuart Little and Stuart Little 2… not just the way the cloth worked and was simulated but the way it was tailored. Stuart Little is something that I still reference in terms of scale. The fibres were not made to a small scale, they were human scale fibres put on a small mouse, which gave it the appropriate scale and made it just that little bit more real for me.”
Another film that did a remarkable technical job with water (and was one of the perhaps most under rated great animated films ever at SPI ) is Surfs Up (2007). Sony Pictures Animation’s Surfs Up called on SPI to develop an advanced digital camera interface for the “mockumentary” style movie that allowed ‘hand-held’ camera operation to be applied directly to the animation. It would go on to earn an Academy Award nomination for Best Animated Feature. “I was more excited to work on this film than any other I had ever worked on,” comments Ford. “I am a surfer – I was like ‘you are kidding me – we get to make waves and animated characters riding on them’. It was a dream come true.”
This film provided many challenges noted Ford, who credits Rob Bredow, the film’s VFX supervisor, as solving the problem of making the animated documentary virtual cinematography work in a way that animated ‘digital cameras’ had not done before. The solution was to build a real world shoulder mounted camera that was effectively the digital camera in the film, with a simple B/W screen for feedback. This allowed the then Head of Layout, James Williams to effectively become the virtual cinematographer of the film. “It was this sense of being really filmed that worked…it was one of those things where the technology was created ad hoc to get what the directors wanted,” proudly points out Ford.
2001 Hollow Man was also nominated for an Academy Award for Best Visual Effects. For Hollow Man, visual effects artists needed to create a photo-realistic and anatomically correct human with animation and mocap techniques.
Hollow Man was a remarkable digital human, not only for the digital performances but it was done without SSS and so many of the technologies no one would consider working without today. “That film was a relentless effort by Scott Anderson the visual effects Supervisor (and second unit director) and Scott Stokdyk (Digital Effects Supervisor) and Paul Verhoeven (Director). The level of expectations was just absolutely relentless. That film was all done with Rotomation off the performance of Kevin Bacon, no motion capture, just hard thousands of hours of labour,” explained Chen.
SPI was also a great source of inspiring talent that would go out into the industry producing remarkable software. SPI has always been a great hotbed for technical talent. Allen Edwards and Arnaud Hervas who helped found Nothing Real, that made Shake (later bought by Apple), were early employees at SPI. They were later joined by Emmanuel Mogenet as an R&D senior developer, Dan Candela (R&D), Louis Cetorelli (head of support) and Peter Warner (designer/expert user) and finally Ron Brinkmann who joined as a product manager. This core group were all among the original Sony Imageworks employees.
Most famously perhaps was Marcos Fajardo’s development of Arnold while he worked at SPI. The new revolutionary renderer was first used on Monster House. The unbiased stochastic ray tracer, Arnold, was born in 1997 by Marcos Fajardo, and was co-developed between 2004 and 2009 with SPI. While Fajardo left to form Solid Angle (now part of Autodesk), SPI continued parallel and harmonious development on their fork of Arnold in-house. SPI transitioned in 2009 from a traditional biased, multi-pass rendering pipeline to a largely single pass, global illumination system using physically based shading techniques in Arnold. The software would be adopted by SPI and used on Cloudy with a Chance of Meatballs (2012) and many other productions such as Alice in Wonderland, and is still used today.
Michael Ford, VFX Supervisor, looking back at Monster House (2006) recalls how the very limitations of the first versions of Arnold helped produce a distinctive artistic look. For example, originally there was no motion blur, which gave Monster House the almost stop frame in camera look that was a creative characteristic of the film. The Imageworks animation team has always done animated features for outside production companies. With Monster House the production company wanted a more puppeted look. “The render that Jay Redd (vfx sup @ SPI on Monster House ) choose was so limited it did not have motion blur, it did not have hair, it barely had specular reflection… it was very much in its infancy. But what it did have was an unique look, and that is what led to us using it… but even then we had to render the whole film at 1K!” recalls Ford. In 2007 Monster House was also nominated for the Academy Award for Best Animated Feature.
SPI has developed or been central to so many Open Source projects and products, like Alembic, OSL, FLIX, Katana, OpenColorIO and many more. In 2011, The Foundry licensed Katana, the innovative lighting software developed and used by Imageworks since 2004. In the same year the first version of the Open Source Alembic format was released at SIGGRAPH in San Diego.
Sue Rowe, while the only female Supervisor on the panel and a new recruit to SPI, is far from the first senior woman at SPI. For example, Production VFX Supervisor Sheena Duggal was with Ralston at ILM and then for many years ran the High Speed Compositing team at SPI. But Rowe believes visibility of women in senior roles is important. She is also committed to making sure men and women can have home lives and a healthy work-home balance.
Rowe is based in Vancouver and a 20 year veteran. She thinks it is important to not only promote more equality, but acknowledge that software such as Sprout was written by a woman engineer at SPI, Daniela Hasenbring (with Jeremy Hoey). Rowe explains that the notion that women are somehow less technical and better suited to more artistic roles is also an outdated stereotype. Sprout is SPI’s proprietary Maya-based tool for hand-dressing digital environments with large quantities of high-resolution assets like trees, plants and rocks. It was developed at SPI to address the need for an interactive artist-friendly tool that was fully integrated into SPI’s existing pipeline. Prior to the development of Sprout, environment dressing at SPI was done primarily in Houdini or procedurally at render-time and was thus the province of just FX TDs.
More recently, films such as Suicide Squad deployed complex fourth dimensional fractal maths. “It is one thing to take a fractal and texture something, but that doesn’t work with 3D Mandelbulbs. These are complex, in some cases way beyond complex,” explains Mark Breakspear who had to take Chen’s vision for the “cool effects with Fractals” and actually produce them for the screen. Breakspear found a Scottish mathematician who actually had written a program to visualise the Mandelbulbs, but when he contacted him, “He just said no, he didn’t want to have anything to do with us… so we just had to work it all out ourselves!”
In celebrating 25 years of SPI the supervisors were quick to thank and praise all of the artists who work at SPI and who have worked at SPI over the last 25 years. “I have always said, and I can’t say it enough, thank goodness for all the stupendously talented people who have worked with me on projects because they make me look good and like I know what I am doing. I try and give everyone a lot of freedom, and it is just great to watch these talented people come up with a such great ideas and see what they can do,” summarized Ralston in thanking all of the staff of SPI.
Looking forward Chen finds that the tools today are “amazing, and it is exciting every time you get a script and you see what you can do on it. You are not worried about basics anymore, you are worried about advancing it to the next level and that is exciting”.
Revisit fxguide’s coverage of Imageworks and Sony Pictures Animation projects over the past few years