Speed Racer defines a new way of imagining visual effects. Instead of setting the bar at reality, key facilities around the world set out to make something visually new, engaging and never before seen --a sort of live action Anime. In a world of increasingly standard pipelines, they broke new ground as their Racer gathered Speed.
To discover the tech behind the glittering lights of Speed Racer, fxg spoke to lead house Digital Domain, as well as Sony Pictures Imageworks, Buf and Rising Sun Pictures about Speed Racer. Our next fxguide feature story coming out later this week will be a companion piece on rendering 3D cars and metal work.
Digital Domain was the lead facility on Speed Racer. The film has several distinctive visual aspects -- a rich palette of super saturated colours, a rapid Manga energy and an unusual relationship between the live action actors shot on greenscreen and their cinematic background.
The film was primarily shot as green screen on a stage, by DOP David Tattersall (StarWars Ep III, Ep II, Ep 1, Die another Day). As Kim Libreri points out after many greenscreen 'sound stage' films such as Star Wars, the team wanted to give the DOP something to work to. Digital Domain wrote specialist compositing software called Sparky, that allowed Tattersall to see a real time comp on set when lining up limited live action performance plates in virtual worlds.
While Production Designer Owen Paterson (The Matrix trilogy) did design and build some sets -- for example, the Racer household -- they were mainly interiors. Apart from a few non-working prop cars, all the cars in the film are live action cockpits comped into 3D cars. Most of the environments out of the car races are people comped into "real time environments or bubbles as we ended up calling (them), " explains Libreri.
The environments were obtained by the world team headed by Dennis Martin, who went all of the world filming "super resolution quicktime VRs," says Libreri. These HDR photographs were then stitched and enhanced by the Environments matte painting team in Berlin headed by Environments Art Director Lubo Hristov (300, Mr & Mrs Smith, Matrix Revolutons).
One of the most interesting aspects of the work is the virtual backgrounds 'accuracy'. Many on this team had used this technique extensively on The Matrix films. However, unlike those films which used the spherical HDR, bubbles to make 3D environments, the Speed Racer environments use the actual spherical mapping and do not reconstruct the entire space back in 3D. This gives the backgrounds a slightly modern Anime feel the film makers thought. In doing so, the backgrounds were also made to move creatively in ways that were not mathematically accurate to the foreground.
"Doing HDR panoramic environments is something that this group of people - this group being the people who made The Matrix movies - has been doing since the first Matrix movie," says Libreri. "But what we did was to make an effort to make it not look as if we were using panoramic photography." In other words, in films such as The Matrix, HDR panaramas were used to make photoreal backgrounds, that looked correct and responded correctly to foreground camera movement. This is not the case in Speed Racer, the backgrounds reflect more an old animation stand, and "....when we placed a character in the middle of the HDR bubble, that was shot for that particular location, instead of putting 3D geometry in there to make it look real as the camera dolly through the environment, - we'd just leave the bubble there- and if that breaks too much or looks too strange, then we'd start to split objects off". This approach leads to shots that may not be considered 'correct' but feel true to the original anime style.
So while some 2D cutouts were used in 'bubble rooms', for example, primarily it was just the bubble background. The filmmakers fully acknowledge that this produces some artifacts that you would not want on a normal film. However, as this film is aiming for something new...something akin to a digital live action Anime...they feel the look is valid. It is also a nice reference back to the rostrum effects of the original TV show. The net result was 2 and 1/2 digital collages, and when characters moved between
On the whole, the car races did not employ the 'bubble' approach although this was somewhat used for the races BUF and SPI worked on (see below). Digital Domain's main car races were the Thunderhead race at the start and the Grand Prix at the end of the film and these were rendered as traditional 3D.
The DD team used an advanced ray tracing global illumination, image based lighting approach to produce the level of visual accuracy needed and photoreal cars. But the ray tracing comes at a price, and that price is render speed, especially with vast amounts of geometry in the scenes "certainly more than Digital Domain has dealt with before, perhaps more than most companies have ever dealth with" he adds.
On top of the hard geometry of the race track and its environment, someone at DD calculated these massive data sets also had some 2.1 million digital crowd members watching the final race. All of which needed to be motion blurred -- both motion blurred accurately and as a stylistic decision and technique. To do this, a combination of Mental Ray and DD's own 2.5 post processing motion blur was used. Doug Roble at DD wrote a new motion blur tool as the actual motion blur varied on different objects in the same scene.
Motion blur is caused by the duration of the exposure which is turn is controlled by the angle of the shutter on a traditional film camera. During post, the Wachowski Brothers along with VFX supervisor John Gaeta and VFX supervisor Dan Glass would set individual shutter angles on separate objects. They would request in dailies "make that car 45 degree shutter, that one 120 degree shutter, cartoon motion blur on that, back on 120 degree here.. etc " explains Libreri. They had "multiple combinations of the shutter angles for the motion blur on multiple objects in the scene, and if the Mach 6 blew by the camera, then they may have a combination of 180 degree shutter angle, mixed and blurred with video shutter angles", so you would see the Mach 6 cleanly but with trailing motion blur speed lines. This was done in the composite in Nuke.
The project was shot on the F23, with a large depth of field. Initially the film makers were not going to have such a stylized look to the film, but during testing the Digital Domain team found they could get the F23 to look 100% real and just like film, an alternative was explored, that of the hyper-saturated, low grain, vast depth of field look.
During the shoot, DOP David Tattersall used the Hyper gamma 4 curve in the F23. The Hyper Gamma 4 is type of modified REC709 gamma curve, more like HD than Film. The files on set were recorded into the new Codex store, as 10 bit DPX log files. By using the Codex they could record DPX but still on set monitoring with standard HD monitor but also get DPX files so the vfx team could composite on set. The Hyper Gamma 4 Curve effectively "brings down the shoulder of the cameras response so it still fits in a standard Rec709 video file " explains Libreri.
During actual production, Pacific Title did the data management and F23 SR conform of the images from the shoot. Digital Domain worked in partnership with Pacific Title to create all the colour management profiling for the movie. This lead to the F23 1920 x 1080 images being distributed to all the vendors with a standard Speed racer LUT applied, giving all the vendors the same standard LOG DPX files as a base line.
At Digital Domain the internal pipeline started with these standard DPX log files, which were then converted to linear, the CGI was rendered as multichannel openEXR, and the compositing was done in linear space. For viewing, the linear files were watched with viewing LUTs that went from the linear files to Speed Racer's "toe and shoulder" curves. These curves are similar to film curves but without the cross talk and desaturation that would would expect from a film 3D LUT.
In fact, it had the opposite effect built in - the viewing LUTs had increased saturation, but this was not 'baked' in earlier in the pipeline in case DI wanted to vary the levels. This would make the images loom contrasty, strong and saturated for director review. When the shots were finaled the files were converted with a much more standard linear to log LUT and passed to Pacific Title who was also handling the DI grade. This linear to log was different from the viewing LUT used earlier so that the images could have the super saturated look added in DI and it would not be baked in - just viewed as such while still at Digital Domain.
Due to the standardized apporach to colour management Libreri proudly points out that there were "no colour screw up at all" on Speed Racer and for what was a complex pipeline, the whole system worked without fault.
The deliverables on the film were 1920 x 1080, but with a safe 2.35 within that. To render sequences, some elements would be much higher resolution such as 4K, but Digital Domain' standard render was 1920 x 1080 with a 10 % over scan to allow some room for camera shakes or minor repositioning.
Digital Domain found working with the digitally shot footage to be very helpful, both due to the low noise structure and the excellent lighting of DOP Tattersall. Digital Domain estimated that some material was able to be keyed in a 1/ of the time film green screens are keyed. We had "green screens like I have never seen before". "the F23 is super sharp, with very little noise if exposed correctly". So much so that some 'digital promist' was added back in to help the actors skin soften a little.
Sony Pictures Imageworks
The Wachowskis turned to Sony Pictures Imageworks for three racing sequences in Speed Racer that represent a blend of 2D, 2 1/2D and 3D techniques. Supervising these 226 shots was Academy Award-winning VFX Supervisor Kevin Mack (What Dreams May Come). His parallel career as an abstract visual artist provided the sensibility to deliver a super-rich color palette to accompany Speed Racer's hyper real pop-anime environments and vehicular Car Fu gymnastics. fxguide asked Mack to reflect on his time on the track.
fxg: Kevin, how did Imageworks become involved with Speed Racer?
Mack: They were pretty well into it when we became involved. There were three really meaty sequences that included a desert race with cars flipping over and doing their car-fu, an alpine race and then they move to being up very high in a snowy environment, with very rocky, craggy mountains. Speed winds up going off-road and driving up and down icy cliffs. It's pretty crazy, its cool!
fxg: There was a pretty extensive testing and pre-visualisation process conducted by John Gaeta and Dan Glass and the VFX team for the Wachowski Brothers. Did they have the production pipeline all mapped out when they came to you?
Mack: More or less, although we took on stuff and just put it together. The big thing they figured out was the previs and they stuck to it. There wasn't a lot of changes to the cut or the shot. It was more or less as previs. They gave us access to a huge amount of data in terms of the bubbles (high res virtual panoramas) they shot and different environments and so we constructed 3D and also our bubbles from that material. They were going for such a stylized look and a supersaturated look that while we used these photo bubbles to begin with they wind up being conglomerations of different ones and then ultimately they wind up painted pretty severely just to get all the colors and drama and lighting and everything that you want into the individual sequences.
Mack: Yeah the bubbles needed to be pretty heavily treated and art directed to get them to be what we wanted them to be. It was a far cry from starting from scratch, because we were given some great panoramas with amazing skies and mountains. In some cases the panoramas had actually been used for the previs so we had a good starting point. Our sequences ended up including a bit of everything: there are shots that are purely the "bubble" some that are 2 1/2D, and stuff that was mostly 3D. The cars are 3D and the roads they're on for the most apart are 3D and its 2D past a certain point. The desert flat is a totally 3D environment. The sky is a bubble but it generally is in any kind of CG environment. We've got mountains and dunes in the distance that are all heavily processed and then the ground is a really elaborate shader we developed that varies throughout the race. In the desert sequence, they travel across a lengthy stretch of desert so there's patches of mud flats, salt flats, gravelly stuff and sand. We really wanted to vary it because otherwise it's just kinda flat. It just becomes a color.
fxg: I understand that in many of the live action sequences they were using multi-layered "bubbles" to create a faux-3D effect?
Mack: We did that as well. So for instance we would build a mountain cliff and then off in the distance we would have multiple layers of bubbles with mountains projected onto them and painted and then the sky so it was a bit of everything really. There are also the conveyer belt gags where the road is moving and it just loops around in a circle. There are cases where they drive on a very long stretch of road that maybe curving and you have the background going by. The road is on a conveyer belt, its looping. Its 3D but it just keeps going.
fxg: You created the 3D animation of "Car Fu", showing the cars battling it out in the race sequences. How extensively was this pre-visualised?
Mack: For the most part the Car Fu was pretty well worked out although of course in the previs the animation is pretty crude so it doesn't have any of the dynamics or any of the subtlety added to it. The action of the cars and the cameras were pretty well laid out. One of the main challenges was matching the live footage shot of a practical cockpit on stage with the animation. There was some flexibility there but in some cases not that much so you couldn't get the cockpit to register with the car. A lot of it is just CG drivers and cockpits.
fxg: For the desert race sequence, for instance, what elements were you provided with before you began work on the shots?
Mack: The desert didn't have much of a bubble. They had a sky back there but theirs was pretty simple. We created the ground and the distant mountains and the sky. There's not much too that because the ground is flat but the ground shader was cool because there's a lot of variation. They had done all of their previs for the car action on what they called a stage mover. The cars aren't actually moving forward, the texture of the ground is moving underneath them. It's kind of a relative thing. We had to reanimate all that stuff but it was very well designed and organized so that we could just go in and do those shots.
The film is inspired by anime, and in anime films one of the cool things is anytime there's any kind of explosion or a cloud of dust or a smoke trail the way they animate those effects is very stylized and it has this cool kinda flowing but solid nature to it. So we wanted to get something that at least suggested that but also had a certain feel of reality. So for the dust clouds left by the car in the desert flat race we created a fluid simulation for all the dust and then shaded it in a certain way and lit it so that it was fairly dense and yet its movement was very realistic and complex and yet had this slightly anime quality to it in the away that it's lit and shaded.
fxg: John Gaeta describes this movie as an experiment that aims to develop something unique, something he describes as "photo-anime"
Mack: The big thing is the saturation of the colors and that was something I was really into. I make abstract fine art (www.kevinmackart.com) so I'm really into the supersaturated colors and I've worked out various ways to sort of cheat and make them more saturated. In dialing up the saturation at a certain point it breaks, it turns posterized and doesn't look good so I found little tricks for isolating specific hues and saturating them individually. And then also just in the design of the colors, like for the desert race, I pushed to get this variation in color on the ground and then also in the sky.
You'll notice that there is a gradation in the ground, you always have little bits of gold and orange and then red, so you've got this gradation from gold to orange to red, these analogous colors, and then in the sky you've got a gradation from the horizon up to the top part of the sky that goes from turquoise to blue to purple and so each of those is the complement of the other on the ground and its a kind of interesting neurological phenomenon with that particular combination.
I call it a super complement because you've got triple complements working with each other and they're centered around blue and orange which I think are the most primally effective complementary colors. Since the time that we crawled out of the sea we've been seeing the orange earth and the blue sky, also with that gradation in it, from turquoise on the horizon to blue purple at the top, and the same thing on the ground the variation from gold to red. So it stimulates something primal in us I think. That's why those colors are so powerful.
fxg: How detailed a color palette was provided to you from the producers?
Mack: There wasn't much for the desert race. We went through a few iterations on the ground texture design for instance. At one point they wanted it to be black sand, at another point it would be salt flats. I suggested that it have this variation in it and that it have these complementary colors and they really liked it so we went with that. On the Alpine sequence, I was going for this feeling of warm and cool again. I wanted to have nice blue shadows next to the warmth on the cliffs and the mountains, where the light's very warm, and so again you get that same super complement where you get the blue and purple and turquoise combine with the gold and orange and red.
fxg: For the race sequences, were there any challenges integrating the greenscreen cockpit shots of the drivers?
Mack: Sometimes when we were doing a shot that they'd prevized we'd realize we're getting too close to this car and we're gonna see the driver. So we'd go to the producers and they'd find some element that would fit or we could rework to make do. When you shoot a whole racing movie with guys in cockpits you wind up with a lot of angles of each of the hero drivers. We're talking about quick cuts where the cars are racing by and you only see them for a moment. Sometimes in these cases we could use CG and you couldn't tell but sometimes the camera got too close and we didn't want to have to chase after creating the prefect CG human when you're not really getting a good look at them.
fxg: What resolution was the HD footage you were supplied and what resolution did you work at?
Mack: It was 2K for the most part, but some of the bubbles were over 10K, because they were stitched panels so they were pretty big. They also shot tons of plates, just elements, textures, mountain cliffs and road textures, distant mountains and skies. It wasn't a challenge to find good material to work with. You get something photographic laid in there and saturate it up and then you can decide if this side of the mountain were better shadowed, or you need to add detail in or there's too much snow or grass here, so you wind up customizing each piece. We had a matte painting department that was very busy for quite a while.
fxg: We've concentrated on the desert shots, were there any other elements to the other shots that were unique to them?
Mack: The alpine sequence we did some great things there where its an interesting combination of 2D tricks and the virtual reality bubble and then really crude 3D geometry all the way to more detailed 3D. Even when its 3D we were projecting textures and lighting onto them. So rather than giving them a shader and lighting them in 3D in a lot of cases it's just a painting that's projected with the lighting built into the painting. I
t's sort of like projecting matte paintings from multiple angles to create a 3D environment. The mountains have neat roads with crumbly retaining walls with aged rocks and the bushes and craggy peaks in the distance. There is some neat vertigo for some of the angles. Cars are grinding along the edge of a guard rail and hanging over the edge and you see down into the valley below and off in the distance you see other cars on switchbacks. You get vistas with crazy views of exaggerated mountain peaks. Lighting in that sequence I was really happy with. I pushed for idealized lighting where cars and roads are always half in shadow and half in bright sun so you get this play of colors and they're always going in and out of that.
fxg: Were there any specific developments to the Imageworks production pipeline for Speed Racer?
Mack: We did an awful lot of matte painting projection, but we've been doing that for a long time. I did a lot of that on Apollo 13 back in 1994. We did develop our car shaders and the ground shader. For us I think probably the biggest science project was the dust trails in the desert sequence, the fluid simulation getting that look down. Although I think that what was unique about the pipeline was that it was utilizing all these different techniques, from 2D gags to 2 1/2D to 3D and the boundaries between those really dissolved so you wind up just doing what you need to do for any given shot.
The pipeline was setup in such a way that you were only doing or building what you needed to. So often on projects if you are doing a 3D environment you model the whole environment and then you write shaders for all of the surfaces because you don't know what you are going to see or what angles you are going to use, so often things get essentially over modeled and overworked cause you don't want to be caught if the director says he wants to change an angle over here and you don't have shaders over there, or you haven't built that part of the model. Here the previs enabled us to just texture what we needed to and so much of it was able to be just projected textures and paintings rather than shaders and lighting.
For the cars we used some deluxe multi layer surface shaders with double highlights from the gloss coat and bottom coat and iridescence and all those things. I think we take that for granted because they've been doing that in car commercials for years. So we were just basically applying all of that technology. I know they did some fancy stuff at Digital Domain with capturing BRDF (Bidirectional reflectance distribution functions) for the car surfaces. We went a little more straightforward but I think we got the same look really. Its very real and at the same time kinda hyper real. We would exaggerate reflections and create reflections that wouldn't really be there and add fake highlights. Something John Gaeta talked about a bit was trying to get that car commercial look.
Rising Sun Pictures
fxguide spoke to VFX Supervisor Tom Proctor at Rising Sun Pictures.
fxg: Who is Cruncher Block? What scenes is the truck involved in?
rsp: Cruncher Block is a notorious criminal figure in the racing world of Speed Racer. Our scene opens in his office, which you come to realize is located in the back of a big rig that is barreling down the Lonely Highway. The mobile nature of his office is revealed when one of his goons opens a hatch and peeks outside. There is a big exterior gunfight between the truck and Racer X's car, which ILM did. We handled all the interior shots including the stylized slow-motion shots of the bullets coming through the walls.
fxg: I see from the credits he was played by John Benfield. Did you have to match move the interior to cockpit shots of the actor, or just supply the interior? (was it just a cabin?)
rsp: The entire interior of Cruncher's office is practical. It was a set piece built on a rocking base, so that throughout the sequence the rocking builds up until the characters get thrown around. We created the CG piranhas that are Cruncher's pets, that he keeps in a large tank behind his desk. As you will see, he uses them to intimidate his enemies. The tank was practical and sloshed around and we had to animate the fish to match.
fxg: How did you evoke the vibrant, candy-colored world of "Speed Racer"
rsp: The Cruncher Block set piece was built with a strong color scheme and we didn't want to clash with it so we developed the look of our CG to compliment it. The piranhas reflect a lot of the blue light coming off the back of the tank and have bright green tones in their scales to fit in with the walls of the office. We also gave them bright red eyes that flash and flicker like cats' eyes. With something like a dozen VFX vendors worldwide, the overall digital color management of the show was critical. We lit and matched all CG to our plates and then viewed our comps through a LUT that boosted the overall saturation and vibrancy.
fxg: What scenes did you provider matte paintings & set extensions for?
rsp: There is an exterior sequence after the gunfight between the truck and Racer X where Racer X picks up Taejo (played by Rain) and they have a conversation. Matte painter David "Woody" Woodland did matte paintings of the forest and tree-lined roadside side of the Lonely Highway. When Taejo and Racer X are in the car together driving along, we created a cycling background in the manner of a traditional animation - repeating the tree elements and sliding the matte painting layers along in 2d as they drive. The simplicity of the technique was intentional and an homage to the economy employed in anime. We also added some glowing anime-styled "speed lines" that whip past to accentuate the motion.
Many of the scenes in the film employ synthetic backgrounds and for these the production employed a "bubble" technique- a series of tiled photos of a location would be used to create a spherical mosaic image that could then be rotated around to match camera pans on a green screen stage. You see a similar technique used in Quicktime VR movies. Another sequence we worked on took place in the office of Togokhan, Taejo's father, and had a "bubble" background that originated in a modern building in Berlin. For that scene we also created a futuristic screen graphic of a koi floating over abstracted stock exchange data.
We also did some shots in Royalton's office in Cosmopolis, using a background image of the skyline supplied by Buf in France.
fxg: What were the visual references you were provided with?
rsp: For the piranhas, we were given some fantastic storyboards by artist Steve Skroce. The original piranha design was based on a sculpture done by the art department but we ended up tweaking it out a bit.
fxg: Where to the CG piranhas appear. Are they Rising Sun's first piranhas, did you model and animate them from scratch?
rsp: The Piranhas appear in Cruncher Block's mobile office. I believe they are Rising Sun's first piranhas. They were based on a sculpture provided by the film's art department, but our artists Ben Paschke and Dan Thompson built the CG ones and ended up getting to enhance the design a lot.
What was the context, i.e. what were the unique challenges involved in creating the piranhas for Speed Racer. Originally, the film called for 2d piranhas but we got carried away and made 3d ones. They looked really good and scary but that one extra dimension can be an expensive and time consuming one if you aren't careful. Our lead animator, Victor Glushenko, and his team worked magic and put a lot of life and humor into the little fish. Another factor is that the piranhas were created in our Adelaide office and I was supervising them remotely from Sydney for most of the schedule.
fxg: Did you have to integrate with the 360-degree panoramic backgrounds known as QuickTime Virtual Reality (QTVR) spheres, also informally referred to by the "Speed Racer" team as "bubble photography"?
rsp: Yes, for the Thunderhead sequence, Togokhan's office, Royalton's office, and the exterior Lonely Highway shots.
"To render the film's myriad of visual effects shots, the Wachowskis achieved what they called a "live-action anime look" using a visual-layering technique that allows the foreground, mid-ground and background to stay in focus, much like that of traditional 2D animation. This technique came to be called by the filmmakers "2½D technology." - Did this apply to any of the sequences Rising Sun supplied?
This technique was applied selectively. In some places, the background was defocused to emphasize foreground dialog or action.
fxguide spoke to Pierre Buffin, Geoffrey Niquet, the VFX supervisor and Simon Vanesse, the producer at BUF.
fxg: How did you approach the city?
BUF: The approach is very old-school. A city map was laid out and each building manually placed to keep the naturalistic feel. The production provided drawings of major buildings. 26 hero buildings were built, then the secondary ones were added. Hundreds of different 3D buildings were constructed. A software was developed to convert 3D camera moves in 2D layer animation. This created a complex anime look to our shots.
fxg: What tools did you use?
BUF: The whole production pipeline at BUF consists of homegrown software. Since its beginning, the in-house R&D team develops our own proprietary tools. These encompass all stages of production; from tracking to rendering, including camera calibration, setup, animation and modeling.
fxg: Was there some key aspects of the realism that you struggled with?
BUF: The conflict between the initial creative idea of a 2D film and the need for action. Indeed, the 2D approach breaks the speed effect, meaning the insider's look into the crazy race.
This original idea had to be adjusted to the creative direction of a more dynamic action movie. So, from the initial manga look, we ended up on some shots with 3D camera moves that allowed to achieve that dynamic.
fxg: How did you find the F23 Log files to work with?
BUF: We didn't encounter any problems. Actually, the higher quality of the data was very helpful for all the many shots that involved green screens.
fxg: How long and how many people worked on the project?
BUF: BUF was involved in project very early on during the summer of 2006. There were tests set up before the actual shooting. Their purpose was to research the best ways for reconstructing sets from stills. During the actual production, our team reached 120 people.
fxg: Was the stylized world a hard one to get right balancing realism and anime?
BUF: The 3D with 2D treatments allowed for this stylized effect. We also mixed the QTVR bubble technique with BUF's famous camera mapping technique.
We've been a free service since 1999 and now rely on the generous contributions of readers like you. If you'd like to help support our work, please join the hundreds of others and become an fxinsider member.