What does it take to win an Oscar? For a start, be part of a VFX team at a company that has already won 15 best visual effects awards. Then work on a film that is both entertaining and also pushes character animation, large scale compositing and digital environments to a whole new level. Oh, and being nice guys also helps, as we found out.
Pirates of the Caribbean: Dead Man’s Chest (2006) holds many box office distinctions, such as:
– highest opening weekend of all time ($135.6M)
– fastest film to reach $100M (in just two days)
– most tickets sold in a single day ($55.8M worth)
– and largest sum earned in seven days ($196M total).
The blockbuster sequel also reached $200M in eight days, tying the industry record. Add to that it was the Academy Award winner this year for Best Visual Effects.
The question is: can they top themselves for the sequel due in cinemas May 25th? With Chow Yun-Fat joining the cast and Keith Richards making an appearance as Jack Sparrow’s father, the franchise seems on track to have an even bigger sequel.
Oscar Winning Work
So what does it take to win a Visual Effects Oscar… ?
When ILM won the Oscar recently, some writers said that it broke a drought for ILM as the company had not won in the Best Visual Effects category since Forest Gump 12 years earlier. To say this is to somehow indicate that ILM was in a slump, which is so monumentally inaccurate as to be farce. In the period between Gump and Pirates, ILM was nominated every single year except one and it was often nominated for more than one film in any given year.
To have a company be nominated for an Oscar is amazing enough, but multiple nominations year in year out, on top of already having 14 Oscars going into last Sunday’s Awards is totally amazing. But then almost everything about ILM is amazing.
In a world where freelancers and people move from project to project, the three senior compositors/supervisors we interviewed (Grady Cofer, Chad Taylor, and Mark Casey) have all each been at ILM for ten years – Taylor for 15 years. Their credit list is impressive to say the least and includes films such as Munich, Master and Commander, T3, MI II, Bourne Supremacy, Men in Black, Saving Private Ryan, AI, and more.
The depth of talent at ILM is second to none. fxguide’s contact at ILM who handles Technical PR, Greg Grusby is himself an ex NY Flame artist. When your highly skilled full time technical publicists are even flame artists you know the company knows what it is talking about.
In fact, everyone we met at ILM was open, friendly, insanely talented and seemed to love working there. Which is not surprising perhaps since the Letterman Digital Arts Center facility that is its home is perhaps one of the nicest facilities in the world. Custom built on federal land of a former military base, it is open, airy and has amazing views of the Golden Gate Bridge.
The Premier Screen cinema in Building A is where we saw ILM’s work projected on their 2K digital projector. It offers the most pristine and amazing cinema experience one could imagine. Actually, to tell the truth, it is better than you can imagine. The theatre is well known for its quality and it is the home for ILM dailies to both artists and directors.
However, it is not until you see a few shots that you think you have seen before in the cinema and on DVD that you grasp how much detail is being lost in a Cineplex projection. Digital projection without weave or float adds a lot, but the whole design of the cinema is so perfectly matched that we were seeing small details and aspects of shots we had never noticed before.
This lead us to ask a the artists if it was not a bit intimidating to see their work projected so cleanly on such a huge screen. We were surprised to learn that not only does the projector allow frame by frame stepping and jogging — which would be bad enough for scrutinizing a shot at dailies — but it had been recently updated to include a zoom feature.
The really impressive stats are the volume, quality and sheer size of the effects work done at ILM. The company has done roughly 250 films and now routinely works on multiple major films at once. At the time of our visit, ILM had five major films in production but more importantly three huge films underway. Pirates of the Caribbean: At World’s End (May 25), Harry Potter and the Order of the Phoenix (June 13), and Transformers (July 4) are being delivered within about six weeks of each other.
To pull this off, ILM has clearly one of the most dedicated and highly skilled IT computer divisions in the industry. We visited the render farm under the Letterman Digital Arts Center which is the source of the entire division’s computing power. Even the digitally projected clips we watched in the Premier Cinema were played off the servers directly.
If you write and visit a lot of facilities it is easy to become blasé with equipment room ‘walk through’ visits, which typically involve opening a door on a fairly big room full or tape decks and racks of computers. At ILM the Equipment room is like a bunker someone built for one of ILM’s own Sc-Fi military Terminator style movies. In fact if someone had told us this was the home of Skynet – we’d have believed them.
When fully pushed ILM has 7000 processors in the farm – including every desktop machine in the artists areas above when they have logged out for the night. The network is able to push through a Petabyte of data in any one 24 hour period – all with 220 terabytes of online storage. Many of the computers now are dual-dual core machines housed in vertical racks. They run so hot that ILM needed to space out the racks to allow the enormous heat to be dispersed by the jet engine style air conditioning systems. All this services over 500 artists in floors above finaling shots at a rate of about 75 finals a week – just on Pirates 3 alone when we visited.
Still, you don’t get an Oscar for impressive stats, you get it for impressive work, very impressive work. This year, Pirates won against ILM’s own Poseidon and the equally impressive Superman Returns. So what was it that the voting members saw in the film and learned about at the bake-off that lead to Pirates 2 being awarded?
If we leave the entertainment value of the film and franchise to one side, Pirates did many technical things exceptionally well, and a couple for the first time. Clearly the film was a big budget project, filmed on location with vast sets and complex physical effects. For example when the Kracken breaks the ship in two, a near real sized ship – but without masts – was crushed and smashed by giant blue screen poles or pylons. The team then composited miniature masts, sails, actors and effects to make it seem if the giant beast was destroying the ship. But it is the crew of Davy Jones that perhaps more than any other set of shots seemed to capture the collective imagination.
Motion Capture
One of the biggest technical aspects of the film was the outstanding character work done by the animation team at ILM. In a unique move, the production decided to shoot real actors on set and then digitally replace them. To do this each actor was scanned and modeled. Initially the plan was to use the actor’s real eyes, and thus the actors on set had eye makeup. But in reality all of the eyes in the film are digital and no real part of actor remains in the final shots.
The biggest breakthrough technically is perhaps the use of single camera motion capture. Most motion capture is related to 3D tracking: multiple camera views or camera positions are used to triangulate the position of a series of markers. These complex data streams are then used to reconstruct the motion of the actor.
However, traditional motion capture has multiple issues. First, it is often done in special studios with multiple cameras. Next, the cameras and tracking markers are special expensive equipment used only in a calibrated environment. Third, the data needs to be cleaned up tremendously, as the data stream has both noise and errors. All of this makes the process complex to set up, expensive to use and highly specialized.
ILM’s breakthrough came from reducing this to a single onset film camera. This removes so many of the restrictions of the motion capture process. Not only does the system allow motion capture from one camera but this can now be done on set. The approach is to model the actor’s range of motion and then use an elaborate system to fit the range of possible motion the actor could do, to the data from the single camera source.
As the computer knows what the actor’s limbs could do from any one frame to the next, it can ignore a lot of mathematical possibilities and add to the solution. Once the solution is constrained by this virtual range of possible motion, a single camera can produce a very powerful motion capture data stream. While the motion capture system worked extremely well, the lip sync was not done this way and instead hand animated.
Sabre compositing
Director Gore Verbinski was himself once a compositor. Verbinski gained fame directing commercials before feature films including the famous Budweiser ‘frogs” Super Bowl spots. Below is a sample of QuickTime clips showing the range of elements that the Sabre team would have to work with. In this shot, the Pirate ship sinks and sails off beneath the ocean waves.
The primary live action and motion capture plate
One of the secrets of ILM has always been to use the latest CGI with live action elements. And so while ILM has Oscar nominated water simulation tools, many of the shots in the film such as this are based on extensive live action photography.
The Stunt Ship submerges, note the lack of masts
This shot was done by the ILM Sabre team, based on Autodesk Flame’s and Infernos – the unique ILM system allows the Sabre team to work in an Open EXR Float workflow even through Flame itself does not work in float. Part of the ILM solution is to have special ILM sparks that allows ‘float like operations’ while only using 12bit files, yet still behaving as if full float operations.
One of several water passes
The Water passes were tracked and then the multiple live action passes combined with CGI elements such as the mast and sails, and the motion captured crew. All the CGI elements were rendered as OpenEXR files but in the Sabre system. ILM received a technical Oscar this year for its development of the OpenEXR standard.
Foam element filmed with the scale ship
Water is known to be extremely difficult to scale and to composite. Pirates 2 represented some extremely difficult composites. The 3D for Davy Jones’s beard has 40 ‘living’ tentacles. As they are always moving, software automatically avoids crossing the tentacles with collision detections, and even simulates friction but the system also allows for detailed individual animation.
The primary water element – note the natural spray effect
Additional fuild elements were added in the final composite (see below) but many of the new elements were added and built on the natural spray and dynamics seen in this clip.
The Final shot of the ship descending below the water
The final composited shot of the ship descending, with extensive character animation adding both depth and humor to the shot. Note the subtle aspects of Davy Jones’ (Bill Nighy) performance.
The Kraken
The Kracken provided a huge set of issues – literally. The Kraken was required to crush the pirate ship, and this required a large ‘Kacken sized” pair of blue screen ‘tentacles’ to fall and crash the ship in half. The following QuickTime movies show elements from the shoot.
The Kraken cgi elements before and after comparison
In the final scenes where Johnny Depp was sprayed with Kracken slime, the team discussed how challenging it was to key slime from near white from the overcast lit sky.
Dramatic B-roll footage of the Destructive power of the Kraken
In this clip the complexity of the keying problems can be seen, as CG is placed in the scene, behind smoke and water spray, in front and behind actors. mixing live action actors and CG actors in one exciting sequence.
Darn the links don’t work
Fixed the links/movies, thanks for the heads up