This week we look behind Oliver Stone’s newest masterpiece, World Trade Center. Double Negative was responsible for recreating possibly some of the most iconic footage of the modern age. It is one thing to recreate or create mythical worlds or fantasy environments, it is something else to reproduce images that are still extremely vivid in the popular culture, and obviously very sensitive.
World Trade Center is based on the real world experiences of John McLoughlin and Will Jimeno who were trapped as part of the September 11 attack. It is directed by three-time Academy Award winner Oliver Stone and produced by Michael Shamberg and Stacey Sher who have produced other real world films such as “Erin Brockovich”.
Shamberg and Sher were struck by the way in which the experiences of the two men spoke to larger themes. “The story of John McLoughlin and Will Jimeno and all the people who helped rescue them is just one story from 9/11,” Shamberg nootes, “but it shows the larger story of how on a terrible, tragic day, people risked everything to help each other. We must remember that.” Sher adds that “It appealed to us because it is about heroism in the sense that it reveals the best in humanity as people came together to help each other.”
From the beginning, the filmmakers’ mandate was to make a film that not only honored the men and women involved in the story of McLoughlin and Jimeno, but told it accurately. This meant including not just the two principals, but their wives, families, and ultimately as many people connected to their rescue as possible.
To bring McLoughlin’s and Jimeno’s story to the screen, Oliver Stone brought together an outstanding team of professionals. The director of photography is Seamus McGarvey, and a two-time Oscar nominee production designer, Jan Roelfs. The primary visual effects house was Double Negative.
The primary visual effect was the recreation of the World Trade Center itself, including the collapse and the aftermath in the following 24 hours.While footage was shot in New York City, much of the filming was done in California which presented Double Negative with some challenging set extensions.
The construction department began resurrecting the World Trade Center while the shooting crew filmed in New York, in order to have it ready by the time Stone returned to California. Devising the set was a challenge for production designer Jan Roelfs because the collapsed towers were very well documented in photographs and on television. While this offered a plethora of research material, it also meant that everyone in the world had seen and remembered the original and no mistakes could be made.
Moreover, Roelfs had to configure a construction that could accommodate the needs of the camera and lighting crews, as well as the creative requirements of Oliver Stone. “I knew what Ground Zero looked like, but how to make it into a set that was shootable and affordable? That was one of the biggest challenges, Roelfs says.
“There was so much documentation of the Ground Zero site so that was very helpful, but, it spanned over 16 acres and that was too massive to build to scale. We started with models. As the set began to take shape, it was clear that there were certain iconic pieces that we would use as landmarks, stark pieces of the buildings that remained standing that were in many of the photographs. Then we had to build it and make it camera ready but also safe. By the end of construction, the set contained 200 tons of scrap metal and 900 individual sculpted pieces and spanned about an acre, 1/16th of the original rubble field.
A year ago, D.Neg completed Batman Begins, during which they developed a complex city pipeline supported by a large database. Based on taking hundreds of digital stills, forming HDRs and creating complex stitched panoramas and reconstructed buildings, It was this pipeline that was used to rebuild New York City.
Once again the Double Negative team took 500,000 digital stills of New York City and then set to rebuilding it and adding back in ‘Ground Zero’. In an amazing piece of pipeline management, the team managed to massively accelerate their development and production cycle allowing for huge visual effects shots to be added with only weeks to go before the end of the film. For a full discussion of the pipeline listen to this week’s podcast. We talked to Pete Bebb who was the CG Supervisor on the film, and Dynamics Supervisor Ryan Cook.
Bebb started at Double Negative in 1998 as Double Negative’s first runner! He has since worked in several departments building up to being a CG supervisor. He also worked on Batman Begins a year ago and many other productions as 3D artist, team lead and sequence supervisor. Cook started his career as a Research and Development Technical Director at Industrial Light and Magic, before joining Double Negative in January 2003 as a digital artist. Cook has worked on several major motion pictures including Star Wars: Episode II, Signs and Alien vs. Predator.
The Twin Towers were a major build of their own. Without detailed images of the World Trade Center it had to be completely CG. The build in itself was not very complex but an inherent problem with linear structures such as the towers is that they create a “moir” effect (a crawling pattern seen when parallel lines are rendered very close together). In order to counteract this, the environment had to be rendered in 4K, and sometimes 8K, creating a huge rendering task.
Also, because the buildings were very regular in appearance, it was extremely difficult to make them look real. Significant detail was added to overcome this, such as the specific way the panels reflect the light and seeing the interior of the building through some of the glass windows, all of which added another layer of realism to the image. The team also had to build CG New York streets which were composited into live action greenscreen footage shot on streets in LA. Double Negative also worked extensively on dynamic smoke and debris effects from the Towers, at Ground Zero and in the surrounding streets.
A huge amount of work was done on researching and building a complex smoke simulation library based on some earlier work for Batman Begins. Several proprietary Maya plug-ins were written or amended to cope with the job requirements. These included rigid body engines and voxel renderers, and Double Negative’s own inhouse renderer. The Double Negative WTC smoke library consisted of all types of simulations from big, heavy, large-scale smoke clouds, to lighter, gossamer wisps of smoke; a thoroughly comprehensive set of fluid simulations that more than merited the amount of work it took.
As Cook points out, simulations from the library were used as a starting point in around 95% of the shots. In addition, Senior Technical Director Graham Jack wrote a script called gto-retime. This tool enabled the artists to post-process (speed up or slow down) any of the simulations without having to re-simulate, which could take up to 7 days, adding even more diversity to the elements available for use. This amount of flexibility was necessary as the smoke also changed during the course of the day, says Ellis. “To start with it was thick, black and oily and as the day wore on it seemed more light and creamy and more like dust and particles, he relates
The work on the smoke didn’t stop there. The R&D department, lead by Jeff Clifford, set about re-writing the company’s proprietary voxel (a kind of 3D pixel) renderer, DNB. DNB was originally devised for Batman Begins, but was enhanced significantly to meet the requirements of the World Trade Center team. Able to handle larger resolutions than other renderers, DNB is much faster and allows the artists to achieve far greater levels of detail and realistic lighting effects on the smoke.
The R&D department developed ways to quickly visualise what smoke would look like, allowing the artists to rapidly layout scenes. This meant that the client could sit with the artist during scene layout and instruct them exactly where to place the plumes and smoke in real time, leading to a much faster sign-off of on the speed and direction of smoke. Previously, this kind of approval could have had a two or three day turn around.
In addition to these major sequences, Double Negative also produced a large number of ‘invisible effects’ shots. These included adding Ground Zero set extensions with added NY city buildings in the background and people, cars, signposts and traffic lights to the streets. The company also provided ‘set dressing’ for many of the street sets, adding in dust, rubble and debris to what had been shot as a clean environment.
These shots required the addition of a number of objects which were seen falling from the towers that day, this included fluttering papers and plummeting lumps of concrete and debris. Therefore the greenscreens required intensive rotoscoping as the falling paper and debris were added over and behind the actors. In order to handle the immeasurable numbers of such objects, Double Negative’s R&D team created a rigid body engine called Dynamite.
A rigid body engine generates realistic physical simulations of objects colliding with each other, and simulates real world forces on objects. This system enabled the artists to produce amazingly realistic and diverse motion on the paper which had to bend and flutter (matching on set practical falling paper) and also the rubble and debris raining down on the ground, which had to have more weight and pace as they fell from the towers. With Dynamite the artists were more efficiently able handle high numbers of objects in their shots.
A major shot is an extended pullout which starts in the midst of the Ground Zero rubble where three of the characters are trapped. The scene rises through a fully CG set, past the remains of the towers and up into the environment. This was actually about three shots stitched together. This incredibly complex shot was a production in its own right, thanks to the geometry involved in re-building the complex field of debris alone. Says Mike Ellis, “On the one hand it was just a big pile of junk and rubble, but we knew it had to be extremely precise, with specific girders and specific beams all with specific positioning. With a two-week turnaround on the rendering of the shot, it was also crucial to get the camera move nailed down early and signed off.
Due to the huge asset library and pipeline built for this feature, when a late request was received for another huge shot of a helicopter over the Ground Zero site on September 12th, the D.Neg team managed to produce the entire shot in just 6 weeks. This was one of the final shots of the movie. Double Negative pre-visualised the shot in advance to work out the framing, speed and altitude the shot should be filmed at. This proved essential for the camera crew and the pilot, who used this pre-viz on the day.
The shoot had to be cancelled once for bad weather, and indeed on the day the weather was still atrocious, says Bebb, “Thanks to the storms we only had a window of about 2 hours to get the shot. The buildings were still wet which wasn’t accurate for the shot, but the majority were too far away for it to be an issue. Once the plate was shot Double Negative’s in house photogrammetry software, dnPhotofit, was used to align the 3D geometry of the surrounding WTC site. The shot was then tracked in a similar fashion to allow for the best 3D track possible. Says Bebb, “This allowed us to provide accurate holdout mattes for the additional smoke elements. It also allowed us to ‘enhance’ the buildings with rubble and damage, as they would have looked on Sept 12th.
A 2 1/2D process was employed to keep the turnaround of the shot to a minimum. 2 1/2D involves painting over a photo/film frame and re-projecting that through the camera on to 3D geometry. This process provides relatively quick and successful results, allowing the artist to retain as much as possible of the ‘photoreal’ look. Consequently the helicopter shot was done in 6 weeks from receiving the plate. Due to the nature of the huge amounts of debris and wreckage at Ground Zero some of the shot also had to be 3D. So specific buildings and ‘iconic’ wreckage pieces (World Trade Center 1&2 remains etc.) were built in 3D, this worked best for the parallax issues associated with multiple vertical objects and a moving camera.
Giant Killer Robots and Animal Logic also worked on visual effects for World Trade Center.