Phil Lord and Christopher Miller, the creative producer team behind The Lego Movie and 21 Jump Street, have brought their unique sensibilities to a fresh and highly original version of Spider-Man, with the groundbreaking visual style of Spider-Man: Into the Spider-Verse, from Sony Pictures Animation.
The film’s genus lies in two key choices: firstly, the film makers did not hide away from the fact that there have been so many different tellings of the Spider-Man story previously, but rather embraced it. Secondly, they designed an original comic book visual style unlike any other film. Together these elements have been perfectly combined to produce a surprisingly original film that delivers the most inventive visuals seen this year.
Led by the directors Bob Persichetti, Peter Ramsey and Rodney Rothman, the artists and visual effects team at Sony Pictures Imageworks (SPI) experimented with a dazzling visual style that pays homage to the look of vintage comic-books. As director Peter Ramsey (Rise of the Guardians) explains, “Of course, dozens of Marvel movies lean on that look while telling a cinematic story, but I can’t think of any other animated film that make this much of a visual statement. That’s why audiences have had such a great reaction to the film.”
Danny Dimian at SPI was the VFX supervisor. He saw the film’s creative journey as a natural evolution of what the studio has been able to achieve over the past two decades. “This return to Spider-Man reminds me of our work on The Hollow Man (2000),” says Dimian. “Like on Hollow Man, most of the technical challenges on this film came out of creative challenges. This film was first and foremost rooted in coming up with something visually new. Through our early tests, we quickly fell in love with the idea that there was something charming about the original comic books. That was the problem we set up for ourselves and we were really happy how we ended up with a look that was really cool and stylized, and yet we hadn’t lost the feeling and the emotion in the characters.”
The animation crew is the largest Sony Pictures Imageworks has ever had, due to the unique process to get to the correct look of the film. There were nine leads, with a majority of the team in Vancouver and a smaller team of animators at the studio in Culver City.
Comic elements in the film include:
- Misregistration to imply defocus
- Graphic elements – used to fill the frame like “BOOM” and “POW”
- Panelization – breaks up action into panels and the animation frame rate mimics jumping from one panel to the next
- Half-tone dots – to render tone and texture
- Colors – broken down into defined shapes to give them a more illustrative feel
- Hand-drawings – certain effects such as smoke, sparks, and explosions are hand-drawn by the artists
Among the many stylistic ways the Imageworks team paid homage to old comic-books was emulating the imperfections of color offsets. “We noticed that sometimes in printing comic books, the color offsets were not aligned properly and this looked like the image was out of focus,” says Dimian. “We took that as an opportunity to explore how to play with focusing the camera based on the offset printing techniques. We looked at lots of different types of comic books in illustration printing, and we saw that it was hard to focus on an image when all the color passes were not properly aligned. We thought, ‘What if the camera didn’t de-focus like a lens?’ So, we splintered and offset the image in a way that is similar to a misprinted comic book page. It has a really cool feel to it that creates this illusion that something is printed on the screen.”
The filmmakers eliminated the idea of motion blurs for the film. The majority of the animation is done in twos instead of ones (images are held for two frames, rather than one – or twelve images per second, instead of twenty-four images), which is rare for CG. In fact, the directors wanted each frame of the film to look like a painting when paused to stay true to the comic book style. For scenes such as this train shot, lines are placed over the train and a rolling shutter effect is introduced to imply motion blur where there is none.
Dimian believes these graphical elements and overlaid lines in the shots were both an homage to the comic books and something that helped solve a problem. As the team had eliminated depth of field, they knew they would also need to eliminate motion blur, but they needed a new approach that was similar, but different to the misregistration used to solve depth of field.
The team decided that they could never get the look of the animation right when it was too splined or too smooth. This lead to the decision to use “step down animation to allow animation to hold poses for as long as they needed. Sometimes it’s two, sometimes it’s on threes – that’s really snappy,” Dimian comments.
There is no traditional motion blur where a camera shutter is open for a certain amount of time and it thus blurs the image in the film. “This is yet another example of a creative problem, that turned into a technical problem. That creative choice in turn rippled through the whole pipeline with a bunch of technical problems.”
Comic book panels
At various times in the story the frame breaks into multiple comic book frames.
From a cinematography point of view, “that was a real headache, because you want animation to be completely aware of these panels and you want to do all the planning when you’re in rough layout. ” Dimian explained. The frame story ideas come out of the storyboards, “we had mock ups of the look right away in rough layout but animation could then change that timing. They could change how the panels worked so that the panels worked with their performance. But, it is an extremely sophisticated thing, when you think about the fact that the panels need to relate to each other.” The action is each panel has to work separately and also as a whole, so multiple action animation sequences need to be choreographed to work together. Added to this an animator could request that their panel piece shape or aspect ratio be modified to better tell that panels story and it is little wonder these shots were some of the most complex to approve. Dimian commented, “It’s all a very different way to tell stories. Our overall intention was to use multi panels, text within the screen, words like thoughts and even words to represent explosions. To use an analogy, these are all kind of different and new ingredients and when we started to make this film was that was the first time we had ever cooked with all these ingredients together at once.”
The filmmakers also wanted a different look for the portal experience of the passageways to other universes. This was to represent what would happen when the smashing of the subatomic particles created a tear in the space-time continuum. The technical team at SPI developed a camera array that allowed them to project seven different angles on the screen at the same time, while enabling them to render each one of those cameras in a different style. Visually, the net effect looks not unlike abstract or modern art.
Halftone and cross hatch printing.
The film uses an incredible digital version of 4-color printing from older comic books and a mixture of cross hatch inking and half-tone dots to add texture and visual interest.
The effects team was given the green light to experiment with new technology that led to the invention of the film’s “glitching” effect. The glitch happens on characters, buildings, and cars when dimensional earthquakes occur as a result of the arrivals of characters from alternative Spider-Verses. They used multiple cameras that mirrored the multiple universes – they are all shot on the same character and same animation, but are treated differently, creating a cubist, fragmented look.
“Computers do everything correctly and so you always have the right perspective and geometry all the time,” Dimian points out. “What’s interesting and expressive about art is all the imperfections that go hand in hand with a human creating things. We had to find a way to break things and allow the hand of the artist to show through. Design and emotion took priority over accuracy and realism.”
While working on the film, Dimian constantly referred back to a quote from a scientist that says, “Success is going from failure to failure with undiminished enthusiasm.” The creative team was encouraged to try new things that haven’t been done before, even if it meant failing, in order to find ways to reach their end goal of creating a unique final look to the film.
Special Modelling consideration
It is not uncommon for a graphic novelist or comic book artist to exaggerate a perspective, but in 3D this takes on a whole new level of complexity – not just one of lensing but also distorting the actual geometry. This was done heavily in the film, especially in the environment work of the city.
In the film when Miles leaps off the building and falls back into the city, “you see that very clearly being used for dramatic effect. All the buildings in New York are actually oriented in kind of a ring around him. Of course that’s not real. That’s not the way most of our buildings were arranged in New York, but it just didn’t have the same drama to have Miles leap off into a grid of buildings,” explains Dimian. “So he falls off into a sort of circular pattern and even then it did not give the feeling that we wanted, so those buildings are not perpendicular to the ground. They are severely leaning and their heights can vary from like five times to eight times the height that the New York set actually was…but these are just things just visually looked cool.”
Integration of Different Animation Styles.
One of the film’s remarkable achievements is providing so many completely different characters on screen in such different styles and yet making them feel as if they are in the same physical space lit by the same light sources.
From a black and white only Spider-Man Noir to an anthropomorphic Looney Tunes style character called Peter Porker/Spider-Ham, the film encompasses an incredibly diverse range of animation and visual styles.
In addition to the usual powers such as web shooting, sticking to objects, speed, strength, incredible hearing and Spidey sense, Spider-Gwen uses graceful ballet-type movements in her fighting, which provided yet another stylistic addition to the mix.
Peni Parker is another one of the Spider-Verse’s unexpected delights. This anime Spidey heroine hails from a future version of Earth, in a robot suit. Alone, any one of these characters would present unique challenges but they often needed to work on screen together in the same shot, something that must have seen almost unattainable in pre-production.
While animators normally create approximately 4 seconds of animation per week, on Spider-Man: Into the Spider-Verse the pipeline was so complex and groundbreaking that they were only able to produce on average 1 second of animation per week. This led to hiring more animators to take on the workload and achieve the unique visual style. In the end, artists from over 30 different countries contributed to the work on the film, integrating different styles.
Faces and Rigs
For the most part the character rigs were not that different from previous rigs and blend-shape models the team were used to. The company has had a lot of experience with extreme poses and characters from their films such as Cloudy with a Chance of Meatballs and the Hotel Transylvania films, “so for the hero characters, they had a lot in common with those previous rigs. Peni was the big exception.”
To achieve the anime look, the animators animated the digital model but then her face was effectively removed and replaced as a ‘decal’. “There is no formal facial geometry for Peni. Her looks are from animation, but they are converted to a decal that is perfectly flat, and shaded flat. So we could make her as ‘Anime flat’ as we wanted. And that was that was important to the spirit of the world she comes from,” Dimian comments.
Classic Comic Book Tools
The film uses many devices and tools lifted directly from printed comic books – most noticeably, the direct use of text on screen as exposition and audience direction. These text boxes are overlaid or move as part of the camera movement.
As art director Dean Gordon, who also worked on the two Cloudy movies with Lord and Miller, explains, “The nature of technology of CG tends to fight a graphic look. We made hand-painted textures, with a level of abstraction to them, that we mapped onto geometry to get a more illustrative feel to our world. We did not want photorealism. We broke down gradations and color values into areas and created shorter transitions between them to get a more illustrative feel in the scenes. We brought the same ideas for the characters’ skin tones. Having the skin tones fit in the same environment and use the same screentones and hatchings we see in comics elevated that comic book look.”
The team at SPI set out to recreate the tactile, granular feeling of a graphic novels, even going as far as recreating the literal graphic flash frames in sequences right out of the older comic-books in terms of style. “You really feel the artistry as you turned the page,” recalls production designer Justin K. Thompson. “I know that one thing the computer does really well is realism. What we wanted to do was to invent our own reality, and then bend all the rules…I learned how to draw by emulating the art work I loved in comics, and Spider-Man was a character that I loved from an early age. That’s why I was really excited when Chris and Phil told me, ‘What if you were given carte blanche and could make an animated movie based on a comic-book?’”
Comic Book Sound FX.
In addition to moving to a more painterly style to indicate focus and depth, this frame adds actual old school text on screen to underscore the sound effects.
While the sound was done post-animation, the team was very aware of designing for sound and allowing the sound effects team screen time to score the action.