A new music video utilizes 200 drones, 3D animation in Cinema 4D and the AR toolkit Moves by Maxon to create a truly unique musical experience.
“High in Heaven,” is sung by an anonymous artist who goes by the name VWLS and is featured in the video as an animated drone face singing across a beautiful midnight sky. While the concept for the music video came before COVID hit, it ended up being the perfect solution for a music video shot during quarantine, as there are no actual actors in this video.
Creative Director of Hobbes, Nick Forshee, is the creative mastermind behind the music video.
Q&A
FXG: Congrats on the project. How hard was it to resolve the actual face to be animated? How many faces did you look at as viable options?
NF: We narrowed down from about 6-8 initial sketches. The first few looked like Josh, but after a week of exploration, we found a more intentional design would translate better to drones.
FXG: The main face has strong nasolabial folds which, along with the single strong drone light for each eye, gives the face a droid like / AI appearance – was that your intent or am I bringing my own world view too strongly to bear?
NF: Funny you mention that! The first time we attempted a face ( in 2019 ) was intentionally made to look like AI because it was part of the finale for an AI conference drone show in California. It didn’t sing, but it taught us that strong line choices really sold the dimension of a face. We really wanted the face to feel like a mix between AI and a cosmic entity. The cheek lines and single drone eyes were definitely droid / AI-inspired, but the brow line gave it personality/wisdom.
FXG: I am fascinated with the issue of lip sync. Human lips can move very quickly but you would have had physical limits on how far (or fast) a drone could move – which in turn would limit the range of visages I assume?
NF: We decided to animate the 3-minute song in real-time then slow the finished flight paths down 5x, equating to a 15min show. After filming, we sped up the footage 5x to restore the lip-sync perfectly. This gave us the flexibility to have lip sync and keep the drones throttled down to 15mph to meet Firefly’s requirements.
FXG: What were your considerations on the overall facial density? I assume there are safe distances between the drones but you could have made the face denser by bringing them in closer… and yet as the later ‘exploded view’ face shows – the effect works very well even as a sparse cloud?
NF: For safety, Firefly requires that the drones remain 15ft apart. This is very tricky for density, so we squeezed as many together as we could. But even with maximized spacing, the resulting face was still around 300 ft tall.
FXG: Given the face is ‘3D’, how directional were the lights? How far off the front position could a viewer be and still ‘read’ the face?
NF: The lights are on the bottom on the drones, which would make them nearly invisible from above (aside from small LED’s on the top for planes). From the front, the face works from nearly any distance. The illusion breaks if you were to view the face from the side where you would see overlaps and unoccluded lines. Guides were given to the cinematographers, especially the video drone operator to avoid those angles. Furthermore, we knew we’d be on the ground shooting up from the hill, so we tilted the entire showdown 8 degrees to give us better filming options.
FXG: One of the immense aspects of the clip is that it is real.. so the lighting up of the hilltop with the Comms Tower all seems really important… how much did this ‘locating and seeing the face over the physical hill’ play into your planning?
NF: This was 1000% intentional. The size of this would have felt like nothing without context. Ski lifts are a great indicator of size as well as the depth of an outlying urban area. Against the sky, the drones just look like dots, but when they illuminate an entire hill… you really feel how big it is.
FXG: With the camera drone, did you explore flying into the face at all? You get fairly close at the side, but it is hard to tell the lens on the camera drone?
NF: Yep! but the camera drone needed to keep a safe distance from the swarm. And since the owner of the mountain was doing us a favor, we were on our best behavior.
FXG: This seems to have been shot on top of a ski run in summer?
NF: Bingo, Pine Knob ski resort in Michigan.
FXG: Did you explore the blinking of the eyes?
NF: We did explore a 2 state blink where one set of drones would turn off to reveal another closed state that had been hidden ( similar to a crosswalk sign ) But that ate up too many drones so we decided against blinks.
Technical issues
FXG: How much control did you have over the lights?
NF: We have complete control over color and intensity. But with great power comes tough design choices to keep it simple and leave an impression of “a giant blue face.”
FXG: Which drones did you use? How big were they?
NF: The drones are designed, built, and operated by Firefly Drone Shows specifically for this type of application. 200 total drones were used + a camera drone. They are each about a foot in diameter.
FXG: How many takes are we seeing in the clip… was this an edit of multiple takes?
NF: We filmed 3 takes in 2 locations. One at Pine knob ski resort and the other at Firefly’s test field in Holly, Michigan. The entire performance can be seen as 1 take but decided an edit would help captivate viewers a bit more.
FXG: Could you outline the major technical steps – with C4D and the AR tool kit, how did you match the physical limits of the drones and the speed they could move? Plus what software programmed the drones?
NF: We used features from Maxon like a swiss army knife. One major milestone being the real-time face capture that Moves By Maxon which enabled us to record 4 unique performances. With pose morphs, we could dial up the “energetic” performance when the song picks up and dial in the “stoic” when we needed the face to be more monolithic/observant. We were able to use a surface deformer to glue single points ( drones ) onto the face mesh. From there we monitored for speed and collisions. Once everything checked out we sent our finished show to firefly who then converted the data for 200 individual flight paths. We were able to accomplish a lot without really going to deep into C4D, most of the features we used have been proprietary for over a decade.
FXG: What was the comms solution? If a drone lost comms was there a plan – A safe return to base? I am sure there would be safety concerns about such things?
NF: Firefly uses purpose-built dual redundant data links to ensure that never happens. However, Firefly has many proprietary failsafe features and procedures to remediate such a problem if it did arise.
FXG: How long would the drone and light battery allow a sequence to run before needing to return to base?
NF: In warm temperatures, 16 minutes.
FXG: What was the clip shot on? How did you balance the brightness of the light with the ISO of the filming and thus the illumination of the ground cover and trees?
NF: Hands down the Sony A7Sii, used by the Firefly crew, captured the light the best. They use this camera for this reason to capture most of the shows.
FXG: Was there much post-processing on the footage? Grading up or power windows to aid readability?
NF: The only grading was to keep the blues consistent. We suspected people would assume the drones were CGI, so we intentionally kept it very raw. Even their amazing glow toward the end is just from the lens getting foggy the night the temperature dropped.
FXG: I assume all the VWLS animation from the face pose was also an issue since the letters need to sit co-planar and the drones would risk collision if they just moved from one major 3D cloud position to the letters? Was that collision detection just a function of the C4D or some custom process?
NF: Unfortunately, this one is a trade secret as well… but took nearly a year to develop something that could handle this complex problem. A drone moving out of the way of another drone can also intersect the flight path of another drone, this problem compounds itself over and over.
FXGUIDE: thanks so much