Fluid sims have become such a vital part of so many visual effects films, yet are not well understood by most general artists. We try and explain the science behind the fluid sims, and look at one in particular closely: Naiad, with help from our friends at Exotic Matter.
Introduction
One of the most significant and commonly requested areas of real world simulation is fluid simulation. From pouring shots to ocean vistas, directors and artists have come to rely on computer simulated water and similar fluids. Fluid dynamics is a complex area and fluid simulations are notoriously computationally expensive, yet when they work they can provide magnificent production value and breathtaking visual effects.
Fluid sims are not confined to just fluids either, they can be used to achieve fire and flames – the fluid being simulated in this scenario is the air itself (a gas). The main difference between gases and liquids is that liquids have a free-surface and they are volume constrained, as opposed to gases which don’t, unless you are dealing with a dual or multiphase simulation in which more than one fluid is being simulated in the same dynamics system. Oil and water, or air and water together are examples of dual-phase simulations.
Fluid simulations (fluid sims) have many applications outside visual effects. The science of computational fluid dynamics (CFD) or fluid sims has been used in engineering problems since the mid ’60s. The equations and maths of fluids are used to describe the physics of many things of academic and economic interest. They may be used to model the weather, ocean currents, water flow in a pipe and air flow around a wing of an aeroplane, for example.
History
Before the computer graphics industry got involved, fluids simulation was being actively modeled mathematically as early as the 1950’s and 60’s. One of the research institutes that had a major impact, back in the 60’s, was the T3 group at Los Alamos National Laboratory. James Harlow was the head of that group. Harlow and his team invented much of what we as an industry use today, including techniques we have explained below such as the the staggered marker-and-cell (MAC) grid structure, along with the Particle In Cell (PIC) method, which was the precursor to modern day FLIP, MPM and other hybrid methods (see below).
Unfortunately, most methods for real world CFD are needlessly complex for visual effects fluid sims and scale poorly. In the area of computer graphics, the key pioneers were Nick Foster and Dimitris Metaxas, for computing incompressible and free-surface flows. As we outline below, compressibility, while counter intuitive to many people, is a key approach to viable solutions. Before Foster and Metaxas, most water in visual effects used non physics-based methods, mostly in 2D or displacement and bump map tricks. Jos Stam of Alias | wavefront was also very significant for work with incompressible gaseous fluids. In his 1999 Siggraph paper he explained why simulation rather than keyframe animation was so important.
Physical models, unlike key frame or procedural based techniques, (fluid sims) permit an animator to almost effortlessly create interesting, swirling fluid-like behaviors. Also, the interaction of flows with objects and virtual forces is handled elegantly. Jos Stam. (1999). Stable Fluids, SIGGRAPH 1999 Conference Proceedings: SIGGRAPH Annual Conference Series. pp. 121-128.
All of this happened in the early 90’s at key places such as Pacific Data Images (PDI) and Alias (3D). As demonstrated by early efforts in the films such as Waterworld and Titanic, images of CG water were possible with a high degree of realism, but the realism was mostly limited to relatively calm, wide open ocean shots. Jerry Tessendorf, Principal Graphics Scientist at Rhythm and Hues (R&H), did great work in the development of water and he shared with three other members of R&H in 2008 a Technical Achievement Award from the Academy for their custom fluid dynamics tools in CFD.
– Watch Jerry Tessendorf talk at TED.
Stanely Osher did seminal work on level set methods for representing dynamic surfaces, and his PhD student Ron Fedkiw later took level sets to graphics and invented the Particle Level Set (PLS) method which reduced the mass loss problem of pure level set methods by seeding particles along the fluid surface to capture sub-grid mass that would otherwise be lost. Fedkiw’s work would become enormously significant. He was awarded an Academy Award for Technical Achievement from The Academy of Motion Picture Arts and Sciences. Currently he is Associate Professor, Stanford Computer Science, where he has published over 80 research papers in computational physics, fluids and vision. For the past ten years, he has been a consultant with Industrial Light & Magic (ILM), and thus has received screen credits for his work on Terminator 3: Rise of the Machines, Star Wars: Episode III – Revenge of the Sith, Poseidon and Evan Almighty.
Robert Bridson, who in turn was a PhD student of Ron Fedkiw, and worked with ILM as one of the original contributors to the PhysBAM project, under the supervision of Fedkiw. PhysBAM remains the backbone of ILM’s physics simulation code base.
Bridson went back to the research of the early 1960’s and to the research of principal researchers such as Harlow and made PIC/FLIP work for incompressible flows while simultaneously introducing his incompressible FLIP method to graphics. The FLIP method and its variants achieve a near total lack of numerical diffusion in the transport stage of the fluid simulation, since all quantities are advected on particles as opposed to a grid.
Along with PhysBam, Bridson helped with cloth simulation code used for Star Wars Episode II: Attack of the Clones at ILM. Bridson then moved to the UK and co-wrote the Squirt fluid simulator for Double Negative, seen in many of their films for smoke, water, fire, clouds, ink-in-water, including Harry Potter and the Half-Blood Prince, 2012, The Boat that Rocked, Inkheart, Quantum of Solace, The Dark Knight, and Hell Boy II: The Golden Army.
It was in the UK that Bridson would meet another industry legend – Marcus Nordenstam and they would form Exotic Matter in 2008. Exotic Matter are the makers of Naiad fluid sim software, which has been featured in some of the most impressive liquid effects to date, including films such as Avatar, Narnia: Voyage of the Dawntreader, X-Men First Class, Harry Potter and The Deathly Hallows Part 2, Pirates of the Caribbean: On Stranger Tides, Rise of the Planet of the Apes, and many others which we have featured here at fxguide.com.
– Naiad simulation shown at SIGGRAPH 2010’s Naiad User Group & Get Together (NUGGET).
Nordenstam now has more than fifteen years of experience in VFX R&D and effects simulation. He co-invented curl-noise, pioneered the use of the FLIP method for fire simulation, and co-authored the “Squirt” fluid simulator with Bridson. Prior to founding Exotic Matter, Marcus held senior engineering positions at ILM, where he was one of the original Zeno programmers. His screen credits include Star Wars Episode II, Spider-Man 2, Hellboy II, Inkheart, Harry Potter and the Half-Blood Prince, and Avatar.
As part of Exotic Matter’s approach, Nordenstam and others from their company have embedded themselves in key facilities, especially during the early days of the company. Most significantly, this occurred at Weta Digital in Wellington, New Zealand for ten months as part of R&D on films such as Avatar.
With the huge success of Naiad it is not unreasonable to point to it now as representing the state of the art in fluid simulation. While no one simulator does everything, Naiad is proving very popular for its scalability, believability and realism.
– A Naiad scene test: ‘Bunny in Trouble’
– Naiad simulation by Igor Zanic shown at SIGGRAPH 2011’s Naiad User Group & Get Together (NUGGET).
Today there are several key companies in the area of fluid simulation. Munich-based Scanline Productions (headed by Stephan Trojansky) and their award-winning Flowline software for example allowed The Moving Picture Company (MPC) to convincingly flood the Poseidon’s interior, and help Scanline wash away large parts of 2012 and re-create terror for Clint Eastwood’s Hereafter. There are also other great fluid off-the-shelf tools such as those in Maya, Houdini and those from other specialist companies and teams such as Blender 3D, RealFlow, FumeFX, Dynamite, ICE (as part of Softimage/XSI), PhyFluids3D, and more, but for this article we will be using an examination of the science of Naiad to provide an insight into the science behind the state of the art.
Basic concepts
Some fluid sims are based or focused on surface properties. Early on, for example, ocean water sims produced semi-flat oceans. Newer simulations are concerned with volumes, both in terms of say a pouring shot, or a flood of water rushing around rigid bodies but also how say where the floor under the water is located in height affects the surface – ie. the way land fall causes breaking waves on a beach.
Some of the fundamental issues that are at the basis of these second main class of fluid sims are:
• conservation of mass. Literally the water doesn’t disappear during the sims (or the maths that it is built on)
• conservation of momentum or energy
• conservation of volume – incompressible – unlike say real water this may seem odd (doesn’t sound travel under water?) but for visual effects this assumption is close enough to reality and makes the maths vastly simpler
• connective acceleration – space controls acceleration, for example as water spreads out or funnels in
• there are two key forces that act on fluid – fluids are assumed to be affected by gravity and itself (pressure) In a paragraph or two we can even show that mathematically (see below)
• we ignore viscosity for most but not all fluid sims, viscosity plays a small role and we ignore it
• boundary conditions – this last point is key – the edge of the fluid is very important, be that a surface or a boundary. Bridson in his book, Fluid simulation for CG (2008), stated that “most of the, ahem, “fun” in numerically simulating fluids is in getting the boundary conditions correct.” There are three things – solid walls, free or freely moving surfaces, and the hardest, other liquids (this last one is rarely seen in films).
To understand volume simulations one needs to think of forces acting on ‘fluids” as being simply: high pressure to low pressure in a 3D space. In the water or fluid sim one has a gradient from low to high pressure and this “vector field” will define the movement based on some reasonable time interval.
Simulation – Solving the Navier-Stokes Equation
Navier-Stokes are a set of equations that describe the behavior of various fluids (including gases). Navier Stokes equations have been around for a couple of hundred years, but as we eluded to above, it is a subset of the general problem with some key assumptions that the effects industry is most concerned with.
The result of a Navier Stokes equation is not a number like 42, it outputs a velocity field – or a complex vector.
The Navier-Stokes equations dictate not position but rather velocity. This is an important point that confuses many people – the vectors are not points in space so much as velocities. What we have then is an advection equation, which is how molecules or in computer graphics terms – particles – move within the velocity fields.
The Navier–Stokes equations, named after Claude-Louis Navier and George Gabriel Stokes, describe the motion of fluid substances. In semi-technical terms these equations arise from applying Newton’s second law to fluid motion, (F=ma) but together with the assumption that the fluid “stress” ie. which way it wants to go, is the sum of a diffusing viscous term (proportional to the gradient of velocity), plus a pressure term.
The equations are useful because they describe the physics of many things of academic and economic interest. They may be used to model the weather, ocean currents, water flow in a pipe and air flow around a wing. The Navier–Stokes equations in their full and simplified forms help with the design of aircraft and cars, the study of blood flow, the design of power stations, the analysis of pollution, and many other things.
A solution of the Navier–Stokes equations is called a velocity field or flow field, which is a description of the way the fluid wants to move at a given point in space and time. Once the velocity field is solved, other things we care about for a believable shot, such as flow rate or drag force can be found.
This is different from what one normally sees in classical mechanics or rigid body simulations, where solutions are typically trajectories of position of a particle or deflection of a movement. Studying velocity instead of position makes more sense for a fluid; however for visualization purposes one can compute various trajectories.
Here is the super simplified maths – based on the assumptions above notably that the “incompressible Navier-Stokes equations” are sufficient.
Explaining the equation (in lay terms – roughly speaking):
what this is actually meaning in lay terms is …
acceleration + something + change in pressure/density = body forces + dynamic viscosity
but with the assumption that liquid is incompressible. Now also assume viscosity is ignored (!)
acceleration + 0 + change in pressure/density = body forces + 0
which becomes if you rearrange the terms …
acceleration = body forces – change in pressure/density
but density is defined by Wikipedia or a textbook as density= mass/volume, and a change in pressure is related to volume …
acceleration of bits of water = gravity(forces) – change in pressure /density
If the density of the liquid is a constant that does not change really….
Thus the particle acceleration is related to the forces on it, like gravity AND the pressure on the liquid
Practical implementations
If fluid sims stopped with some complex maths, it would be fairly easy to just program, but there have also been a lot of variations due to the complexity to compute those equations. The whole implementation of fluid sims in the effects industry is a balancing act between computational strain, realism, computer power and compromise. A full solution is out of the question so as with much of computer graphics we end up with clever workarounds and insightful cheats. One such approach, for example, is the Surface Tracking Euler method. This approach uses height mapping solutions and is just for surfaces, it ignores what is happening below the surface of the fluid we are viewing.
Some algorithms don’t scale well between say glasses of wine and ocean vistas. Still others have maths flaws that mean the volume of water in the container appears to decrease over time, a sort of rounding error that would see the glass box holding the mermaid in Pirates 4 decrease in water volume over time. (This apparently happened during testing on Pirates 4 and Naiad was used on these shots to solve the problem).
Here are some key methods of producing outstanding results in a sensible time frame that have developed over the years in effects fluid sims research.
Smooth Particle Hydrodynamics: SPH approaches
SPH is a particle system, driven by the Navier-Stokes equations move around the simulation. These are then converted to polygons and then rendered.
In simple terms:
Fluid sim to produce particles, then polygonize the particles to produce the waves.
The idea of particles in a container being converted this way dates back to initial work by Jim Blinn (JPL, Microsoft), the man who invented much of what we think of as fundamental CG today (such as bump mapping).
Particle-based so called Smooth Particle Hydrodynamics (SPH) models can do great things such as pouring shots, but they have a hard time with large-scale effects such as a giant expanse of water.
The first of these systems or the SPH-mesh free solution is called a Lagrange system. One of the best SPHs is Multi-physics, (ships as part of XSI/ICE), as it exhibits elasto-plastic behavior. In the case of Multi-physics we featured an interview with its author and shots he had done feeding his particle solution to the Em-polygonizer plugin.
Volume GRID approaches
Surface liquids are sometimes called Volume Fluids, such as in Houdini, where there is an assigned distance field or height field designed on some surface – perhaps the ocean – inside some regular voxel grid. The waves effectively exist inside the voxels defined to have a certain height and frequency.
The surface has a problem of course if it is too thin – effectively it disappears through the floor of the voxel space. Surface liquids like this have no particles. The second approach uses a grid rather than particles. Grid-based methods were great for large ocean effects, but not fine detail close up visual effects work. Things can look globby and unrealistic at small scale (thin fluids).
This approach falls into what is formally known as the Euler approach. Maya fluids is an example of this grid or lattice solution. It maps high and low pressure to get back a velocity field but with no particles. Maya fluids is based on a system called (perhaps misleadingly) a Semi-Lagrangian method.
Semi-Lagrangian method is a pure Eulerian method and does not use any particles at all. It is only called “semi-lagrangian” because the way the method works involves tracing through a velocity field like a particle does, but there are never any particles actually instanced. The most famous example of a solver using the Semi-Lagrangian method is Jos Stam’s Maya fluids simulation system. Stam in fact invented the method.
A great example of this approach is Exocortex’s Maelstorm FX. Fxguide has spoken to Exocortex in the past – showing their adaptive tetrahedral shape / mesh approach. The Marching Cubes os a tetrahedral shape which is based on an older algorithmic approach of marching squares (for those who recall much earlier popular 2D Siggraph algorithms).
Marker and Cel method: MAC
The MAC grid – MAC stands for Marker And Cel method – is used to represent a velocity field – three vectors per gird cel. MAC is not a simulation method like SPH, rather it’s just a convention used in how to represent data on a grid for an Eulerian method. The ‘Staggered MAC’ arrangement does not suggest three vectors per grid cell, merely one vector, where each component is stored on the cell faces. In a non-staggered MAC grid, you still only have a single vector per cell, but all three components are stored at the cell center.
Pixar adapted this method for example on Ratatouille for the early water escape sequence to use cels of different heights. Rather than divide the water into a set of equally shaped cubes, the Pixar team had cubes at the surface and near the bottom of the geometry but then long tall cells in the middle, where not much was happening. This was a classic example of reducing computation while keeping the important surface accuracy and fidelity and also all the fine detail interactions at the bottom of the water sim (a rough pipe or water sim floor would mean more turbulence at the surface than a very smooth pipe or base for the simulation).
Particles in Cels: PIC
Particles In Cels (PIC) works by evaluating particles as groups of particles in cels (like voxels). The particles affect the grid and then transfer their vector ‘information’ , the equation is then solved, the program moves the particles and the process is repeated. Moving cels is particles and not evaluating each particle individually is vastly better computationally, and the size of the sim data can be reduced. Sim data can account for vast amounts of data in a major facility. As one might imagine the PIC method can lead to a smooth simulation as particles in the cel are affecting each others next result, not unlike averaging.
FLIP
Another key approach is a FLIP solver. A FLIP solution is a type of hybrid between a particle based and volume based fluid simulations. It is Naiad’s FLIP and DEFLIP maths that solved the loss of fluid in the Pirates 4 mermaid box example.
A great advantage of the FLIP solver over say a Particle / SPH fluid sim is how many times you need to evaluate the maths per frame. For a SPH solution to look good the program needs to evaluate it multiple times per frame – a sort of temporal anti-aliasing. The maths needs to be run 10 to 100 times more per frame to work. (Although for some very fast moving colliders, people do start to find it useful to ‘subframe’ evaluation FLIP models also, or you can miss an interaction inside the fluids). If you do not sub-sample per frame in the case of a SPH solution, particles appear to just fly off or explode away from the fluid.
Houdini also has three types of fluids including FLIP fluids via the FLIP Solver Dynamics Node(s), the other two being a volume fluid and SPH fluids. In relation to Houdini 11, Jeff Lait, Senior Mathematician at Side Effects, has said:
“When FLIP fluids are solved, a temporary velocity field is made. The particle velocities are transferred to this grid and the grid is used to perform the fluid projection. This is what prevents the particles from all going on top of each other and start moving in similar directions. FLIP fluids are also useful because particles can be placed on top of each other without destabilizing the system. SPH tends to blow up if you move particles too close. Various fields are used to tame the FLIP Solver so that you can run far fewer points at far fewer time steps and the interspacing between particles can be random. You can introduce new particles at any time with little to no consequence. For example, introducing splash particles with their own property attributes are possible.”
Naiad Case Study
Exotic Matter’s Naiad uses hybrid methods like PIC-FLIP, deFLIP and even newer, as yet unpublished methods. Exotic Matter CEO Marcus Nordenstam explains:
“There are further improvements such as the FLIP and deFLIP method PIC-FLIP family of solutions use particles to represent the fluid. Then at each step in the simulation, a grid is constructed from the particles and the velocity and surface information is transferred from particles to this grid. The pressure solve then happens on the grid, and the particles are updated using the result of the pressure solve (e.g the velocities are updated on the particles). The particles are then transported through this grid, using the new velocities, and the step is complete.”
fxg: Can Naiad allow new particles to be added, for example, introducing splash particles with their own property attributes?
Marcus Nordenstam: Yes, there are various particle emission operators available to the user in Naiad, and they can be run as part of the primary fluid simulation, or as secondary simulations purely to increase particle count and spatial detail, such as splash, spray, foam etc. It doesn’t matter if the added particles are primary fluid particles or secondary particles, to Naiad they are just particles, and therefore any attributes can be attached to them (Naiad calls them “channels”).
fxg: How accurate is Naiad now? Do you feel you have a long way to go?
Marcus Nordenstam: Naiad’s accuracy is around first-order, which is pretty much the state-of-the-art for incompressible, possibly viscous, unsteady free-surface flows with complex, moving boundaries. CFD engineering packages costing 10x more than Naiad will be first-order accurate as well for these types of problems – if they even attempt to solve them. Naiad can also compute simulations with rigid bodies fully-coupled to fluids. Many other simulators have to solve the fluid and rigid-body separately, which degrades the resulting accuracy and visual detail.
Another important point is detail vs resolution. Naiad is well-known to provide a lot of detail, even at very coarse resolutions. Naiad does not define resolution in the traditional sense, it focuses instead on the size of the smallest resolvable feature, which we call “cell size”. The cell-size is essentially the size of each fluid “voxel”. The cells are mapped out in space in an adaptive, dynamic sense, and they are not constrained by a “fluid box”, or domain, which the artist has to manage. This is possible because of Naiad’s unique 3D tiling system wherein a Naiad simulation adaptively refines regions of space where the fluid is, storing and solving those regions only.
fxg: Is it possible to have one solution that works across a range of scales of fluids?
Marcus Nordenstam: The Navier-Stokes equations hold true all the way down to a molecular level. They also hold all the way up to very large scales, so the underlying physics we are modeling does scale across the full range. The only thing that really changes as the scale changes are which terms of the equations become dominant. For example, at very small scales, surface-tension becomes more influential than any other force, whereas at large scales the surface-tension is negligible and does not need to be modeled at all. From what we can tell, Naiad has performed quite well at both small and large scale simulations, and there’s plenty of examples of both large and small-scale simulations available both on vimeo’s “Naiad channel” as well as in several of the movies playing in the theaters right now.
fxg: What is involved in producing animation that does account for viscosity?
Marcus Nordenstam: From a solver’s point of view, what Naiad does when computing a viscous fluid is make the fluid resist deformation throughout. Unlike most other viscous solvers out there, Naiad does this for each component of velocity separately – which is more physically accurate. This leads to Naiad’s viscous fluids being able to coil, buckle and bend naturally when encountering a solid, without having to induce any rotational motion into the fluid during emission.
From a user’s point of view, achieving viscous fluids in Naiad is as simple as dialing up the viscosity property on the fluid body they wish to simulate. Variable viscosity can be achieved by mapping a scalar 3D field (like a 3D texture) across the fluid body, where the field values correspond to viscosity values. This also works for variable density fluids. In fact, you can model variable-density, variable-viscosity, incompressible flows in Naiad using this method.
Further recent examples of Naiad’s viscosity can be found in these 3D animations posted on vimeo:
fxg: Naiad works up to what point in the pipeline – what is the output from Naiad? For example, would I take the output into Houdini?
Marcus Nordenstam: Naiad has no default output – instead there is a graph that represents your simulation. As a Naiad user, you build this graph yourself (just like a compositor builds a compositing graph using Nuke, for example). The Naiad graph includes file operators to cache out data to disk. What you cache out is up to you, but, for example, many of our users will perform meshing inside Naiad, and output the final mesh. But they could output the particles instead by changing their graph slightly – or output both. They could also output the 3D field data, such as velocity, the fluid level set, or the distance to nearby collision bodies. They can place file operators all over the place, to output the particles of a fluid body at different stages in the solve itself, before or after certain operations, etc.
We have even added render operators to the Naiad graph. They currently only work with Solid Angle’s Arnold renderer since many of our customers are also using Arnold – but we are working with the Chaos Group right now to also add built-in V-Ray render support, right from our graph, and other renders will likely follow. So using these new render operators, we have example graphs that do not output even a mesh – but a final fluid render!
The most common way of interfacing Naiad with existing pipelines is using one of our plug-ins available for exchanging data between Naiad and other 3D packages such as Maya and Houdini. For example, the artist models the animated objects and emitters in Maya, exports the geometry into Naiad, runs the fluid sim in Naiad, and then brings back into Maya whatever data the artist cached out in Naiad.
fxg: How fast is the research moving today – is this an exploding area of research or has it slowed and is it thus maturing?
Marcus Nordenstam: Most of the research right now seems to be on making the fluid solvers more efficient on modern hardware architectures. There’s still a lot of work to do in fluids and multiphysics (e.g. viewing physics/dynamics as a single simulation rather than separating them into fluids vs rigids vs soft bodies etc).
fxg: What can users do to improve sim times on Naiad – what drives the sim times?
Marcus Nordenstam: The resolution of the underlying computational grid drives the sim times more than anything else. Naiad is stable regardless of time-step counts, although you generally get smoother motion if you take 2-3 time-steps per frame instead of 1, but that will also slow you down. Basically, most adept Naiad artists try to use more particles (since that doesn’t slow you down very much), and larger cell-sizes to get good spatial detail.
Future issues
One huge area of growth for fuild simulations is the degree to which fluid sims can be made parallel and use GPUs. Most major facilities have explored GPU render farms for accelerating fluid sim calculations. “This comes down mostly to maths rather than physics,” says Marcus Nordenstan. “The linear systems of equations arising from discretizing the navier-stokes equations need to be solved as efficiently as possible. There are many kinds of linear solvers, some well suited to parallelism, while others are not. Most researchers seem to be favoring multigrid (MG) methods since they are trivial to parallelize and therefore well-suited to running on GPUs and multicore computers.”
Fluid sims were of course featured in a number of presentations at the recent SIGGRAPH conference in Vancouver. fxguide’s Mike Seymour moderated an NVIDIA panel covering the GPU experiences of some leading CTO’s and R&D experts in the visual effects industry. The panel, which included representations from ILM, Weta Digital, Blue Sky, Double Negative and Imageworks, discussed fluid simulations amongst many other things. You can listen to the audio from the panel discussion here.