Cloth Simulation, opening the Kimono

Physics-based animation is commonplace in animated feature films and even special effects for live-action movies. Cloth simulation is so ubiquitous with most virtual characters (hopefully!) wearing some sort of clothing. Highly believable simulated cloth is typically generated by an algorithm that evolves over time discrete mathematical models. For that model to look believable it needs to describe how the material would move, its material properties, boundaries, external forces, and collision with objects in the scene. But in animation and VFX we are not so concerned with simulation accuracy as believable and controllable simulations.

Cloth simulations are a key part of most animation packages and dedicated software such as Marvelous Designer (MD). Marvelous Designer Version 8 arguably provides one of the best standalone applications for CG artists to efficiently create beautiful 3D clothing. With a pattern-based approach to modelling, users are able to easily create and simulate quality garments that are compatible with most programs. The program, like others such as CLO3D and Optitex, is used in both the entertainment industry and outside in actual cloths design.

The MD Korean based cloths simulator has been used on films fxguide has reported on, such as Ted and Ted 2 by Iloura and at companies such as Weta Digital (The Hobbit).

Ted’s cloth building and simulation was done in MD. The CG outfits were stitched together with ‘real’ stitching, which provides the digital cloth tension where it belongs to produce an accurate cloth simulation. If there’s an area that’s loose or under tension in Ted’s suit, then the digital suit has the same properties that a real world piece of clothing would have and it accurately buckles and creases like real clothing.

MD is also used extensively in Games with clients such as Blur Studios’ Far Cry 5, Crytek’s Hunt:Showdown and Ubisoft with Assassin’s Creed. Below is the new 2018/2019 MD show reel.

Cloth Sims explained: New Book.

Pixar’s Coco was the story of a kid named Miguel who discovers his family history on Dia de los Muertos, Mexico’s celebration honoring ancestors on their after-life journey. Miguel’s costars are skeletons, which presented some of the most unusual technical issues to Pixar’s cloth simulation team. “We knew going in we just didn’t have the technical ability to even put clothing on skeletons because of our simulator,” explained cloth and tailoring lead Emron Grover, in a Wired article. The problem is that the Pixar simulator, does exactly what it should – which is drape cloth over the skeletal bones. The problem is that the results were naturally not very cinematic. It was designed to be accurate and not to lay cloth over characters made entirely out of thin objects such as bones. Grover says, “We spent three years revamping our collisions system so that we could be able to have robust collisions,” between the skeletons and what they were wearing, and he added that the new system kept the characters’ bony nature prominent even under fabric. “We tightened all the clothing,” he explains, “We cinched it around the spines and tightened it around the rib cages so you could really feel the ribs underneath.”

David Eberle at Pixar was the lead who solved the revamped collision detection system and one of the other  team who helped solve the cloth in Coco was Tuur Stuyck. He has just published a book with the catchy title “Cloth Simulation for Computer Graphics”. Fxguide caught up with Stuyck at Siggraph Asia in Japan to discuss his new book. What makes this book different is that it is a bridge between the complex work of hard core SIGGRAPH papers and a more informal (non-technical) understanding of cloth simulations.

Stuyck started working with computer graphics as early as 2006. He created several computer-animated short stories that were selected by international film festivals.  He got a Bachelor’s degree in engineering, majoring in computer science, at KU Leuven and graduated cum laude in 2013. And then in 2013 he graduated magna cum laude with his Master’s degree. He obtained his Ph.D. in the Computer Graphics Research Group in 2017. During his Ph.D. research, he was awarded second place in the 2016 ACM SIGGRAPH research competition for his work on art directable simulations.

Stuyck has collaborated with Adobe Research working on natural media simulation. He has interned at and then joined Pixar Animation Studios as a Postdoctoral Research Scientist, researching art-directed cloth simulations for feature film production. He currently works as a Postdoctoral Research Scientist at Facebook Reality Labs (previously known as Oculus Research)

The focus of this book is physics-based cloth simulation. If you want to understand cloth simulations beyond a graphical user interface then this book explains the maths and acts a primer to allow you to connect with the amazing research being done and presented at conferences such as SIGGRAPH, SIGGRAPH ASIA and CVPR. It is not a picture book with menus of popular applications but it is one of the best books for quickly going from basic maths to understanding complex approaches to cloth sims.

It starts by providing background information and discusses a range of applications. The book provides explanations of multiple cloth simulation techniques. More specifically, it starts with the most simple explicitly integrated mass-spring model and gradually works the viewer up to more complex and commonly used implicitly integrated continuum techniques in state-of-the-art implementations.

The book is only 95 pages, but as it was only written in mid 2018, it is up to date in a very fast moving field.

Cloth simulations of course rely on the environment, the forces of gravity, wind and collision, but as Stuyck points out, it is the internal forces combined with collision and friction that make cloth so interesting. The cloth has to respond to internal forces such as stretch, shear, and bend forces that act on the particles that are used to represent the virtual garment.

To compute motion over time, time integration is applied to compute the dynamic forces. In other words: calculus. And this has to be done in a world where approximations are often needed, so maths such as Forward Euler Integration is required. This is just a way to approximate or do explicit integration.  If you cant remember any maths from school, then this is not the book for you, but if you remember the basics of calculus then the book goes on to build from this and starts with the classic cloth solution of a Mass and Spring model, which approximates the key internal forces of cloth.

Stuyck gives an intuitive explanation of cloth techniques and additional information on how to efficiently implement them on a computer. This book discusses explicit and implicit integration schemes for cloth simulation modelled with mass-spring systems. A mass spring system consists of a set of point masses that are connected by springs. The physics of such a system is straightforward and a simulator is easy to implement.

In addition to this simple model, he explains the more advanced continuum-inspired cloth model. This method is commonly used in industry. The later chapters that deal with the Continuum approach to cloth outline how it is more complex but produces a better result, or rather a more realistic result, unless you are brilliant at working out exactly where and how to apply spring connections to match a particular cloth material.

But just as Stuyck’s background would indicate, in addition to these simulation approaches he also discusses how cloth simulations can be art directed for stylized animations. Controllability is an essential component of a feature animation film production pipeline.

A key last section of the book is briefly devoted to collision detection and response. Both self collision which is cloth on cloth and also cloth colliding with other objects, often the model they are worn by.

Real time

Among the force based approaches, one of the simplest methods to represent and simulate cloth as mentioned above is with mass-spring networks.  However, there are some significant drawbacks of the simple method:

• The behavior of the object depends on the way the spring network is set up.
• It can be difficult to tune the spring constants to get the desired behavior.
• Mass spring networks cannot capture volumetric effects directly such as volume conservation or prevention of volume inversions.

A Finite Element Method (FEM) solves all of the above problems because it considers the entire volume of a solid instead of replacing it with a finite number of point masses. Here, the object is discretized by splitting the volume into a number of elements with finite size. This discretization yields a mesh as in the mass spring approach in which the vertices play the role of the mass points and the elements, typically tetrahedra, can be viewed as generalized springs acting on multiple points at the same time.

In both cases, forces at the mass points or mesh vertices are computed due to their velocities and the actual deformation of the mesh.

In real time applications, instead of these approaches there has been a focus on position-based simulation methods which omit the velocity and acceleration layer and directly modify the positions of the particles that define the cloth’s shape.

This approach has been implemented in many state of the art physics engines such as NVIDIA PhysX, Havok Cloth, Maya nCloth and Bullet. This approach avoids the need for much of the more complex maths and instead works by modifying the positions of particles directly. It is not as accurate as other methods but it produces visually appealing results in “a surprisingly small amount of time,” comments Stuyck. He also points to a 2017 Eurographics tutorial course that covers this type of real time positional simulation.

Looking forward with GANs

Cloth modelling and garment simulation have a long history that dates back to the mid 80s. There are two mostly opposing approaches to this problem. One is using physics-based simulations to generate realistic wrinkles, and the other captures and reconstructs details from real-world data.

Stuyck’s book was written in the middle of 2018, but even so new methods have appeared that generate accurate and realistic clothing deformation from a different approach. These new approaches use real captured data. Cloth simulation has been shown to be a prime candidate for using Generative Adversarial Network (GAN) to solve via real data input.

Previous methods for realistic cloth modelling discussed above mainly rely on the intensive computation of physics-based simulations, with numerous parameters. While there has been also been R&D on building models reconstructed from footage, in the past their output has typically suffer from a lack of geometric detail.

At ECCV in August, Zorah Laehner, Daniel Cremers (Technical University Munich), and Tony Tung (Facebook Reality Labs, USA) presented DeepWrinkles: Accurate and Realistic Clothing Modelling.  It proposed a new approach consisting of two modules that work jointly to represent the surface details with high fidelity.

Firstly, global shape deformations are recovered from a subspace model learned from 3D data of clothed people in motion, while high frequency details are added to normal maps created using a conditional GAN whose architecture is designed to deliver realism and temporal consistency.

This all leads to unprecedented high-quality simulation of clothing deformation, where fine wrinkles from (real) high resolution observations are referenced. In addition, as the model is learned independently from body shape and pose, the framework is suitable for applications that require retargeting. Their experiments below, show high quality results with a flexible model. It is an entirely data-driven approach to realistic cloth wrinkle generation.

The team worked on results for two different shirts: one female and one male. They trained the cGAN (cloth GAN) network on a dataset of 9213 consecutive frames. The first 8000 images composed the training data set, the next 1000 images the test data set and the remaining 213 images the validation set.

The paper can be downloaded, found here.

Interestingly the team compared their results to different approaches including a physics-based simulation done by a 3D artist using Marvelous Designer. The MD version returned a mesh simulating similar material properties as their AI result, with a comparable amount of folds, but containing 53,518 more vertices (i.e., an order of magnitude more vertices).