SideFX and its Houdini team have been heavily honored by the Academy in this year’s Sci-Tech Academy Awards. The first award is an Academy Plaque for four members of the Houdini team, and if that was not enough, the entire company got its own actual Oscar along with Mark Elendt, who was honoured for the fourth time at the Sci-techs. As a highlight of the 2017/18 Sci-tech awards, Mark and the company got their own coveted Oscar golden statuette.
As the big Arts Oscars are this weekend in LA and today (Monday) is Mark Elendt’s birthday, we want to join in congratulating him and the whole SideFX team for their outstanding work with Houdini.
Happy Birthday Mark!
To put this honor in perspective, the Oscar Statuette is not even given out yearly. In fact, for such a technical achievement, this is only the 50th time one has ever been awarded. Moreover, it is being done in the same year as the Academy Plaque is also being presented to the Houdini team. It is a remarkable honor and it clearly demonstrates just how significant Houdini is to all of film making, and not just visual effects.
Officially these awards read as:
ACADEMY AWARD OF MERIT (OSCAR STATUETTE)
To Mark Elendt and SideFX Software for the creation and development of the Houdini visual effects and animation system. At the 84th Oscars, SideFX’ Andrew Clinton and Mark Elendt were awarded for their invention and integration of micro-voxels in the Mantra renderer. This work allowed, for the first time, efficient rendering of volumetric effects such as smoke and clouds in a micro-polygon imaging pipeline. Mark’s work on Houdini, also earned a Technical Achievement Award from the Academy of Motion Picture Arts and Sciences in February 2012.
With more than twenty years of continual innovation, Houdini has delivered the power of procedural methods to visual effects artists, making it the industry standard for bringing natural phenomena, destruction and other digital effects to the screen.
SCIENTIFIC AND ENGINEERING AWARDS (ACADEMY PLAQUES)
To Jeff Lait, Mark Tucker, Cristin Barghiel and John Lynch for their contributions to the design and architecture of the Houdini visual effects and animation system.
Houdini’s dynamics framework and workflow management tools have helped it become the industry standard for bringing natural phenomena, destruction and other digital effects to the screen.
Fxguide spoke to the team, who were naturally elated and somewhat surprised by the extent of honour and respect being shown to their years of work. Mark Elendt has been working on Houdini since day one and is a truly approachable and friendly character, known throughout the industry. It is hard to judge if it is more surprising that he seems to be so humble or he leads a technology team as advanced as Houdini and yet he doesn’t even have a mobile phone! Rather than surf YouTube on a smartphone on the train to work at SideFX, Mark avidly reads science fiction books. He actually started his career doing database work in an insurance company “before I found my passion at SideFX”.
As Mark is being honoured for the fourth time, he joked, “you know, my face may not be as recognizable as George Clooney, but this is my fourth Academy Award !” While he is honored and genuinely humble about his award, he confessed to enjoying “talking about directions that the industry is taking more than I do about awards…Awards are nice and all, but when it comes down to it, you’re being given an award for what you’ve done for the industry”.
Mark Elendt has previously won with Paul H. Breslin, Greg Hermanovic and Kim Davidson for their on going development of the procedural modeling and animation components of their Prisms/Houdini in the 2002/2003 Sci-tech Oscars as well Product specific innovations such as Micro-Voxels in other years.
Side Effect’s ‘Dream’ team’s Industry perspectives
Kim Davidson, CEO and President, joined Mark Elendt along with Jeff Lait(senior mathematician), Mark Tucker (Senior Software Developer), Cristin Barghiel(VP Product Development) and John Lynch (Senior Mathematician) to discuss Houdini, the state of research and what they are researching currently.
SideFX’s Houdini covers all the major areas of 3D production, including:
- Modeling – Most geometry entities incl. Polygons, NURBs/Bézier Curves/Patches & Trims, Metaballs
- Animation – Key framed animation and raw channel manipulation (CHOPs), motion capture support
- Particles – A huge aspect of any Houdini, both separately and as a part of various simulation pipelines
- Dynamics – Rigid Body Dynamics, Fluid Dynamics, Cloth Simulation and flash simulation
- Lighting – Node-based shader authoring, lighting and re-lighting
- Rendering – In addition to Mantra, it supports a variety of renderers: Renderman, Arnold and V-Ray etc
- Volumetrics – Generations/population/manipulation/rendering of scalar- and vectorfields
- Compositing – Full compositor of floating-point deep (layered) images
- Mantra is still the SideFX Houdini packaged renderer. It is very similar in many ways to Pixar’s RenderMan, a renderer that many Houdini customers also use.
The development of Houdini was not driven by a rigorous formal product development agenda. All the developers sit is a large open office and the company relies on water cooler discussions to allow organic internal dialogue. In addition, all the developers are constantly talking to customers directly via the SideFX forums and support lines. It seems haphazard, but clearly it works. “There is a lot of ways clients or customers can talk to us”, comments Elendt. “There are the forums, the bug database, emails, the world wide forums, user groups – there are all these places where we are listening to customers and try to distill what the issues are, and what we could do that would benefit the most users.” It is clear the best way to understand the SideFX agenda is to just sit with the team. This is after all a privately held company that can have a long term focus and is not constrained in quite the same way as publicly listed companies when discussing roadmaps. With this impressive base of developers around the table, Fxguide started our discussion by focusing on Rigging.
Rigging
Viewed from a distance, rigging is not something that Houdini is immediately known for, but many customers and industry observers have noticed that rigging has become somewhat of a strong focus for the company in recent times. It also represents a major area of the pipeline that lacks standardisation and is very much propriety at each facility.
While very separate from Houdini, at this year’s Sci-Tech Oscars a lot of other awards went to riggers and rigging development teams. So it is perhaps a good place to start with SideFX. We put it to the team (to paraphrase an old movie) that while animation is the face of CGI, if not the ‘head’ of 3D excellence, rigging is the neck, and the neck turns the head. If Houdini can be as dominant in rigging as it is in effects animation, it would expand its overall use dramatically, but for today, Houdini rigging adoption faces some challenges, due to both current tools and legacy use.
Rigging spans really two communities who work together, riggers and animators. This reflects the divide of rig creators and rig manipulators. So many facilities have developed in house tools that changing tools is very hard. Houdini, by its very nature, let’s someone get in and ‘get dirty’, which should be an advantage to riggers needing to solve complex problems. However historically, character animators have ended up in Maya and the riggers have followed suit to solve the rigging needs of a project in Maya. The SideFX team admits when many of the animation pipeline decisions were made years ago, Houdini had flexibility but lacked the same user friendliness of some other products. While that may have changed today, many pipelines are entrenched. Kim Davidson adds that “rigging is one of the last areas to standardise and as it is at the core of most pipelines. Rigging is just very hard to standardise… I think we have a great system but the issue is, have we moved far enough with all our other tools to make this now become a standard suite? We see standards in modelling and lighting, but this area of rigging is the last and hardest areas to see any dominant standards in.” Alembic and FBX standards cover some things but exclude pose space deformations, character effects or a range of specialist character rigs and proprietary controls.
SideFX would welcome any third party standards work in the area of Rigging but they just don’t see any on the horizon. In the meanwhile they have been working quietly on improving their own. For example, they have recently new and improved auto rigs for bipeds and quadrupeds, as well as new crowd tools. SideFX believes their actual rigging tools are strong, but the area they feel Houdini is lacking in, is direct animator tools. The team commented that they understand that some animators do not get the same frames per second playback on characters in Houdini, and still other animators find that they can’t have as many characters live in a Houdini session as they would like. The team is quietly confident the actual rigging tools are approaching best in class, but the use of those rigs by animators, and some of the primary animation environment tools needs more work. This is now one of their primary focuses, so that animators can work as fast as possible and focus more on the performance. The team would particularly like to have more GPU power especially in facial animation.
The team have set themselves to a long term goal of surpassing the industry best case for animation interactivity. This is directly being worked on today and will hopefully appear in a point release or the next major release, but they are under no illusions as to the complexity or importance of the task.
Flesh Simulations + Muscle FEA tools
One aspect of creature work is flesh simulation. Houdini is known well for its simulation work and replicating real world phenomena. Facial animation alone can be blendshape, bone-based, FFD, physically based simulation or some combination of these. SideFX wants to lead in flesh simulation especially, given their dominant history in effects animation. In particular, the company is working on facial rigs that are both encompassing traditional rigging and also the Houdini ‘specialist’ simulation approach. This work is not expected to be released “anytime soon”, but flesh simulation is known to both provide great realism and also be extremely computationally taxing, making it an excellent long term project for the development team.
While FEA is used in some destruction simulation, at SideFX most of their FEA is focused on flesh and muscle simulation. The team’s focus in this area is to get FEA volumes to interact in a way that muscles, fat and bones interact in a live creature, but not loose Animator interaction.
Real Time and Game Engines
One aspect for the team is to make Houdini more real time interactive, from facial rigging, muscle simulation, fluids or cloth sims. Interaction time is critical to simulation work and improving iteration time is a ‘huge goal’ for the development team.
“Clearly, we are a long way off scrubbing a full water river bed animation in say UE4, but the developers do see the vital need to allow animators to scrub version, lower resolution examples or approximations in real time both in and outside game engines” comments Barghiel.
Related to their general effort to use more GPU power for specialist animator GUI acceleration, is the growing interest in real time playback and game engines. For some time now Houdini has offered Houdini Engine, which lets users load Houdini Digital Assets into other digital content creation apps such as Autodesk Maya, Cinema 4D or the Unity or Unreal (UE4) Game Editors. One can use the Houdini Engine API to create plugins for proprietary applications. In batch mode, Houdini Engine can also be used on the farm when someone seeks to distribute key tasks such as rendering, simulation, motion and geometry caching.
This playback side of real time is also linked to Houdini been seen as a team player and integrate well into other pipelines and tools. SideFX is particularly working closely with both Unreal and Unity to be as integrated as they can be and provide Houdini inside these key game engines. Real time playback in games does not negate complex long evolving prep work to provide those runtime effects. The team points to the work in Unreal’s Fortnite, which they believe has truly remarkable real time simulations, but they note that it took months of expert work to deliver those impressive effects, such that they now can be played back and interacted with in real time. (See our story on Fortnite’s effects simulations here)
One possible way to leverage Houdini into more interactive runtime effects is deep learning and having Houdini produce synthetic data that feeds a deep learning AI pipeline. This allows for real time interaction in plausibly impressive, but often not highly accurate, AI simulations.
The (possible) Role of Deep Learning
Real time AI simulations based on deep learning limit the effect simulation to the solution space defined by the scope of the training data. The simulations of a river bed can inform and help a real time water sim of a river bed, but this will not inform or help directly with an ocean simulation of a yacht. Houdini is centrally placed to provide a body of training data that can allow seamingly impossibly fast real time interactive effects animation. Beyond this synthetic data role, the Engineering team are exploring all the current Deep learning in VFX papers that are being published with an eye to also using these deep learning approaches in other ways internally. But the team also “don’t believe that the industry will get there in year and a half,” comments Davidson.
“We are keeping our eye on this,” Elendt adds. “And don’t forget, Houdini has been used for on set previs for years. For example, Houdini was used with CHOPS (raw channel manipulation), in the mid 90s on Stealth (2005) to help make sure the motion control camera rig did not break the actual physical rig”. Elendt sees a strong role ahead for the company in virtual production and real time filming in general. The company recently worked with a local Canadian company to provide live simulated particles visualized on set and this work led directly to enhancements in their UE4 and Unity plugins. “We are keen to move up the pipeline for sure, and see more Houdini live on set,” adds Davidson.
Destruction Sims and Fluid Sims
Houdini is perhaps best known for dynamics more than anything else in the wider vfx community. Jeff Lait, Mark Tucker, Cristin Barghiel and John Lynch are all long time employees and as such they all had a hand in the destruction and fluid sims side of Houdini at one time or other. They have all contributed to the dynamics at the core of Houdini. For example, John Lynch started with destruction and fracture tools outside SideFX before he was asked to join the team. Mark Tucker has been at SideFX for a total of 15 years, with a gap of four years.
Tucker points out that for much of his time at SideFX the issue has been scaling up simulations for large scale productions and that is still the case today. At Siggraph each year Tucker almost never attends the technical papers but always focuses instead on the production papers. He goes to hear what problems senior teams have had in achieving some problem, and most often that comes down to a matter of scale. “You’ll hear customers saying they needed a massive scale, or we needed this massive character walking through the water and this is what they had to do to achieve the shot, – and often there is no way to work around it.” explains Tucker. “So a lot of the latest release work has been narrow band stuff. We were looking at how can we get away with simulating less stuff but with the same effect. We want people to be able to do things across a single machine instead of 5 or 6 machines.” The narrow band approach in fluids is to look a fluid simulation of a river and while there is velocity fields simulating all the water in the FLIP Solver, in reality most artists only care about the surface. Houdini can now biases the solution to have high scale detail at the surface, and perhaps just a volumetric representation for the lower water. The program does not need to do a full fluid sim for all the water. This can end up looking almost identical but saves vast amounts of computation. The process is more complex than just looking at the surface. Similarly it could just as easily be solving a level or depth of water at the top and dealing with collision objects on the river bed at the bottom, but just not doing all the fluid sims in between. The savings with such approaches can be considerable even with collisions etc, but this is not meant to be a long term solution. The narrow band approach is designed to help people be happy with the quality of the simulations while also handling much larger simulations and yet accommodating the desire of certain users to stay on the one personal machine, and not extend to a farm render. SideFX has moved, on the whole, to a great position where effects artist are not generally wanting better simulation quality, but more it is an issue of faster and maybe on a larger scale. “Which is great because that was not maybe the position we were in 5 years ago,” comments Tucker.
Jeff Lait is now looking at similar advances in gas and smoke simulations, to again reduce computation, increased adaptability and deal with very large simulations while maintaining artist control.
One thing that has been a constant in Houdini for a long time is the use of Bullet. The majority of destruction using Houdini is still done using the Bullet Library. The new Boolean operator tools introduced in v16 have proven very powerful, with these assets then feeding into Bullet. The team is now working on “complex convex decomposition to take those broken shards and turn them from crazy concave elements into convex elements that the Bullet library can handle much better”, says Lait. The team does have other very complex FEA tools for pre-fracturing stresses and bending etc, for the millions of pieces that a production shot requires, but the team stands behind Bullet as the best way to work currently. (Houdini has had FEA for destruction shots since V13).
Team effort
While the Academy singled out various key senior long term members of the Houdini team, the individuals are quick to point out how much of a team effort the product is and Davidson in particular believes these awards honor all the members of the team.
