Project Bifröst was shown in Anaheim during SIGGRAPH 2013 for the first time as part of a brief technology demo inside the Autodesk User Group. Bifröst is the code name for an incredible new fluid sim development project of enormous scope. It also answers a question fxguide asked when Naiad technology and its developers joined Autodesk, namely, was the team going into Autodesk to be seen as “The Death of Naiad – a horse’s head in the bed or vast opportunity?”
Our question centered around the sudden departure of Exotic Matter’s Naiad – something that could be viewed as either a small company being lost due to patent or economic reasons, or a chance for the Naiad developers to get the keys to vast resources to make something even better. To be frank, until SIGGRAPH it seemed like it could go either way, but now Project Bifröst closes the book on the debate. Bifröst looks to be everything Naiad was and then some. In short it is one of the most impressive pieces of new tech we have seen from Autodesk in a while and promises to have long term implications over a range of products.
fxguide was given access to the developers to try and explain the thinking and directions that the R&D is heading in. But this exclusive look is not a product review as, quite simply, there is no product yet. Rather we aim to outline the areas that are being developed and explored and what this might mean to artists. To be clear, the developers can neither price nor schedule nor even promise that what we saw will be released, but this look behind the curtain does show enormous potential if they can pull it off.
The name Bifröst is just a code name, and yet it hints at something very significant about this development. If you google Bifröst you will see it is a burning rainbow bridge that reaches between Midgard (the world) and Asgard, the realm of the gods, from Norse mythology. In this context it is a bridge between a few things. Firstly, it is a bridge between procedural and explicit water/gas simulation. But it is also a bridge between the Naiad developers and the original ICE developers (the name Naiad was itself a reference to Greek mythology). ICE is/was one of the great innovations from the XSI Softimage team. In mid 2008, the Softimage announced XSI 7. According to Wikipedia, XSI 7, “introduced the ICE (Interactive Creative Environment) architecture. ICE is a visual programming platform that allows users to extend the capabilities of Softimage quickly and intuitively using a node-based dataflow diagram. This enables artists to create complex 3D effects and tools without scripting. Among the main uses for ICE are procedural modeling, deformation, rigging and particle simulation.”
Naiad was a nodal based fluid simulator which proved itself in production as being able to deliver incredible and subtle water simulations. At the time of the Autodesk deal, the Exotic Matter guys were exploring Naiad 2. Bifröst represents Naiad and ICE meeting on a whiteboard and a completely new technology or engine emerging. Bifröst is not Naiad 2 – although it clearly has Naiad DNA in its algorithms. But then it also has the ICE graphical programming power that has allowed so much in Softimage.
Project Bifröst could be a stand alone product or just as easily could be some form of engine or plugin to Max, Maya, Softimage etc. The technology has been shown inside Maya only as it needed to be shown inside something, but of course the project is not a product, so the user interface shown and the performance are all not the final and should not be seen as representing what it may become. In fact the name is really just a code name for a group of technologies.
That being said, what was shown was incredibly interesting. “Architecturally the system is independent of Maya – it’s stand alone. Speaking technically Bifröst itself doesn’t know that Maya exists, it is completely stand alone,” Marcus Nordenstam told fxguide. Nordenstam was the co-founder Naiad and is now at Autodesk. “Obviously an artist needs an environment to edit the graphs or nodes, and look at the preview of the result, and of course light and render, so we have picked Maya as the first environment to do that in as we know Maya will definitely be one of the environments people would want to use it in. But in the near future you would be able to export the graph you make in Maya and compute it independently in very much the way one might do with a Naiad Graph, and then solve it on the command line. We want to be able to export a sim and run it off the command line, but it opens it up to being finished perhaps on the ‘cloud’, say.”
Interestingly, in this respect Project Bifröst is an engine in much the same way as the recently released Houdini Engine. And again Houdini Engine can run inside Maya, but is not limited to Maya. “It is interesting,” says Nordenstam, “Naiad was sort of a engine – like Houdini Engine – and Project Bifröst very much is an engine, so we have been building an engine and what we are asking ourselves now is which applications to put this engine into. It’s really exciting actually.”
As a cloud implementation is very possible, this opens the door to exploring even a mobile implementation. “That’s the power of what we are doing, we are decoupling authoring environments from compute,” says Nordenstam. “They could be the same, but in theory you could imagine an iPad app where you use your finger to move (Project Bifröst) nodes around, but the actual compute is in the cloud. We are certainly not announcing that, but this is what the architecture could allow.”
Just as Naiad before it, the code for Project Bifröst is written for multi-threading but it is being built in a more distributed, multi-machine approach than Naiad ever was. For example, on Naiad you could not run a single fluid simulation on more than one machine, on Project Bifröst you can. Even with the multi-threading the actual simulation had to be on one machine. A multiple machine solution working on one graph would make the Project Bifröst implementation suitable for say running on the Intel Xeon Phi. But Autodesk is exploring all implementations including GPU implementations.
At the core of the Naiad engine, and this is still true of Project Bifröst, is the graph that defines the simulation. What is new is how the graph is evaluated. Other node-based software (including Naiad) present the graph to be evaluated in the form of a graph, making graph traversal itself a key part of the evaluation. The computer needs to traverse thousands of nodes to compute the solution. Each node may be simple, and often they are, but surprisingly the time taken traversing the data structures is significant.
In the real world, projects produce very complex graphs and so Project Bifröst tries to directly address this for the first time. The team has decoupled the user’s UI representation from the computer’s data graph representation. This means there is an internal system that compiles the user’s graph on the fly into more efficient machine code, and of course all the computer worries about is the output of this. In effect it compiles the graphical nodes into hard coded C++ code on the fly thus dramatically improving performance. Nordenstam suggests that this hard coded code that gets complied and runs is like having an artist working on a shot in the node based view and having “your best friend sitting beside you who says ‘write me your best C++ program that does exactly what my graph does’ – and then that run that for me!”
Critically, this real time compile does not have to just traverse the tree, “it does this magic analysis of your graph and generates the code from the tree in a non-graph form.” To use an example to illustrate this, imagine you have to create four nodes:
- create fluid and then
- add gravity
- do collisions
- cache to disc
In Project Bifröst it would not run the first node, then run the second etc. Instead it would look at everything and then it would explode all those nodes to the lowest level sub-networks, exposing all the sub-nets and having a huge real nodal tree. Then, knowing what all these subnets do (including loops), it produces efficient machine code, and then runs that rather than a series of steps one after another as mirrored directly in the higher level UI. This is much more than an old-school Shake trick of concatenating like-type nodes. This in no way stops one from viewing the process at any stage in the graph, as Project Bifröst recompiles instantly, but just to that point in the graph, by doing a partial nodal tree compile.
The interesting thing is, if Autodesk decides it wants to support a GPU version, then Project Bifröst would just compile GPU code instead. It is a very elegant solution for a multi-targeted product code base. “Not only that,” suggests Nordenstam, “we could say ‘Hey, let’s generate Xbox code, let’s generate Playstation 4 code, for example. This can go quite far. It is the next generation graphing type technology. I think you are going to see a lot more people doing this sort of thing in the future, and we are one of the first.”View-Dependent LOD of Large Datasets: video from Siggraph
The data sizes of fluid sims can overrun the GPU memory limitations when viewing large data sets, such as for large smoke simulations with many billions of voxels. In a program like Maya one would be hard pushed to view it, it would either not fit in GPU RAM or it would cause large amounts of swapping. From an artist’s point of view the user experience would be poor. Autodesk is very keen to make any implementation very interactive, so the team has developed a paging system that pages in low res for distant smoke and only high res in the close up. This is just a viewing tool but the same approach can be found in the simulation side. “That is the other big exciting thing in Bifröst – this adaptive data structure that we are doing,” says Nordenstam.
The adaptive data structure was shown indirectly at the Autodesk Users Group in Anaheim. “We are able to represent a single fluid now finally – adaptively,” adds Nordenstam. “This area for example is higher resolution than that, so we can have a single fluid that actually is fundamentally represented at a higher level on the surface where you want the detail than say at the bottom. And no one else is doing this that I am aware in the commercial space.”
The distributed nature of Bifrost combined with the heavy use of adaptivity led to the development of a native Bifrost voxel structure as opposed to simply using OpenVDB. However, OpenVDB may still be supported as a one of the voxel file-formats.
In a sense Project Bifröst is a visual programming environment. Through the clever use of sub-nets and compounding the team still presents a very simple high level workflow to an artist. But don’t worry if you are a hard core TD, you can still open it up and get in under the hood for some really complex custom solutions.
Is it just for fluid sims and gas sims? Or is Bifröst tech applicable to other simulations? The question is hard to really answer today, but certainly one gets the impression the team has a very long term view. Even before the resources and support of Autodesk came to bear, the former Naiad team wanted to extend their work to rigid body simulations, particles, elastic equations and soft bodies. And Nordenstam feels it is “even more realistic to do that now because we are at Autodesk and they have more resources and great people, so I have even more faith we can succeed.” An example would be deforming geometry based on rules or procedurally in the same way fluids are created.
Naiad is in the DNA of Project Bifröst but the code is all new and it benefits from the experience of the ICE Softimage programmers who joined the ex-Naiad team. It is not just Naiad 2, it is a visual programming environment for fluids that learns from both the graphical approach of Naiad and the power of ICE. An example of this is how the ICE team helped allow the Naiad graph nodes be grouped into super nodes – something not possible in the old Naiad. Project Bifröst has both complex new nodes and the ability to enter nodes and modify them. To write your own nodes in Naiad you had to write your own C++ nodes, with Project Bifröst most of the nodes will be more like ICE nodes internally, and you will be able to adjust them, study them and tweak them without having to write any code.
Away from the technical underpinnings, most artists need tools to say selectively accentuate peaks or for sharpening features, removing holes or smoothing areas. These ‘crafting’ steps are what one needs in production – after all, fluid sims in production are art directed and modified based on the director’s vision and the needs of the story. Here Nordenstam decides to quote and reference Bruce Lee of all people. The analogy is based on Lee’s use of multiple martial techniques to win a bout. “One style does not win the fight,” quotes Nordenstam. “I have taken the same view with the Bifröst design. Proceduralism is a very powerful thing but it is not the only style that can win, and I think if you only use it you will loose, and so my opinion is that interactive workflows – like Maya has traditionally excelled at – things like interactive modeling and interactive animation – that is all very powerful. A lot of the tasks such as fixing holes etc lend themselves to the interactive workflow in the viewport. So we don’t want to force proceduralism onto a user. The trick is to always keep an eye on whenever we try and make a workflow or a task for an artist in the practical world of day to day, we need to ask is this something you want to do in the viewport with a mouse or is this something that you can control with a parameter, is it interactive or procedural?”
At the end of the day, image quality will define the success of the project. For example, does the new Bifröst handle very thin sheets of water (which can be an issue with fluids ‘spider webbing’)? “First,” says Nordenstam, “I would claim that you could do some of that stuff already with Naiad. I would certainly expect that with the same people behind Project Bifröst you will have a lot of the same power and flexibility and a lot of that should be working right out of the box. It should be in pretty good shape.” With reference to the specific problem of thin sheets of water and the surface tension acting upon them, Nordenstam agreed that recent research with explicit mesh based patches in this area is exciting, which is very close to finite element analysis, (and further away from a fluid sim FLIP approach).
Autodesk is actively exploring some hybrid solutions, and Nordenstam says, “I wouldn’t be too surprised if in the future if all of a sudden we introduced some mesh based solution. But a lot of people are working on this and who knows if we will be the first ones!”
Certainly, image quality is central to the research and key to the success of Naiad when it was launched. Nordenstam: “I personally think that one of the key things about Naiad was that there were a lot of things we did not try and do with Naiad, all the things like dynamics. But I really think we did kick the crap out of numerical methods that were stable, and did produce a renderable mesh that did look like liquid and not rice pudding! That was something we really worked hard on.”
Users should expect that level of image quality from Project Bifröst as it is based on the same meshing algorithm as Naiad, slightly improved but similar. The team remain obsessed with great high end quality for major visual effects, and beyond. It is worth remembering that in the Naiad days the guys could only produce particles or meshes or voxels – they had no influence or say in rendering. The company was new and small. Neither of those statements are true of Autodesk. “With Autodesk we have a set of really powerful renderers that we have a really close relationship with, including V-Ray, Mental Ray, Arnold and RenderMan,” outlines Nordenstam. “In time, I expect the quality from Bifröst – with time – will greatly exceed anything Naiad could do.”
This project is aiming to seriously raise the bar over time.
One area of interest to effects artists is the interaction of rigid body and fluid sims. While this is not on the early release plan, in the medium term “the idea of coupled rigid body dynamics and even soft bodies are on the radar,” says Nordenstam. The destruction pipeline ideas seem really advanced and it is too early to outline how it will appear but clearly the team is researching complex debris, fracturing and destruction solutions.
One of the most interesting comments made at the Users Group if you listen to the stage presentation was when Nordenstam referred to skipping the voxel polygonization or meshing step. This leads to direct ray tracing of level sets. In Naiad normally one had to mesh. A standard user workflow would be to mesh, and Project Bifröst will allow meshing – which is needed for many reasons. You might need it for collision detection for example. But to understand the new approach one needs to jump back a few years to a cocktail bar at SIGGRAPH and a meeting between Solid Angle’s Marcos Fajardo and Nordenstam.
Over a set of martinis, Nordenstam and Fajardo wrote an Arnold plugin that rendered Naiad directly, avoiding meshing and this was even used in a 3D shark feature film called Shark Night.
“That’s true,” recalls Fajardo. “It was like 2 or 3 years ago, while we both were in Los Angeles, perhaps for SIGGRAPH? We met at the Marriott Marina del Rey for a quick drink, and Marcus explained how the Naiad API works, and I explained what geometric primitives we had in Arnold. Luckily, the Naiad API already provided a way to evaluate distance fields, and Arnold already provided an ‘implicit’ primitive which allows to ray march an arbitrary field (which I wrote after reading these fantastic slides by Iñigo Quilez). So all the pieces were in place, and it was relatively easy to write a little Arnold shader that calls the Naiad API and then is attached to the implicit primitive. Yeah, the cool thing about this approach is that it doesn’t need to tessellate the fluid, and also that it can capture topological changes in the fluid during motion blur (like when parts of the fluid surface break apart) which are always a pain to capture with tessellation. One of the early testers of this shader was Reliance Media Works in San Francisco for the movie Shark Night.”
Generally for CG volumes and smoke, these are rendered through a density field and are never tessellated or converted to meshes. Plus some people/companies have been known to not mesh back in Naiad – due to relatively slow serial execution and instead mesh in say Houdini. To address this, and learning from the Arnold experience in the cocktail bar, Nordenstam and the team have produced a system that no longer routinely needs to include a meshing stage. At SIGGRAPH this was working in Mental Ray. We asked if this was going to be an option or the standard way of now working. Nordenstam replied, “I have no idea but I can’t wait to find out what people do!”
A whale of a time: a worked problemWhale video shown at Siggraph.
Shown at the User Group during SIGGRAPH was a whale breaching shot created by the Autodesk team. A classic problem with fluid sims is how to combine this with a more general perhaps height field wider ocean simulation.
For example, the whale sim in a much wider shot of the Pacific Ocean at sunset would blend the closeup specialist water sim with the huge simpler wider ocean solution.
Option 1: Mesh
This is the standard approach and Project Bifröst will support the current solution or approach of either rendering two separate animations and combining them in comp, or they would deform the meshes using deformers – weld them together in say Houdini – before rendering.
Option 2: Non-Meshed
Using the new approach, one could turn the large ocean into a volume and voxelize it. You then have two voxel sets – the specialist whale sim and the wide scene sim. “And blending those two as voxels is far more interesting,” points out Nordenstam. “It will look way better than trying to blend the meshes.” This would be something you should be able to do easily in Project Bifröst. “When we get to that point – yes – we will certainly have powerful CSG operations for voxels in Project Bifröst,” he adds.
Option 3: Get rid of the dual approach!
The third option is to get rid of this approach of the very idea of dual sims – the complex and the simpler sims – and just do one. Just sim the whole thing. “You might look at me and say Marcus you are mad!,” Nordenstam exclaims. “But the thing is, the reason we have gone down this adaptive road is for exactly this type of problem, and for this sort of power.” The adaptive approach would allow more complexity on the surface and much less below. This reduces computation and storage.
The adaptive data structure is specifically designed to allow the Autodesk team to shoot for a solution down the road that will not even require a problem like this to be required to be broken up into sub-sims. “We are going to define a region at the surface and we will let the adaptive drop off below that,” explains Nordenstam. A voxel off in the distance will be rendered to say one pixel. We don’t need 10,000 voxels there and we don’t need to simulate 10,000 voxels there. I know this is a bold statement, but this third option could be a real revolution.”
At the end of the day the brief of the team has always been to “fix the fluid volume space and help allow water TDs to approach complex fluid shots and be able to go home and see their families at 6pm,” comments Nordenstam. “And that is still very much what we are trying to do here. There are lots of challenges but we want to make fluid simulation accessible to generalists and also very powerful for the hard core fx TD. And that whole spectrum is what Project Bifröst is all about, and after all ‘the Bifröst’ is a rainbow from our world to Valhalla!”