One of the more thought provoking talks at this year’s DigiPro conference was by Eugene d’Eon, Chief Scientist at 8i (and formerly a senior researcher at Weta Digital). d’Eon is a world leader in graphics research, and while 8i is a leading VR startup, his talk was on a much wider topic – the parallels and lessons from research done in nuclear physics, among other areas, and how that connects and mirrors – perhaps even informs – CG rendering.
Central to many rendering algorithms is computing the light transport equations, and it turns out that linear light transport theory actually overlaps with neutral particle transport, some of which was published years ago by the world’s pioneers in atomic research. As multiple Sci-tech winner Doug Roble (Digital Domain) joked after hearing d’Eon explain his work, “How many people understand this stuff? Like, (pointing jokingly at d’Eon in the audience) – I think it’s just that guy!”
Thus, it was a rare chance to get an insight into some of the most advanced research areas and how they can provide a real context to CG rendering research. To even access this material required d’Eon to travel to key libraries around the world and to hand read the old paper journals since much of this high level research is not even online, dating from the 40s and 50s.
So what is the common thread that links a new path traced CG teapot with an atomic researcher at Los Alamos decades ago?
Linear transport theory is the study of multiple scattering – bulk scattering – within a random media. d’Eon is a world renowned researcher in sub surface scattering (SSS) skin algorithms and this is directly related to the same multiple scattering density equations and mathematical linear transport theory that nuclear scientist’s face. When dealing with atomic particles or photons, the problem is one of statistical behavior to approximate the results or your wave or particles (light is both of course – that is the essence of the duality of light). At the linear transport level, the specifics of one particle is no longer relevant, everything is statistical behavior.
Of course it is not just skin, fog, god rays or smoke. Atmospheric scenes featuring many different volumes are required every day. While a naive Monte Carlo brute force option is available – on the scale of a film – it is not practical and so we all use complex variance reduction algorithms or deterministic solutions to get the job done. Unsurprisingly, when you think about it, researchers in nuclear reactors or weapon design, optics, remote sensing plasma or even earthquake research hit many of the same issues in estimating multiple scattering. It is easy to think our problems are exclusively the domain of our own R&D gurus but astronomy to animal migration researchers have done similar work – in a sense: different particles, same maths!
Regardless of particles or waves you can solve a bunch of cool problems with this one equation – our much beloved centrally important “volume rendering equation”. In fact, we have been researching the render equation for 30 years and researchers from Jim Blinn to Henrik Wann Jensen have been publishing about it at various SIGGRAPHs in one form or another. It is also d’Eon’s observation that many of these papers have referenced or actually referred to books on light from outside CG, such as Nobel laureate Subrahmanyan Chandrasekhar’s 1960 book Radiative Transfer or Akira Ishimaru’s Wave propagation and scattering in Random Media from the 70s. These books are the foundation for analysis of stellar atmospheres, planetary illumination, sky radiation and much more.
But d’Eon thinks we should look further than just other research areas that have looked at light, he feels we should look to areas like neutron transport – literally nuclear physics of a pre-computer era. “The neutron group often looks at finite heterogeneous configurations, they are Monte Carlo legends and had powerful approximations from times where computers were slow (if they existed at all). A lot of the ‘transport purists’ also seem to come from this camp,” commented d’Eon.
For the nuclear guys, the SIGGRAPH of Transport Theory conferences is ICTT (the International Conference on Transport Theory), held every two years, with the last one being in New Mexico. To slightly paraphrase the 23rd ICTT conference, it brought together researchers from the international mathematics, science, and engineering communities to present recent developments in theoretical models and methods, discrete and continuous mathematical analysis, and numerical algorithms for the solution of the linear and nonlinear Boltzmann equation. The goal of the ICTT is to facilitate the exchange of ideas in classical and emerging areas of radiation transport research through a biennial series of conferences, affectionately known as the “Blacksburg Meetings.”
Topics covered included:
• Kinetic Theory of Gases
• Neutron and Photon Transport
• Charged Particle Transport
• Radiative Transfer
• Plasma Physics
• Quantum Kinetic Theory
• Visual Effects Modeling
• Inverse Transport Problems
… so in other words some seriously, serious research! And on Day One in 2013, at 9.30am on the Monday, d’Eon was invited to present. The ICTT is 46 years old, and to our knowledge this was only the second time a ‘Hollywood type’ presented(!). The first was 2 years earlier when both d’Eon and Jerry Tessendorf presented in Portland!
Of course, d’Eon was heavily supported in his work by Weta Digital at the time, and their reputation along with d’Eon’s is very strong outside just the CG community. Still it was a great honor.
“The group was very friendly, open to discuss and thinking about the problems particular to us, and blown away by what we do with the same equation they use for very different science. They invited me to organize a special graphics themed issue of their journal TTSP and also recently appointed me their librarian,” explained d’Eon.
The last point may seem very curious, why would a graphics researcher, even one as important as d’Eon be made their librarian (even if it is a bit of an honorary invented title) ?!?
It turns out d’Eon has been on a quest, one that has lead him to be uniquely qualified to serve in this role, as odd as that might seem. While at Weta Digital he was doing extensive research on SSS for, amongst other things, the Engineers in Prometheus which were rendered with Weta’s then breakthrough QD (Quantized Diffusion) model of SSS developed by d’Eon and Geoffrey Irving (SIGGRAPH 2011).
As part of his quest to solve SSS in 2010, d’Eon wanted to answer three key questions:
- Why does dipole work? (dipole being the landmark first approach to SSS)
- Could it be improved for curved regions – like ears?
- Could it be modified to better match Monte Carlo techniques of rendering?
In researching these questions he discovered neutron transport theory from the 1940s. “Ultimately interest in these 3 questions led me to discovered works in neutron transport theory that contained some answers,” he says.
It was actually a couple of almost throw-away lines in two books that made d’Eon take note – Ishimanu’s book mentioned above and a medical book on trying to find irregularities that might flag skin cancer called Tissue Optics: Light Scattering Methods and Instruments for Medical Diagnosis by Valery Tuchin.
“The dipole paper referenced Ishimaru regarding diffusion theory,” he recalls, “and one day in 2010 I noticed a comment in section 12-6. It made reference to solving any finite volume problem in terms of infinite medium problems by creating a special surface source and treating the whole medium like an infinite scattering one. And Ishimaru simply cited this book on nuclear reactor theory and moves on with another approach and I was like, wait what was that !??”
The second equally minor reference was a brief comment about “Grosjean’s papers for improving the accuracy of diffusion results – again I was like what? where? …”
d’Eon hunted it down – literally it was in an obscure journal, with weird notation, and barely cited in the 60 years since it was written, but it fed into that Engineer skin required for Prometheus right before the character disintegrates on the edge of the waterfall.
Having now discovered this world of old yet incredible relevant research, “I wanted more,” says d’Eon, “lots more – I wanted it all. And it was a struggle. I had to dig through so many papers on the scattering cross sections, time resolved methods, multi-group multi-energy simulations with single scattering albedos > 1 and plenty of reactor physics. Notation and terminology also took a while to get used to. Most of the text books are old and out of date and expensive.”
But he did, finding for example a book Grosjean published in 1951 with complete and exact solutions for all iostropic point source problems in infinite media and, ignoring d’Eon’s own citations of it, it had only been cited six times in over 60 years. Literally it is a gold mine and no one knew it even existed. By August this year d’Eon had collected over 4,000 papers on transport theory! And thus the role of librarian is an easy position to now see. As most of these articles are only online if published after 1997, d’Eon has had to travel to four separate academic libraries, none of which are of course anywhere near his adopted home of New Zealand or Weta where he worked.
d’Eon has spent three years collecting over 9700 photos which lead to papers such as the 2014 paper on zero-variance based sample scheme for Monte Carlo SSS that was published this year at SIGGRAPH in Vancouver. This work can be directly mapped back to both current research in nuclear physics and work published in the 1950s – and completely unused in CG until now. It references Caseology, “which I wouldn’t have known without haven’t studied the classic texts from the neutron camp,” says d’Eon.
Dr Paul Zweifel, who literally wrote the book on transport theory with Ken Case in 1967, joked about Caseology during the 50th anniversary talk at ICTT following d’Eon’s presentation. He joked that ‘there have been hundreds of PHD theses written about Caseology across a large number of scientific fields such as medical physics, but maybe now Caseology can be used for something important like making movies !’
The work does not stop there. A second paper at this year’s SIGGRAPH called Refractive Radiative Transfer Equation was similarly informed by prior work way outside computer graphics. And the paper fxguide highlighted just prior to SIGGRAPH – Unifying Points, Beam and Paths in Volumetric Light Transport Simulation – there was similar or rather relevant work in this area broadly speaking in the mid 1960s. But it is not all a one way street, in our world of CG there is little work that would equate to bi-directional path tracing and the CG concept of multiple importance sampling – so important since the work of Eric Veach (he won a Sci-tech for his 1997 thesis this year).
During his talk, d’Eon went on to discuss new possible areas of research such as non-classic transport (from a very high temperature so called the pebble-bed nuclear reactor which uses a non-exponential free path transport), and compressing complexity (a new statistical way of dealing with millions of collisions of say clusters of glass objects) and even fractal fog – which appeared at first glance to be extremely promising (see below).
At least four different groups or fields of science are independently are looking at the same generalization of transport theory that includes the classical exponential form as a special case:
- Graphics (Moon et al. 2007),
- Remote sensing (Davis and Marshak),
- Pebble bed reactors (Larsen and Vasques).
The good news is that it is relatively easy for a serious programmer to include this work in a physically-based renderer.
This image above for example is of “fractal fog” and was done with a modification of the Mitsuba physically based renderer to include non-exponential transport as a first-class citizen of the renderer by computing next-event estimation with a non-beer-lambert attenuation and sampling free path steps using the generalized FPD (free path distribution) functions. To the best of d’Eon’s knowledge, it is the first image to include this new form of generalized transport theory in a path tracer.
d’Eon noted that like all researchers he follows in the “footsteps of giants”. He recalled a paper from 1990 where Russian Roulette and splitting were introduced by James Arvo and David Kirk, “which was one of the first strong messages about looking at the particle side of transport literature. Arvo and Kirk were the first proponents of the particle transport message for graphics”. He also flagged that Eric Veach has a section in this 1997 thesis that describes the relation of rendering to neutron transport “very clearly”. But this is not a widely referenced part of Veach’s landmark thesis and d’Eon has taken the area farther than anyone else fxguide is aware of, and along the way delighted a whole community of nuclear physicists and mathematicians – as well as helped advance skin rendering and much much more.
His closing remarks were to point out that these complex issues and maths we face are not ours alone to solve and radiative transfer is just one of several possible paths of research. Furthermore, based on the comments of the, let’s face it, card carrying geniuses that d’Eon spoke to at ICTT, we are doing great work even by nuclear rocket science standards here with our entertainment use of the rendering equation!
– so that’s good to know.