Sébastien Lagarde explores the world of water for accurate wet environment simulations in games. This is third part of our series where we explore game environments. Read parts one (on rendering) and two (on rain).
In the earliest stages of the game planning it was clear streets and environments could be either wet or dry, below is the exact same street rendered as concept art dry and wet. Note how expressively the right image looks wet, and how the wet surfaces even here are darker than the same surfaces when dry.
The aim here is to help in the understanding of physical processes involved in rendering wet surfaces. Wet surfaces are really complex to get right and the complexity involved is particularly difficult to master in the context of game development (dual layering, porosity, subsurface interaction, etc). It is impossible to explain or highlight everything but this should give an insight to the complexity of game environment development. As the context is high end gaming, the solution will lie not in forcing new complex lighting models but rather tweaking BRDF parameters from dry material to achieve some active realism while remaining inside a render budget. However, this can be done with the benefit of using a physically based lighting models with physically based values. From the correct physical parameters one can identify the nature of surfaces. Building out from this the approach can also provide useful information when dealing with procedurally based aging and weathering, and many other similar gaming environment problems.
Physically Based Rendering of Surfaces
Physically based rendering (PBR) is now common in games. When Sébastien Lagarde introduced PBR in his company a few years ago the team was actually working on rain. At this time they questioned if the new lighting model should be used to simulate wet surfaces? With classic game lighting models, the way most people chose to show wet surfaces is to darken the diffuse term and boost the specular term. The wet diffuse/specular factors being eye calibrated. Lagarde wanted to go further than simply adapting this behavior to PBR and thus required a better understand the interaction between water and materials. This story is a mix of the research and findings. “The post describe how water influences materials and provides ways to simulate wet surfaces with a physically based lighting model,” he explains.
Section 1: Reference
Wet Environments? Reference from real life
For this study it was decided to focus on wet urban environments rather than natural or landscape scenes.
The first thing one notices when it’s raining in the night is the long stretched highlight reflection of the bright light sources.
Highlight reflection varies with the roughness of the underlying surface.
The anisotropic reflection appears to follow a standard ‘Blinn-Phong’ behavior as seen above.
The highlights get dimmer when the surface is rougher. This is energy conservation (above)
When the surface is smooth we have a perfect reflection (left), whatever the distance. The right image above shows how a distributed surface can make the green light appear twice:
What is important to note is that the reflection ‘fuzziness’ depends only on
- view angle,
- light angle and
- index of refraction (i.e specular) +
- roughness of the surface
Water accumulates in low depth level first which helps to break up puddles. (This is an important technique we see deployed in gaming to break up large repeating surfaces below).
When there is enough water accumulated, it is sometimes possible to see a distinct double layer behavior. We can see both a water layer and an underlying surface layer as in this image above.
The reflection of the water layer follows the Fresnel law. This mean there is full reflection at a grazing angle (~0°) and low reflection when looking perpendicular to the surface from above (~90°).
Looking at the last pictures of both of these two series above (i.e. from above looking straight down), one can just make out the photographers body not reflected very clearly. With an index of refraction of 1.33, water reflects only 2% of incoming light for a 90° angle (vastly less than at the other end of the series when the camera is glancing across the top of the water). Depending on the brightness of the incoming lighting and the albedo of the underlying surface, the reflection can appear faint or may even be totally swamped by the other light sources. In the left case below the photographer is not bright enough be seen as a reflection, but the light coming from the sky is strong enough and can be seen as a very light white tint in the water (overcast sky). In the right case, we can distinguish color appearance of the reflected building because the sky light bounces on it whereas the photographers body is facing the ground and does not bounce enough light.
When underlying surfaces have dark albedo and you have a clear sky, low disruption, the water surface behaves almost like a mirror.
To produce game effects, Lagarde filmed a huge amount of reference material, not to use, but to study. Above is a selection of clips showing the points raised above. For example, as seen at the start of the reference video, rain causes ripples only where there is sufficient accumulated water. Ripples are concentric circle interacting and the number is linked to rain intensity. Wind has a major influence on water puddles (1:34).
Key to observe is that most wet surfaces are diffusively darker and specularly brighter in an urban environment, as seen in this image below.
Urban environments clearly need to be populated with a wide range of props and in a city environment many of these will be made of plastic or metal. Interestingly, their properties are slightly different from pavement or road surfaces.
But not all materials are subject to a darkening of the diffuse term, with the exception being plastic or metal.
When drying, specular strength disappears faster than the darkening of the diffuse. Of course, surfaces don’t dry at the same speed (Left is totally wet, middle is drying, right is almost completely dry).
Section 2: Implementation
People are able to distinguish between a wet and a dry surface by sight. The observation post show many pictures to illustrate this point. The main visual cue people retain is that wet surfaces look darker, more specular and exhibit subtle changes in saturation and hue when wet.
This behavior is commonly observed for natural or human made rough material/porous materials (brick, clay, etc…), powdered materials (sand, dirt, soil…), absorbent materials (paper, cotton, fabrics…) or organic materials (fur, hair…). However this is not always the case, smooth materials (glass, marble, plastic, metal, painted surface, ..) don’t change.
For example, there is a big difference between a dry rough stone and a wet rough stone on a street but a very small difference between highly polished wet stone and highly polished dry stone in a kitchen. We will focus on wet surfaces that are mostly rough and diffuse materials quenched in water and having a very thin water layer on their surfaces. as this is the most common types of materials found in our target urban exterior environments.
Why rough wet surfaces are darker when wet? Because they reflect less light.
There are two optical phenomena implied in this decrease of light reflection:
- A rough material has small air gaps or pores (which will be filling by water when wet), and
- When the pores are filled, there is “water saturation”, water ends up on the material as a thin layer.
Let’s first see the impact of the thin layer of water. The rough surface leads to a diffuse reflection. Some of the light reflected from the surface will be reflected back to the surface by the water-air interface due to total internal reflection. Total internal reflection occurs when moving from a denser medium into a less dense one (as in fibre optics). Of course it is not as simple as one reflection based on partial internal reflection, since this bouncing light is being scattered, the reflected light from the surface is then subject to another round of absorption by the surface before it is reflected again. This light’s back and forth result in darkening of the surface.
As to the original point above – of filling the small air gaps and pores – the new water has a higher refractive index than the air it replaces. (1.33 vs Air at 1.0) This new water soaked material thus has a refractive index which is closer to the index of refraction of an even rougher (dielectric) (non reflective/conductive) material. The scattering of light under the surface is more directional in the forward direction. The increased scattering before the light leaves the surface increases the amount of light absorbed and thus reduces the light reflection. In other words it appears darker.
The darkening of the material is also accompanied by a subtle change in saturation and hue. In the spectral reflectance testing it can be shown that’s the surface color becomes more saturated because of this reduction. This is a property of the different wavelengths of light being scattered differently. As a wet material reflects long “red” wavelengths more than short “blue” wavelengths, it appear that the objects color has shifted in hue and become a more saturated color.
Why rough wet surfaces are more specular?
The air gaps of the rough surface are progressively filled with water and water has almost a mirror-like behavior. In simple terms, the material becomes smoother. A polished or coated stone and a wet stone are very similar – both look like smooth surfaces.
In the case of a smooth material, the water doesn’t fill inside the material and does not produce darkening from within the material. But there is still some of the darkening effect from the thin layer of water on top as described above.
From a rendering point of view, one needs to take care of both optical phenomena: the one due to a layer of water on the surface and the other considering water inside the material. A two-layer surfaces reflection and subsurface scattering lighting model was presented by Jensen et al in “Rendering of Wet Materials” in 1999. The surface model they provided was a thin layer model, with a thin layer of water between the external air and the denser material. The dense material is modeled as scattering volume.
This great lighting model was not proposed for games and is too expensive to be used in real time.
But the notion of solving both parts of the problem is still what the team at Remember Me dealt with. In other words solving a thin layer and some type of variation in sub surface scattering effect from environments being wet.
Realtime double layer material
To solve this fundamental rendering problem for a real time game application and thus allow the level of realism the team wanted in Remember Me, Lagarde needed to have a double layer BRDF, the top layer BRDF will be the water and the bottom the original BRDF. There was one usable real time implementation which he decided could be adapted for their case – a model from Weidlich and Wilkie.
- The BRDF of the topmost level (1.) is evaluated for the two given, arbitrary incoming directions wi, and wo. (seen here in pink and blue). This yields a reflection component, and, normally, two refraction directions.
- Any energy that is refracted into the next level (2.) follows the two refraction directions and is partly absorbed.
- These two refraction directions are assumed to meet at a single point on the next layer (2.), and the process is repeated from step 1 until an opaque layer is hit.
- On returning from the bottom, the individual BRDF components are affected by the Fresnel transmission coefficients for the level above them, and added to the total BRDF.
In other words, they refract down, until they hit an opaque layer, and then bounce back up, but subject to the refractive index (which could cause a fully internal bounce back down).
Using a simple layered surface model like this, the team demonstrated how a surprisingly large number of interesting and important surface types could be efficiently rendered in the game engine. From the artists point of view they also showed how handy the approach was, as it works with only a few intuitive parameters.
Weidlich and Wilkie presented a layered BRDF combining several micro-facets BRDFs into a single unified model that accounted for internal reflection and absorption, and each ‘layer’ can have any arbitrary BRDF. This became the approach for surfaces with a thin layer of water on top of the material (but not say a puddle – see below).
Here is a summation of the process thus far:
We began this article by studying the influence of water on lighting for wet surfaces.
Then we have seen some real time implementations with analytic light for both optical phenomena which were based from the observation. They have some drawbacks – they miss an image based lighting implementation and their cost for in-game usage is still a problem for XBOX360/PS3 games. In computer graphics there is a different path available to simulate optical phenomena other than changing the lighting model. One could simply create/edit/capture both wet and dry surfaces BRDF parameters (diffuse, specular, specular power…) with the same lighting model. This is not really a “simulation” as one knows the final wet state of the surface but you can now render wet surfaces and to dynamically wet the game world. Simply by interpolating between dry and wet BRDF parameters without changing the lighting model!
Why not then just have two sets of textures the Wet and the Dry?
The benefit is the simplicity of the process and it is still compatible with any kind of lighting such as image based lighting. The drawback of having both a dry and a wet set of BRDF textures is the time to author and store them. A full wet lighting model approach required more instructions whereas this wet/dry approach requires only few extra instructions. However in game development, doubling the storage in memory/disc space and the number of textures to author is prohibited.
The solution – try and find a way to tweak the dry BRDF parameters to get an approximation of the wet BRDF parameters and thus avoid the inconvenient of storing and authoring new textures.
“Almost all games I know currently choose to follow this BRDF parameters’ tweaking path, for example: Stalker, Uncharted 2/3 (on the main character), Assassin’s Creed 3, Crysis 2/3, Metal Gear Solid V etc…”, explains Lagarde.
This is not surprising as the method seems simple and it fits very well with a deferred shading renderer: “You can tweak dry BRDF parameters in the G-buffer without complicating the lighting systems. However the wet BRDF parameters generated by these games are either coarse or wrong approximation (in a physical sense, I agree that visually the look can be OK). Most use the same eye-calibrated factors to attenuate diffuse and boost specular (old fashion) on every wet surfaces of the scene regardless of material properties (roughness/smoothness, porosity, metalic/dielectric…). Assassin’s Creed 3 even does an additional wrong step by changing the strength of the factor based on the type of rain: he says referring to the fact that any type of rain on a porous surface can make it water saturated. “A bit differently Tomb Raider : A Survivor Is Born, uses a “dark light” to attenuate the light receive by the diffuse part of the wet surfaces, the specular part is modified as other games. As they use lights to produce rain with a light prepass renderer, I think they intent to make up the missing of a diffuse parameter in the small G-Buffer with this method. Which again apply wrongly the same modification factors on all dry surfaces.”
One of the purposes of the remainder of this section of the article is to improve the BRDF wet parameters generation from the dry one. We begin by talking about the tweaking of the diffuse (or subsurface scattering part) and the specular parameters for porous dielectric material then for other kind of materials, and end with the effect of the thin layer of water which can accumulate above surfaces and the case of thick accumulated water like puddles.
Porous dielectric material
The team needed find a factor which can be applied on a dry diffuse parameter to get a wet diffuse parameter and equivalent for the glossiness parameter.
Looking at other implementations and solutions, for example: the asphalt in the driving simulator, Nakamae, a factor between 0.1 and 0.3 to attenuate the diffuse albedo and a factor of 5 to 10 to boost the specular was used. As many approaches, this is done without taking into account the properties of the surfaces (like roughness).
The key is to better explain the darkening of the albedo. Clearly the team wants to find an equation where for ANY albedo in a DRY urban environment there is a way get the WET value. Since in the game there is a huge range of materials and they need to be at various time dry and wet.
Life would be simple if there was a simple relationship such as wet = a 90% of the dry value of albedo. The team discovered that the relationship or the forumla is a bit more complex. There is actually a non-linear relationship, a bit like a gamma curve – that maps between Dry and Wet as you increase or decrease the albedo of a surface.
The highest differences between wet and dry albedo occurs for surface in the middle range of dry albedo. Dark surfaces will tend to absorb more light on first contact with the surface, the contribution of internal reflections will be less important – so decreasing the effect of wetting. Bright surfaces will tend to reflect much more light than is absorbed – also decreasing the effect of wetting. In both cases the relationship between dry and wet albedo depends only on the index of refraction (IOR) of the surface, the IOR of the water and the dry albedo.
Having this curve provided the Remember Me team with a key to only store one BRDF parameters set and generate the other. This formula provides a great way to then store only the Dry (or Wet) value and calculate the other value on the fly rather than storing both numbers.
End of the problem? Not quite.
As with the reference above to the game Nakamae, Lagarde was concerned that the solution not just be based on albedo and IOR properties of a surface. From all the reference at the top of the story he knew that porosity is an important factor as is the roughness of the surface.
Other research had seemed to also go in this way. The team therefore explored material databases but did not find a useful simple solution or another formula relationship that would factor in how pouros the surface was or how rough the surface was, yet they knew both affected this darkening when wet key rendering point.
“We now know that’s low albedo, rough and porous materials tend to have larger wetting effect. We also know that high albedo, smooth material or no porous tend to be less affected by water. Roughness and porosity should be considered as two invisible micro-geometric features. But roughness and porosity are certainly correlated in some way. And it seems that in the specific case of rock there is a linear relationship but we can’t generalize. It is non linear for some rough surfaces and there is rough surfaces that are not porous at all e.g vinyl, leather, etc.”
In the absence of any published work, “for the context of our game development – we will just rely on intuition, that’s why we linked porosity and roughness”, and the Remember Me team offered an additional control to the artist, including a porosity mask and extra controls.
Puddles
When the team had just a thin layer of water there was still a reflection corresponding to a disturbed normal from the underlying surface. But for a thicker layer of water – like the puddle it is possible to have a perfect reflection coming from flat surface normal. In other words the surface is flat – even if the bottom of the puddle is not flat. Thus the team gives the artists a progressive blend between normal map and vertex normal can provide visually correct result for increasing height of the water layer. One way it means the puddle’s reflection ignores the surface under the puddle, the other way it is affected by what’s below.
To sum up this section, the team ended up with controls:
- for darkening of the diffuse and bosting the specular (wet surfaces not real surface water)
- for thin layers of water on top of surfaces, – a small amount of water in control called AccumulatedWater causing a slight smoothing of the normal and a slight boost in specular (smaller highlight, better reflection definition, increase in intensity).
- For puddles – the ability to have have totally flat and reflective water.
The system described here provided good results, was simple to implement and is suitable for XBOX360/PS3 games.
The last step was to define and refine a system to dry out wet surfaces realistically. Assuming dynamic weather, the sun may come out and they would need to solve both the drying and wetting process.
The process of drying is more complex than it appears at first. In the observation post, we have seen some drying patterns. The drying process is non-homogenous.
Moreover the wet surface shows a spatially-varying reflection. When drying, the specular reflection disappears but the diffuse reflection is still darker than the dry surface.
A complete study of these drying have been done by Lu et al. (2005). The speed of drying differs between spatial location due to object geometry and shape of the wet surface. The drying is also linked to exposure to say the sun which in turn affects drying, air temperature, plus there is exposure to fire…, air motion (wind), gravity and layout of the fluid within the material (the distance of any point on the surface from dry material has great influence). See Lagarde post for more theory of drying.
The final system the team designed allowed for dynamic rain, varying conditions, surfaces to get wetter and to dry out, it allowed for the same models and textures to be wet or dry with the same lighting model and it allowed a huge amount of flexibility in between.
Ironically….
In Remember Me the game ended with static weathering condition (!) – “we have used only a small part of what was presented here. The rain being static – we created wet textures for everything. This is a gain in quality as artists have full control on the rendering and in performance. This is the moral of the story, fit your features to your game context,” comments Lagarde.
Conclusion
Above are many factors that help in understanding the physical process involve in the darkening and brightening of wet surfaces. Wet surfaces are really complex to get right and the complexity involve is difficult to master in the context of game development, “I myself still don’t get all, but hopefully you can see that we are not forced to construct a complex lighting model to get good wet surface feelings, simply tweaking BRDF parameters from dry material can do the job. The main point I want to highlight is the great benefit of using a physically based lighting model with physically based values. From physical parameters you are able to identify the nature of surfaces. Useful information when you deal with procedural effect based on aging and weathering,” Lagarde explains.
The takeaways points are:
– Darkening and specular are due to two optical phenomena
– The change due to water induce an increased saturation, a hue shift and a brightness decrease of the diffuse part and an increase smoothing of specular part
– Only porous materials are affected by rain
– Example of complex lighting model to handle wet surfaces with dual layer/porous BRDF
– Few simple ways to handle wet surfaces by tweaking BRDF
– Using a physical lighting model with physical values allow to identify the nature of material
– Porosity is a parameter to consider for dynamic aging and weathering effect
– Drying process is a complex, drying is non homogenous and wet specular vanish quickly compare to we diffuse.
To conclude, this article does not cover everything, nor did the team cover every aspect of material wetness (such as increased transparency when say paper is wet), but it highlights the amount of work that goes into just getting environments looking wet in a modern complex and highly engaging game. Especially when faced with the memory and performance requirements of a high end game in the real world.
Our thanks to Sébastien and the team for allowing fxguide to show their great work on Remember Me.
____________________________________
Part 1 and part 2 of our series focused on rendering and effects animation of rain.
Note: This article is an edited down version of an article in a different form that was published on Lagarde’s blog. Mike Seymour worked with Sébastien to publish this edited version for fxguide. (The longer version includes pseudo-code and can be found at his blog.)
Pingback: The Street – Other Practitioners | Emerald Sword Gaming