December 06, 2017
Written by Harsh Agrawal and Leif Pedersen.
Based on developments by Christophe
Photorealism has been the holy grail for computer graphics, especially in the visual effects world, where recreating digital humans or matching live action film plates is paramount. In recent years, stylized feature film animation has also shifted towards the same need, with an ever-increasing demand for shading and lighting realism. This has pushed the animation industry into physically-based rendering in order to push the visual complexity.
To create such realistic images, attention to detail plays an important role, because when it comes down to judging reality, humans are capable of subconsciously catching the most subtle anomalies. This minute attention to detail is otherwise trivial in our daily life, but recreating it can be a challenging task. Pixar technical artists took on this challenge in order to bring the Cars universe into a new standard of realism.
Current state-of-the-art images are a result of various leaps we have taken, like switching to physically-based rendering systems and improving on shading algorithms, for example, Marschner for hair illumination, or
Most materials on a microscopic scale contain an incredible amount of surface variation and undulations. Even a smooth looking Hard Disk platter, one of the most reflective surfaces, second only to a mirror, contains a lot of variation when inspected in detail. It might not be visible in the image below...
...but micro-detail on the surface of the platter is clear when using a magnetic force microscope.
Even though presenting microscopic
Such minute details were not taken into account until Cook and Torrance  introduced the microfacet shading model. The idea of the microfacet approach is to characterize the portion of a surface visible at a sample by a distribution of surface orientations, and use the distribution to calculate an appropriate specular reflectance. Meaning, thanks to the work by Cook and Torrance, the surface itself communicates minute changes in shading to the renderer.
So, problem solved, right? Not entirely. Though these microfacet approaches take surface orientation into account and give us realistic results while shading, the problem is that they don’t take into account the geometric microfacet details contributed by Bump or Displacement mapping which can end up inside a single pixel and get filtered out.
In practical terms, the relationship between the texel (texture pixel) and
At this point, these geometries are acting as microfacets themselves and thus should be contributing to microfacet calculation. Bump-Roughness is a way to take these geometric details and convert them to roughness when filtering. This avoids the dependence of brute force sampling to find these micro details, such as the example below. In this example, we're simulating a
Let's look again at our hard disk platter example. As we can see from these two renders, Bumps are visible with traditional methods, but they're not acting like microfacets and contributing to the overall roughness of the hard disk platter. Bump-Roughness makes all this possible while improving the accuracy of the specular anisotropy, which leads to further realism.
So what does Bump-Roughness do from a practical standpoint?
Hero characters in Cars 3, such as Louise Nash (shaded by Yeon Kang), have an extensive wear of the clear coat layer across their paint job, this wear can create interesting patterns which break up the specular in interesting ways. Bump-Roughness helps us keep these micro details in medium and wide shots, as shown in this closeup example.
Not only is this an improvement on technology, but it is a storytelling tool to communicate wear and aging.
-Junyi Ling, Character Co-Supervisor Cars 3
Another great example is this work-in-progress test. The clear coat scratches are a bit more exaggerated than the film's final frames, in order to showcase the effect. In previous films, such micro detail would be lost at this distance or it would be expensive to render due to increased sampling or small shading rates necessary to find such details.
“Storytelling is always the focus at Pixar, so we sometimes needed to dial the effect down to make sure our focus was on the characters, not the technology,” says Junyi Ling, character co-supervisor on Cars 3.
Another great feature about Bump Roughness is its built-in LOD (Level of Detail), which gives artists the right amount of detail at every distance. "The automatic LOD trickles down to other departments, for example, the lighting department no longer had to spend lots of time finding paint flakes with the right lighting angle," said George Nguyen, Character Shading Lead on Cars 3. This is crucial with characters such as Jackson Storm, where a new lacquered paint job and visible paint flakes are part of his shiny "new-guy" persona.
Even though Bump-Roughness is great for paint flakes and clear coat scratches, its usefulness doesn't stop there, it can be used to significantly improve render times by adding detail without having to increase sampling. Also, the tech benefits a wide range of materials, from carbon fiber to textiles as shown below...
...but it's also incredibly useful in organic and heavily displaced surfaces because we can keep microfacet detail at more coarse sampling variance settings, as illustrated in this Lizard render.
-Junyi Ling, Character Co-Supervisor Cars 3
Notice all the lost displacement and bump detail when using 1 sample per-pixel. We need to use 100 samples per-pixel to bring back the original look-dev on a wide shot. Bump-Roughness, on the other hand, can use only 1 sample per-pixel while keeping the same visual appeal, as illustrated in the closeups below. The best part is that "the added detail is gained out-of-the-box and the artist experience is seamless because it all happens automatically when RenderMan converts the textures," said Junyi Ling.
Another big success story for the technology has been skin. "We've been using Bump-Roughness on human skin with great success," says Junyi Ling. "Specular detail on pores and other previously lost detail is more vivid than ever thanks to Bump-Roughness. It's taking realism to a new level at Pixar," he added.
In the image below we can see how Bump-Roughness is maintaining highlight information more accurately, which improves the specular breakup in pores, this is even more pronounced when zoomed out. "Refining the specular model was crucial to revealing textural detail. We were able to light shots more accurately without resorting to tricks," said Jean-Claude Kalache, lighting DP at Pixar, "before, we had to rely on manual light rigs to reveal minor detail at glancing angles," he added.
To see how much he could improve his renders with the new Bump Roughness tech, Jean-Claude Kalache did several lighting tests to match photography. "Since the beginnings at Pixar, we've tried to make skin look realistic, and I thought Bump-Roughness could get us a step closer," said JC. "Traditionally, at
The sets department was also able to leverage the developments with Bump-Roughness. "We noticed a significant improvement on environment detail, especially surfaces such as concrete and marble," said Junyi Ling. "Typically with Marble, we try to avoid a glassy look, where the marble is too shiny. With Bump-Roughness we were able to capture all the subtle imperfections that give marble its most distinctive properties," said Junyi.
The benefits don't stop there, as previously mentioned, Cars 3 saw huge improvements in render times. Bump-Roughness is over 35% faster than bump alone because RenderMan no longer has to calculate all 3 vectors, it now uses the newly supplied vector to decrease the shading context complexity. "Using Bump-Roughness was fundamental to achieving this level of detail because supersampling was never an option...render times would have been too long," said Junyi.
-Junyi Ling, Character Co-Supervisor Cars 3
Overall render times are also reduced when surfaces are shiny because the BXDF is importance sampled, this means the secondary ray is also importance sampled, and so on...this gave the Cars 3 team upwards of 30% reduction in render times in many scenes, and even though scenes with rough specular surfaces saw fewer render time reductions, there was never an increase in sampling times. "Future feature films at Pixar are using Bump Roughness with great success, from skin shading to shading most arbitrary assets, the improvements to detail preservation are the new status quo for surface quality," said George Nguyen.
This overall improvement in sampling variance has led Pixar to new heights of image quality when defining micro detail, in which quality doesn't have to compromise render times.
Looking ahead is always in Pixar's DNA and Christophe Hery, Head of Pixar illumination group and research scientist, is already planning to expand RenderMan's Bump capabilities even further by implementing leading work in the field. "There is more work to be done, in regards to Bump in general. The remaining big issue with Bump is that it is not reciprocal (at least as currently implemented) and we resort to alternative solutions (such as nudging the bumped normal to stay on the visible hemisphere, i.e. on the same side as the geometric normal), in order to keep a consistency with bi-directional transports. This might lead to dark non-lit areas on a piece of bumped geometry. To help avoid these issues, we are investigating several compelling ideas including Unity Labs' research on Microfacet-based Normal Mapping for Robust Monte Carlo Path Tracing," said Christophe.
Photorealism is a never-ending journey, "it's so fun to get carried away by the possibilities of visual complexity" said Junyi, "but at the end of the day, technology has to help the visuals in a subtle and useful way, not the other way around, the focus should always be the characters and their journey." This is the balance between technology and storytelling we strive for at Pixar.