Photorealism has been the holy grail for computer graphics, especially in the visual effects world, where recreating digital humans or matching live action film plates is paramount. In recent years, stylized feature film animation has also shifted towards the same need, with an ever-increasing demand for shading and lighting realism. This has pushed the animation industry into physically-based rendering in order to push the visual complexity.

To create such realistic images, attention to detail plays an important role, because when it comes down to judging reality, humans are capable of subconsciously catching the most subtle anomalies. This minute attention to detail is otherwise trivial in our daily life, but recreating it can be a challenging task. Pixar technical artists took on this challenge in order to bring the Cars universe into a new standard of realism.

Cars 3 © Disney/Pixar

Cars 3 benefited from RenderMan's physical shading and lighting.

Current state-of-the-art images are a result of various leaps we have taken, like switching to physically-based rendering systems and improving on shading algorithms, for example, Marschner for hair illumination, or fully path traced subsurface scattering. Each of these steps improves upon the problem of bringing us within the grasp of photorealism, which brings us to the new steps at Pixar, and a particularly important one for car rendering: keeping minute detail, such as scratches, paint flakes, and other subtle effects, especially when an object is far away from the camera. This is exemplified in two of the latest research papers at Pixar: Pixar's Foundation for Materials and Geometry into Shading.

Most materials on a microscopic scale contain an incredible amount of surface variation and undulations. Even a smooth looking Hard Disk platter, one of the most reflective surfaces, second only to a mirror, contains a lot of variation when inspected in detail. It might not be visible in the image below...

CC BY-SA 3.0 - Eric Gaba, Wikimedia Commons user Sting

Hard disk platters are some of the smoothest surfaces in the world.

...but micro-detail on the surface of the platter is clear when using a magnetic force microscope.

Even though presenting microscopic closeups was not a part of the cinematic goals of Cars 3, determining how they affect the overall look of surfaces was crucial to realism. 

CC BY-SA 3.0 - Author: Januszartur

Hard disk platters at micron scale.

Such minute details were not taken into account until Cook and Torrance [1982] introduced the microfacet shading model. The idea of the microfacet approach is to characterize the portion of a surface visible at a sample by a distribution of surface orientations, and use the distribution to calculate an appropriate specular reflectance. Meaning, thanks to the work by Cook and Torrance, the surface itself communicates minute changes in shading to the renderer.

So, problem solved, right? Not entirely. Though these microfacet approaches take surface orientation into account and give us realistic results while shading, the problem is that they don’t take into account the geometric microfacet details contributed by Bump or Displacement mapping which can end up inside a single pixel and get filtered out.

In practical terms, the relationship between the texel (texture pixel) and pixel is always changing, so filtering becomes an issue. Objects can have surface details through Displacements and Bump maps, but even with a microfacet shading model, when the object is zoomed far away a lot of these geometric details can end up inside a single pixel and thus the renderer discards this detail as meaningless. This is where Bump-Roughness steps in...

At this point, these geometries are acting as microfacets themselves and thus should be contributing to microfacet calculation. Bump-Roughness is a way to take these geometric details and convert them to roughness when filtering. This avoids the dependence of brute force sampling to find these micro details, such as the example below. In this example, we're simulating a pathtracing sampling pattern with some red and green dots; as you can see, when we zoom out, it is possible that a scratch will not be sampled, forcing the user to increase sampling so that the renderer can find these small details. This can have a prohibitively long impact on render times.

Let's look again at our hard disk platter example. As we can see from these two renders, Bumps are visible with traditional methods, but they're not acting like microfacets and contributing to the overall roughness of the hard disk platter. Bump-Roughness makes all this possible while improving the accuracy of the specular anisotropy, which leads to further realism.

Bump affects the anisotropy and roughness which leads to greater detail.

So what does Bump-Roughness do from a practical standpoint?

Hero characters in Cars 3, such as Louise Nash (shaded by Yeon Kang), have an extensive wear of the clear coat layer across their paint job, this wear can create interesting patterns which break up the specular in interesting ways. Bump-Roughness helps us keep these micro details in medium and wide shots, as shown in this closeup example.

Not only is this an improvement on technology, but it is a storytelling tool to communicate wear and aging.

No detail is lost with Bump Roughness. Check out the micro scratches!

-Junyi Ling, Character Co-Supervisor Cars 3

Another great example is this work-in-progress test. The clear coat scratches are a bit more exaggerated than the film's final frames, in order to showcase the effect. In previous films, such micro detail would be lost at this distance or it would be expensive to render due to increased sampling or small shading rates necessary to find such details.

“Storytelling is always the focus at Pixar, so we sometimes needed to dial the effect down to make sure our focus was on the characters, not the technology,” says Junyi Ling, character co-supervisor on Cars 3.

Another great feature about Bump Roughness is its built-in LOD (Level of Detail), which gives artists the right amount of detail at every distance. "The automatic LOD trickles down to other departments, for example, the lighting department no longer had to spend lots of time finding paint flakes with the right lighting angle," said George Nguyen, Character Shading Lead on Cars 3. This is crucial with characters such as Jackson Storm, where a new lacquered paint job and visible paint flakes are part of his shiny "new-guy" persona. 

Even though Bump-Roughness is great for paint flakes and clear coat scratches, its usefulness doesn't stop there, it can be used to significantly improve render times by adding detail without having to increase sampling. Also, the tech benefits a wide range of materials, from carbon fiber to textiles as shown below...

© Disney/ Pixar

Many surface types benefit from the added detail

...but it's also incredibly useful in organic and heavily displaced surfaces because we can keep microfacet detail at more coarse sampling variance settings, as illustrated in this Lizard render.

-Junyi Ling, Character Co-Supervisor Cars 3

Notice all the lost displacement and bump detail when using 1 sample per-pixel. We need to use 100 samples per-pixel to bring back the original look-dev on a wide shot. Bump-Roughness, on the other hand, can use only 1 sample per-pixel while keeping the same visual appeal, as illustrated in the closeups below.  The best part is that "the added detail is gained out-of-the-box and the artist experience is seamless because it all happens automatically when RenderMan converts the textures," said Junyi Ling.

© Disney/Pixar

Shading Rate at 0.01 vs Shading Rate at 1

© Disney/Pixar

Shading Rate at 0.01 vs Shading Rate at 1 + Bump Roughness

Another big success story for the technology has been skin. "We've been using Bump-Roughness on human skin with great success," says Junyi Ling. "Specular detail on pores and other previously lost detail is more vivid than ever thanks to Bump-Roughness. It's taking realism to a new level at Pixar," he added.

In the image below we can see how Bump-Roughness is maintaining highlight information more accurately, which improves the specular breakup in pores, this is even more pronounced when zoomed out. "Refining the specular model was crucial to revealing textural detail. We were able to light shots more accurately without resorting to tricks," said Jean-Claude Kalache, lighting DP at Pixar, "before, we had to rely on manual light rigs to reveal minor detail at glancing angles," he added.

Creative Common 3.0 - Lee Perry-Smith - Infinite Realities

Skin also benefits from retaining subtle micro surface details

To see how much he could improve his renders with the new Bump Roughness tech, Jean-Claude Kalache did several lighting tests to match photography. "Since the beginnings at Pixar, we've tried to make skin look realistic, and I thought Bump-Roughness could get us a step closer," said JC. "Traditionally, at Pixar we approached our shading and lighting aesthetic with stylization as a priority, from which point we tried to make things as realistic as possible, but now, we've shifted to relying on light transport first and manipulating its stylistic possibilities," added Jean-Claude Kalache. 

The sets department was also able to leverage the developments with Bump-Roughness. "We noticed a significant improvement on environment detail, especially surfaces such as concrete and marble," said Junyi Ling. "Typically with Marble, we try to avoid a glassy look, where the marble is too shiny. With Bump-Roughness we were able to capture all the subtle imperfections that give marble its most distinctive properties," said Junyi.

The benefits don't stop there, as previously mentioned, Cars 3 saw huge improvements in render times. Bump-Roughness is over 35% faster than bump alone because RenderMan no longer has to calculate all 3 vectors, it now uses the newly supplied vector to decrease the shading context complexity. "Using Bump-Roughness was fundamental to achieving this level of detail because supersampling was never an option...render times would have been too long," said Junyi.

-Junyi Ling, Character Co-Supervisor Cars 3

Overall render times are also reduced when surfaces are shiny because the BXDF is importance sampled, this means the secondary ray is also importance sampled, and so on...this gave the Cars 3 team upwards of 30% reduction in render times in many scenes, and even though scenes with rough specular surfaces saw fewer render time reductions, there was never an increase in sampling times. "Future feature films at Pixar are using Bump Roughness with great success, from skin shading to shading most arbitrary assets, the improvements to detail preservation are the new status quo for surface quality," said George Nguyen. 

This overall improvement in sampling variance has led Pixar to new heights of image quality when defining micro detail, in which quality doesn't have to compromise render times. 

Looking ahead is always in Pixar's DNA and Christophe Hery, Head of Pixar illumination group and research scientist, is already planning to expand RenderMan's Bump capabilities even further by implementing leading work in the field. "There is more work to be done, in regards to Bump in general. The remaining big issue with Bump is that it is not reciprocal (at least as currently implemented) and we resort to alternative solutions (such as nudging the bumped normal to stay on the visible hemisphere, i.e. on the same side as the geometric normal), in order to keep a consistency with bi-directional transports. This might lead to dark non-lit areas on a piece of bumped geometry. To help avoid these issues, we are investigating several compelling ideas including Unity Labs' research on Microfacet-based Normal Mapping for Robust Monte Carlo Path Tracing," said Christophe. 

© ACM

Vincent Schüßler, Eric Heitz, Johannes Hanika, and Carsten Dachsbacher. 2017. Microfacet-basedNormalMapping for RobustMonte Carlo Path Tracing. ACM Trans. Graph. 36, 6, Article 205 (November 2017), 12 pages.

Photorealism is a never-ending journey, "it's so fun to get carried away by the possibilities of visual complexity" said Junyi, "but at the end of the day, technology has to help the visuals in a subtle and useful way, not the other way around, the focus should always be the characters and their journey." This is the balance between technology and storytelling we strive for at Pixar.