Since the beginning of computer graphics, achieving photorealism to match film plates has been a huge focus, which has led the CG industry to continuously research and develop groundbreaking tech to represent the subtleties of rendering humans. This has allowed artists to represent people of all ethnicities on the big screen, but such developments don’t always reach aspiring artists, especially those with limited access to the technical infrastructure needed to achieve such advancements.

The world of Soul is inhabited by characters with a wide variety of racial and cultural backgrounds and Pixar is making strides to ensure its production pipeline is designed with representation in mind.

Skin Shading

Rendering skin seems like a straightforward effect to accomplish nowadays, especially given the multitude of online references, such as the Humanae Project by Angelica Dass, which aims to document the almost endless variation of skin colors, but simulating skin in a computer is not as straightforward as assigning a color to character, many technical challenges remain, including the understanding of complex algorithms. This level of technicality is important in order to bring some level of ground truth to skin representation instead of relying simply on our subjective artistic eye, which many times sees through the lens of our unconscious biases.

To accurately depict the range of skin tones in Soul, shading artists had to dial-in specific controls for the way the light scatters inside the skin. In CG this is called Subsurface scattering, or SSS for short. “We did some work to tune the shading to ground-truth before the work on assets started. Mainly it was to remove a lot of varying tinting from the absorption color, as well as tweaking the distance the light is absorbed into the skin. This allowed us to rely more on albedo color for characters, therefore being more consistent between ethnic groups,“ said Junyi Ling, character supervisor on Soul.

Soul © Disney/Pixar

RenderMan has many sophisticated controls for SSS, including an inverse albedo approximation, which means the artist simply gives the renderer a color or texture and RenderMan takes care of all the hard work to make sure the skin glows with life and is representative of the original art direction and concept art. “We switched to a more physically based non-exponential path traced subsurface model for Soul, which is more principled on physics, but also provides a lot of flexibility when needed. Subsurface values were modified based on artistic preferences once we got into production, relying on feedback and reviews to make sure we were staying true to the artistic concept,” said Junyi.

To ensure skin tone was represented correctly, the creative teams at Pixar consulted with renowned live-action cinematographer Bradford Young. “We consulted with Bradford early in production. Taking into account his experience in live action cinematography led us to specific and subtle tweaks to skin materials for African-American characters. The results of these consultations, along with our usual research, better informed us when creating the shading and lighting setups,” said Junyi Ling, character supervisor on Soul.

Soul © Disney/Pixar

From left to right, Steve Pilcher, Bradford Young, Pete Docter, and Ian Megibben

As shading and lighting have become more and more sophisticated, artists at Pixar have found many live-action makeup tricks relatable when creating characters. “Lights on set are very bright and hot, so it’s very common to have to matte caucasian skin with powder to avoid overly bright highlights, but for darker skin it’s quite the opposite, we had to add more specularity to shape characters,” said Alex Marino, Lead Shading and Grooming Technical Director on Soul. “The specular accuracy was 80% of the work,” he added.

Some technical challenges had to be ironed out to make sure the skin was as accurate as possible, so the team started with a baseline UV layout for most characters, which allowed them to try different techniques for skin variation to make sure all characters were unique. “Darker skin doesn’t have a different absorption color in real life, as the variation in color comes from the melanin content, but renderers don’t work this way, we have to modulate the SSS attributes manually to make sure they represent these subtleties, for example by tinting the absorption color,” said Jonathan Hoffman. “Differences in albedo color was the more predominant factor in representing color differences, but adding a layer of subjective artistic touch was crucial to reaching the intended look,” said Alex Marino.

Skin Lighting

Lighting is one of the main storytelling tools in a movie, giving artists the ability to shape the dynamics of a sequence through form and color. Not surprisingly, lighting affects surfaces of different colors in different ways, for example a red light affects a blue or green ball differently. That’s physics!

Light can have dramatic effects on different surfaces.

We can begin to see how something so simple poses a concern for properly representing skin, since changes in lighting can affect the skin color of a character. This can result in unintended consequences for the film, such as changing the tone of the storytelling, or creating an unwanted aesthetic for a character.

Bringing emotion to Soul was a key part of the storytelling and collaborating with other departments was crucial. “We really needed to sell what it is like to appreciate the small things in life and we were lucky to partner with the look development department on the common goal of creating a tactile world through the shading and lighting,” said Ian Megibben, Director of Photography for Soul. “I also spent some time working closely with the layout department pre-lighting shots which gave me the opportunity to wrap my head around a sequence early. While the experience of lighting in a GL preview is technically limited, it served as a great jumping off point for the lighting keys that the talented painters in the art department would then go on to make,” he added. “This collaboration and the process of working rough-to-fine allowed lighters the freedom and flexibility to organically discover the beauty of each shot.”

In order to focus on the story and the goals of the character while representing their skin properly, lighting artists have to make sure their lights are representative of real-world physics. For this purpose, RenderMan has analytical lights with controls for color and temperature, as well as more artistic controls with light filters, this ensures that artists are working with a reasonable baseline for predicting lighting behavior. “We referenced a lot of black and white photography of African-American Jazz legends. Roy De Carava’s photography was a great inspiration,” said Ian Megibben. “We aimed for a natural and believable light quality, so we relied on the physicality of the lights, going as far as drawing inspiration from live action techniques. We also took a more physical approach to light blockers, to avoid discrepancies in falloff and to maintain the relationship between the diffuse and specular contributions,” he added. “To make sure that the skin reacted predictably, we provided the look development department a variety of different lighting setups to use while shading, some of which emulated scenes from live-action film, such as Selma,” said Ian.

Soul © Disney/Pixar

Exploratory Jazz Photography Aesthetic

Cinematographer Bradford Young [Selma, Arrival, Solo: A Star Wars Story] also made an impact on the lighting team. “We’re all big fans, and were super excited to bring Bradford in. The experience was even more impactful than expected,” added Ian Megibben. “We discussed techniques for lighting a variety of skin tones, but we also talked about digging backstories for each and every character. Applied to our work we discovered that lighter skin tends to soak in the light, so we rely on a diffuse response to shape our characters, but the light’s specular response plays a much more important role with darker skin,” said Ian. This helped balance not only the skin shading, but brought a new level of realism to the lighting in the scene.

Some traditional conventions were also tweaked with Bradford’s help. “We tended to overlight or overexpose darker skin with the traditional lighting method of shaping with diffuse. It was as much as ¾ stop too bright in many cases,” said Ian. “Using the color of the light was also a wonderful and creative aspect of the concert scenes. The night club venue lent itself naturally to exciting lighting scenarios,” said Ian Megibben. Thanks to departmental collaboration and live action cinematography consulting, the Soul cinematography team was able to take the lighting of Soul to a whole new level of classic jazz authenticity.

Hair Grooming

Rendering hair is also a very important part of representing a character. There’s a huge amount of hair diversity within a multitude of ethnicities, which means the artists at Pixar had to do a lot of research to make sure hair was represented authentically. To achieve this realism, RenderMan allows a great deal of customization and sophistication when it comes to rendering hair, including a new material shipping in RenderMan version 24 called Lama Chiang Hair, based on the studio's PxrChiang material. “This new hair material is way more flexible for the artists, especially with varying colors within a single hair groom,” said Ian Megibben. “The relationship between the three specular lobes was one of the most important aspects of the shading and lighting and the volume in the hair groom was also super important, especially making sure we could shape the silhouette accurately with backscattering,” added Ian.

Soul © Disney/Pixar

Dorothea’s hair shading and grooming

Silhouetting characters has become one of the cornerstones for separating them from the background and was fundamental to the grooming needs for Soul. “Backlighting was a baseline for aesthetic qualities for hair and garments. Dorothea’s hair was particularly challenging, especially the density of her hair, so we had to make sure we got the groom right or else the backlighting wasn’t working,” added Jonathan Hoffman, Character Shading and Groom Artist on Soul.

Evoking a character’s look is all about attention to detail. “Bringing out textural qualities in the art direction was important for Joe’s hair, especially up close detail,” said Jonathan. “Getting that 5 O’clock shadow on his stubble was also a super important detail to get right,” said Jonathan.

Soul © Disney/Pixar

Attention to detail brings characters to life

The art of caricature is the art of exaggeration and naturally walks a fine line of representing or misrepresenting a particular group of people, so it was very important to the artists at Pixar to take all the steps necessary to make sure all the creative choices were made in the most meaningful and educated way. “It was important not to over accentuate racial caricatures and to take into account historical context for cultural and racial sensitivity,” said Alex Marino. “We make stylized movies, so we always make sure we don’t accentuate offensive stereotypes. It’s important to understand racial and cultural nuance and to avoid reductive thinking when representing different races,” added Jonathan. “To avoid stereotypes it was important to compensate for the lack of primary knowledge by reaching out, asking questions, and doing heavy research, so we reached out to production leadership which could relate directly to these ideas. We are also very lucky to work with co-workers from a range of backgrounds who always help us better understand racial context for better representation. This combination helped us reduce unconscious bias,” added Alex and Jonathan.