Written by Leif Pedersen. Edited by Dylan Sisson.

Stylization ... Stylization Everywhere

Stylization is an essential aspect of Pixar's aesthetic, and the technology departments at Pixar play a crucial role to empower the director's creative vision and the stylization of that world. Since its creation in 1988, Pixar's flagship renderer, RenderMan, has evolved dramatically to deliver images of ever increasing sophistication and complexity, but RenderMan has always remained focused on delivering uncompromised creative freedom by providing new tools for stylization and storytelling.

Stylizing images goes far beyond manipulating final pixels. It is a crucial element of the entire creative process, from abstracting shapes to generating line work.

Modeling and Camera

When building a world, the modeling artists at Pixar collaborate with illustrators and sculptors to bring the worlds to life … sounds pretty straightforward right? Well, it turns out that interpreting the expressiveness of 2D characters and world designs can be challenging. Creating a memorable character silhouette and conveying ideas through set design are just two examples of how stylization plays an essential role in storytelling.

To facilitate this collaborative process, Pixar has developed many technologies, some of which are open source. Two critical ones are Pixar’s Universal Scene Description, or USD for short, which allows us to describe all of our 3D work in a simple way across departments. The other is OpenSubdiv, which is a very efficient way to represent smooth surfaces, a critical part of representing cartoony looks. These technologies have not only revolutionized the way we work at Pixar, they power some of the biggest movies, series, and products in the world.

Rendering these stylized assets accurately, and with maximum control is crucial, which is why RenderMan was designed as and continues to be a micropolygon dicer. This means RenderMan breaks up the model into tiny grids every pixel, which allows the rendering of perfectly round surfaces in a very efficient way, automatically adapting as the camera moves.

Materials and Surfacing

Creating materials is an expressive part of our moviemaking process, where artists can control the appearance of a character or prop and define colors, patterns, and material properties, all in service of the story.

Most times these material properties are plausible representations of the real world, and other times these properties need to represent a surreal or abstract concept ... not easy! To support this flexibility, RenderMan created the RenderMan Shading Language, or RSL for short. Today, RenderMan uses Open Shading Language, more commonly referred to as OSL, which continues many of RSL’s innovative ideas about making creative shader writing possible through an approachable programming syntax.

For example, this is a modern RenderMan compliant OSL pattern which represents marble:

{
      shader marble( float Kd=.8, float Ka=.2, output color resultRGB = 0 )
      {
      float diffuseF, sum ;
      point Pshad;
      float i, freq;
      color Cs = color(1, 1, 1);

      diffuseF = dot(I,N) ;
      diffuseF = (diffuseF * diffuseF) / (dot(I,I) * dot(N,N)) ;

      sum = 0;
      freq = 1.0;
      Pshad = transform ("object", P);
      for (i = 0; i < 6; i = i + 1) {
      sum = sum + 1/freq * abs(.5 - noise( 4 * freq * Pshad)) ;
      freq = 2 * freq;
      }

      getattribute("Cs",Cs);

      resultRGB = Cs * sum * (Ka + Kd * diffuseF ) ;
      }

      }

… and this is an old school RSL Marble shader … pretty similar right!? All that friendly syntax makes it much easier for technical directors to get creative.

surface marble( float Kd=.8, Ka=.2 )
{
    float diffuse, sum ;
    point Pshad;
    float i, freq;

    diffuse = I.N ;
    diffuse = (diffuse * diffuse) / (I.I * N.N) ;

    sum = 0;
    freq = 1.0;
    Pshad = transform ("shader", P);
    for (i = 0; i < 6; i = i + 1) {
	sum = sum + 1/freq * abs(.5 - noise( 4 * freq * Pshad)) ;
	freq = 2 * freq;
    }
    Ci = Cs * sum * (Ka + Kd * diffuse ) ;
}

Another core element of RenderMan is its ability to break physical rules with its materials. Artists don’t have to be bound entirely by the rules of physics when crafting looks, as they can express their material properties by mixing and matching several different material layers, or lobes. This allows for impossibly gummy fish in Finding Dory, expressive particle effects surrounding emotions in Inside Out, or ethereal beings in Soul.

Lighting

Shaping characters and environments is a critical part of guiding the eye through the frame. This keeps audiences engaged in the story and gives the director creative control over what the viewer should be focusing on. Lighting plays a pivotal role in creating this illusion and many times artists have to bend the rules of physics in order to achieve creative goals.

A great example of bending the rules are the stylized shadows in Luca, where lighting artists needed to tint and art direct the silhouette of the shadows to match the concept art.

To achieve this look, the RenderMan team created a whole new way to manipulate the shadows using a new light filter called Shadow Fringe. This light filter allowed artists to manipulate the sharpness, penumbra, and color of the shadows, by isolating its halo.

This allowed artists to dramatically extend the look of the shadows by breaking up their halo with textures and fringes of color, bringing to life a very stylized look reminiscent of magic hour shadows in summer.

Rim lights are another typical stylizing scenario, where characters need to pop from the background in order to read them properly in the frame, but the creative uses for lighting can be pushed far beyond their intensity or direction.

A good example of this, are traditional 2D animation needs, where artists need the gradient between light and shadow, called the shadow terminator, to have a sudden falloff, as exemplified in the Pixar short, Sanjay’s Super Team. Tricks with RenderMan materials and lights were combined to achieve this look.

Simulation and Effects

Sometimes stylizations can be quite the challenge, especially when heavy art direction clashes with physics. A great example is Pixar’s latest film Elemental, which answers the question: “how can we give our volume rendering team nightmares?”

Elemental called for art directable volumes during the character animation process, which was a dramatic change in the traditional animation pipeline. In order to avoid redefining the animation process, the technical wizards at Pixar created neural networks to train the volume data to behave like input images, such as noises or drawings. This drove the simulations and allowed for art direction with tools that were familiar to artists.

The results speak for themselves! Notice how Ember’s fire is cartoony, giving the character’s silhouette an expression that matches her performance instead of relying on the realistic yet unpredictable physical properties of a fire simulation.

Another great example of stylization are the water effects for Luca, where ocean foam required stylized and art directable silhouettes, to match the illustrative look of the director, Enrico Casarosa. Naturally, this is not how a simulation software would interpret the physics of waves, so a lot of interesting work needed to happen to manipulate what the 3D application thought a wave’s natural shape ought to be.

In order to achieve this effect, the simulation artists developed a system to create stylized wave shapes by advecting the foam particles radially outwards from repellant centers.

The resulting points from their stylized simulations were further stylized with polygon meshing and smoothing operations. Finally, bubbles from the original foam simulations were reprojected to the stylized results as instanced spheres and voila! Stylized water foam.

Here is a final frame showing stylized foam around a boat.

Rendering

This is RenderMan’s bread and butter, of course! Creating great pixels is our thing, and delivering innovative looks in new films continues to be a driving force for RenderMan’s new tech.

To represent the needs for final pixel stylization in production, RenderMan has developed Stylized Looks which has been implemented in RenderMan as a native and interactive rendering solution for non-photorealistic rendering. Stylized Looks was recently used to stylize Cars on the Road with art directable line work.

An interesting approach to stylization with this new toolset is the ability to daisy chain different effects, such as line work, toon shading, or hatching. Artists can also guide the effects based on a large number of data inputs, such as lights, LPEs, AOVs, and more, which can really help convey traditional illustration styles with temporal stability for animation.

...of course we stylized the Walking Teapot!

Compositing

Sometimes you just need to get creative in post. That’s what the creative teams at Pixar did with the Spark short film Out, where building a fully stylized pipeline wasn’t in the plans, so creating the look was best accomplished by manipulating RenderMan render passes in Nuke to stylize the rendered frames, which we call “beauty” renders.

These render passes, technically called AOVs and LPEs in RenderMan, allow artists to accentuate a particular light, or isolate a single material lobe for distorting it with painterly effects. Although there are limitations, the ability to transform the frame in post in real time is a great way to experiment with looks…and a lot more forgiving for render times!

Another great example are the hand painted effects in Turning Red, which were used extensively to create a dynamic 2D look without having to go through a simulation process. Some of these hand drawn elements defined the beautiful anime aesthetic of the film. 

Beyond the Studio

An incredibly important aspect of RenderMan is its programmability, and its ability to meet the creative ambitions of other studios. Thanks to robust APIs, RenderMan allows studios to expand its capabilities to fit their creative needs.

This is exemplified by Disney’s Chip N’ Dale: Rescue Rangers, which was created at MPC, and used a custom set of cartoony shaders to mix live action, photoreal, and stylized rendering in a single frame.

Netflix’s Entergalactic also greatly exemplifies stylization and showcases RenderMan’s production versatility. By extending RenderMan’s capabilities to achieve a hand drawn look, the technical teams at DNEG went to great lengths to merge their 2D and 3D pipelines with new projection techniques, including a new way to achieve depth of field without traditional camera effects, and instead using a changing level of detail approach based on camera depth.

Thanks to new creative movies and shows pushing the boundaries of stylization, the road to new forms of rendering will be full of unexpected turns in order to express exciting stories, such as ILM’s upcoming work on Ultraman, a new stylized feature film from Netflix.

Stylization in film is ever present and always changing. The art of stylization is not just about creating pictures that look cool, but rather, stylization draws upon principles of perception and psychology to help support a creative narrative. Through developing new technologies and techniques for artists, we broaden the types of stories that can be told, and it’s exciting to see where the challenges for stylization will take us next.