Boasting worlds which surpass 1 trillion polygons, Toy Story 4 is Pixar’s most complex movie to date. Thanks to ongoing rendering innovations and advances in data management, technical artists at Pixar were able to realize this creative and complex new vision. Pixar’s Universal Scene Description is at the forefront of these workflows and has transformed the film production pipeline into what it is today.

What is a pipeline you ask? Well...in simple terms, it’s a set of tools which make complex processes predictable and reliable. The diagram below illustrates an overview of a traditional pipeline, where departments engage in linear collaboration to get to a final frame, meaning a department has to wait for the previous one to finish their work in order to make progress, for example, animation can’t start unless the character models exist.

Pixar has a long history of making tools for inter-departmental artist collaboration, and USD is the evolution of decades of experience. Thanks to these new asset pipeline tools, teamwork between departments has continued to transform the way we work at Pixar.

...but why? Well…”time is money” and relying on a predictable toolset which promotes collaboration between departments in a non-destructive way, is fundamental to achieving feature film production deadlines.

...that sounds awesome, but doesn’t my favorite application already solve my problems? Sure, if you stay within that application, but at Pixar innovation is incredibly important to us, so we make sure each department can make and choose the best tools for their needs. With this in mind, USD serves as a clean and robust exchange format to describe the work from all departments without getting in their way, this might include models, animations, materials, lights, cameras, and other typical necessities in a production environment.

Using such an exchange format makes collaboration very predictable and in turn allows our artists to focus on creative endeavours instead of data management. This decoupling also means we can project ongoing innovation with USD without impacting other departments.

Let’s break this collaboration down by departments...

Staging and Layout

At Pixar, USD workflows are revolutionizing our world-building process. Because of the format’s efficiency, we are able to create and access more complex worlds while visualizing them in ways which would have been impossibly slow with older technology. USD’s Hydra enables artists to interact with their world much faster than before, while adding OpenGL approximations of textures, materials, and lights to the process.

This allows us to realize the aesthetic of the film earlier in the pipeline, which has had a positive impact for collaboration, thus facilitating approval processes which were traditionally left for later in the pipeline. Less stress, yay! ...but doesn’t this mean I’m getting in the way of other departments? Well...that’s the beauty of USD, it’s non destructive! Its layering system treats data very much like Photoshop does, with a set of individually stacked operations which can be turned on or off without affecting any of your data. Do you want to see how master lighting is going while building your scene layout? Great, simply turn on the layer for better context!

© Disney / Pixar

USD Layers

Assets and Shading

Thanks to the “variant” system in USD, RenderMan is able to call any arbitrary number of versions for a given object, including differences in shape, shading, texturing, and more. For Toy Story 4, Pixar created multiple variants for each asset, so that artists could have full control over a shot. This also meant they could do more with less by taking advantage of USD’s great use of instances, which means that any slight change in shading or texturing doesn’t have to affect the geometry loaded in memory, efficiently managing this limited resource.

USD Variants

For example, in the Antique Warehouse there are thousands of assets created from much lesser amounts thanks to USD variants, creating a more rich and efficient world than would have otherwise been possible.

Say you’re working on a shot and the art director wants to change a dominant color in your framing, well, instead of having to modify the entire sequence which affects other artists, you can simply choose a differently colored bookcase or set of chairs in order to create the right mood for your shot. Simply go through the available variants in that particular asset and voila! new assets!

Since USD can include material definitions, we can collaborate between Pixar’s proprietary shading tools and commercial texturing applications such as Foundry’s Mari, which helps departments leverage their strengths without having to worry about data management. Because our internal shading system has slight differences with commercially available tools, RenderMan has gone a step further and now ships with a Mari approximation for Pixar Surface Material, giving artists greater shading context during their texturing process.

To help manage the logistics, Pixar has developed internal tools to deal with all the conversion and tracking processes for USD and more, giving artists the ability to manage their assets easily. Building tools such as these is non-trivial, but more predictable and accessible to external pipelines thanks to a single and consistent USD API and Scenegraph, as well as a rich set of tools to read, write, and edit scene data. 

Speaking of scene data...there are over a thousand assets from all Pixar movies in the Antique Warehouse, such as the Pixar Ball, or the bike from Red’s Dream, can you spot them all?

Animation

Our animation software Presto was designed around the concepts that would eventually be refined and extracted to become USD. The first feature film to use it was Brave, giving USD a decade of code innovation and stabilization. These efforts are available to everyone as part of Pixar’s open source initiative, which include OpenSubdiv, OpenTimelineIO, OpenUSD, among many more research projects readily available to the public.

One of the strongest features in USD is the ability to load complex data sets, and stream them with a low memory footprint, which means more interactivity for the animator and greater context of the surrounding scene, for example, a character artist can manipulate and preview hair in real-time, giving them a more accurate way to judge their character’s performance. This greatly increases the amount of work they can iterate through, producing more polished shots and sequences while providing greater support for the character’s performance.

These advancements are possible thanks to Hydra, USD’s back-end and front-end imaging framework and OpenSubdiv, Pixar’s open source mesh rendering project, both working in tandem to represent complex scene data with maximum speed and fidelity. The crowds department also greatly benefits from these pipeline advancements, as you can easily visualize large data sets and work with simpler versions of said assets, taking advantage of USD’s fast caching system for real-time playback, and when ready, switch back to production quality assets by changing “schemas,” a USD-centric approach for facilitating wide-affecting scene changes.

Lighting

A crucial part of USD is its ability to dynamically switch between render delegates in the viewport, in this example, between OpenGL and RenderMan, which means a better lighting approximation during earlier stages of production. This allows the Cinematographer and Director to give the most accurate feedback possible, so that the artist can worry about polishing and being creative with their shot, instead of concerning themselves with fundamental color and staging differences with the layout department.

Rendering

A final rendered frame is generally associated with the very last step in the pipeline before it goes off to color timing and editing, but thanks to sophistications in USD and Hydra, RenderMan can now render pixels far in advance, helping many departments have better visual context for their work.

Remember our Pipeline chart at the beginning? Well, this is how many departments USD and RenderMan interact with...pretty neat!

Having more fidelity during production means artists can preview intricate effects generally designated for final frame rendering, such as depth of field, motion blur, and shadows, all in real-time. Approximating in-render effects is fundamental, as Pixar is keen on leaving the rendered pixel as intact as possible to avoid unnecessary pixel filtering and ensuing image degradation during post-processes. This is important for departments such as Stereo, where pixel fidelity is crucial.

Available with the USD distribution are tools such as USDView, which can load any kind of USD file, be it ascii (usda), binary (usd), or archive (usdz). You can simply switch the render delegate to RenderMan and see the shot or asset with lighting context. All these collaborative workflows improve the final frame and give everyone better context for their work, so that feedback sessions (called dailies) can be more accurate and productive.

Here’s that trillion polygon shot we spoke about, with detail down to a single pine cone needle, courtesy of USD!

How do I get my Hands on USD?

Houdini

RenderMan has taken a leadership role in supporting Pixar’s USD format by shipping hdPrman, an imaging plugin for USD’s Hydra, while delivering support for Sidefx’s Houdini 18 and their innovative Solaris toolset for look development, which is completely built around USD. 

SideFX demos Solaris at the RenderMan Art & Science Fair

Maya

Thanks to USD’s open source initiative, Pixar has been impacting the VFX and feature animation world for years. Studios such as Luma Pictures and Animal Logic have not only embraced it, but have given crucial feedback and development resources which benefits all users. These plugin development efforts will soon be rolled out for Autodesk Maya, making the USD ecosystem more robust than ever.

You can learn more about Autodesk's plans for USD on their dedicated USD website.

Katana

Foundry's flagship lighting tool is committed to support and create compelling workflows which match the extensibility of the USD toolset in upcoming versions of Katana. 

Foundry outlines their plans for USD in Katana

Roadmap

Pixar's USD will continue to evolve as developers from all over the world take the code into new and creative territory. RenderMan is committed to supporting and evolving with this innovative format as it continues to impact the computer graphics industry.