Asset Management

Asset Management

Asset management is the term used to describe both how files are organized on disk as well as how one file on disk refers to other files. In the context of RenderMan Studio, a Slim Session may refer to multiple external Slim palettes comprised of appearances that refer to Slim templates and external texture and .slo files. A Maya file may refer to Slim-produced .slo and texture files, and so on. In multi-person productions this simple reality can become the source of considerable headaches or it can be the key to ensuring a smoothly operating production pipeline.

It's difficult to come up with a scheme for asset management that works for all the people, all the time. Small productions have different requirements (different budgets) than large productions. Large projects are divided into smaller chunks that can be produced by individuals and may often have a middle supervisory contributor not needed in smaller productions. In all but the smallest productions, artists need to work both independently and collaboratively. Individuals must be free to iterate on their assignments, which may refer to assets produced by others. The work of any individual mustn't affect the rest of the team until the asset has been approved. And when an asset is approved, it must automatically be available to team-members who reference that asset. Next, artists who refer to shared production assets should be able to provide local overrides for any subset of these assets. For example, in a particular shot, a different shader for Woody's belt-buckle may be called for.

These problems have been solved in many ways by many studios. We hope that the mechanisms built into RMS applications and described herein are sufficiently simple, flexible, and noninvasive that they will "play well" with solutions already practiced within your facility. The complete asset referencing machinery for RMS components is exposed and configurable. All asset references are expressed via asset reference templates: strings that contain TCL variable references. These templates are routed through a small number of exposed TCL procedures whose job is to interpolate them (substitute variables, evaluate sub-expressions) and produce final asset references.

We'll start by introducing some terminology used across the RenderMan Studio family of applications, then we'll tie these concepts together to describe a means by which the requirements set forth above can be met.

RMSProductionModel

To accomodate a wide range of production models, we start by introducing RMSProductionModel. This setting is specified via RMS.ini and characterizes the basic workflow employed at your site. Currently we support two models:

  • ProjEqualsProd: your site uses a single Maya project directory for a single production. Multiple artists are simultaneously operating below that directory. This is the traditional production model embodied in all prior versions of RMS and RAT tools.
  • ProjWithinProd: your site uses more than one Maya project directory for a single production. Typically a small number of artists operate in one project at one time.

As productions scale, the first model can be said to have important limitations. Larger facilities embrace a wider variety of tools and this makes untenable the reliance on Maya's workspace as the central arbiter of policy. For these sites a larger notion of production is often required. Generally it's a good idea to make a large number of policy decisions prior to undertaking any but the smallest of productions and choosing the RMSProductionModel that best suits your requirements is the first step. If you are happy with a ProjEqualsProd model, it's possible that you need read no further. This is the default mode RMS software ships in and asset management concerns may not be a high priority. Even so, some of the concepts discussed below may be helpful in planning your production pipeline.

RMSPROD - The Root of a Production

The idea that a production needs a single root directory in a file system hierarchy is the foundation of most production pipelines. Specifying the location of that root directory to a number of different interoperating tools is required. For this discussion, we'll use RMSPROD to represent that root directory. Changes to files outside of this area are considered "beyond our control", thus all production-critical assets should either reside within RMSPROD or be locked down outside of RMSPROD. Cross-RMSPROD asset sharing can be managed through explicit/manual copy/synchronize mechanisms to ensure that one production's priorities cannot disrupt another's.

When running in ProjEqualsProd mode, the root is completely specified when you select a current project. But when running in ProjWithinProd mode we must follow a different tack. First, we consult the environment variable RMSPROD for a value. If the environment variable isn't present, we search up the directory tree from your current project for a file named RMSProduction.ini. The mere existence of this file causes RMS to interpret its directory as the production root. RMSProduction.ini can also be used to govern the policies for all RMS tools within the production context, and can contain any of the settings usually found in RMS.ini, invoking SetPref, et cetera. The fact that it may run when the notion of RMSPROJ changes (e.g. during a Slim or RfM session) means that it has some unique properties when used to control, for example, prefs. These possibilities make RMSProduction.ini the preferred method for specifying the root of a production. Note that it is an error to run in ProjWithinProd mode and not specify a production root; RMS tools encountering this condition will automatically revert back to the ProjEqualsProd mode of operation.

Whichever production model you choose, the production root plays the critical role of ensuring that each production asset has a unique identifier: its relative pathname below the root. Because all newly created assets will be produced and referenced in relation to this location, it is critical that it be carefully chosen and then locked down.

RMSPROD_GLOBAL

Larger productions usually require some mode for editing assets offline. While many tools may be employed to achieve this effect (SVN, Git, Perforce, rsync, scp, cp, ...) the gist of the idea is that we have an area for work-in-progress and another area for final versions. When operating in ProjWithinProd mode, RMS supports the optional RMSPROD_GLOBAL setting. As we'll see below, this can used to represent a fully-populated public area to search after looking for under-development versions of the same assets. In large workgroup settings we recommend that RMSPROD_GLOBAL be set to a globally visible, read-only, production-specific root directory and RMSPROD be set to a user's or team's (writable) development area. As with RMSPROD, you can set RMSPROD_GLOBAL via the environment. For those sites that prefer to establish the RMSPROD by the presence of RMSProduction.ini, the RMSPROD_GLOBAL value is established by defining its value therein via:

SetPref RMSPROD_GLOBAL /net/path/to/my/production_global

Note that in this regime we always terminate our search upon encountering the first enclosing RMSProduction.ini. While it's common to place the online, work-in-progress area below RMSPROD_GLOBAL, it's certainly not the rule, nor is it required to work in this online-offline modality. Since it's dangerous to cross network mount-points in such a traversal and to support the notion of multiple RMSPROD_GLOBAL areas we have not implemented a file-nugget-based methodology for defining RMSPROD_GLOBAL.

RMSPROJ - The Workspace

RMSPROJ represents a location within RMSPROD from which work is performed. Put another way: all new assets are produced below the current RMSPROJ directory. In ProjEqualsProd mode, the workspace subdirectory structure is employed to prevent asset collisions while supporting cross-scene asset referencing. In ProjWithinProd mode, modelers working on a single model might work in the model's project whilst folks composing shots work in the project associated with the shot.

To obtain a value for RMSPROJ we employ a few schemes:

  1. when running tethered to, e.g. Maya, follow the lead of the host app
  2. inspect the commandline option -workspace
  3. inspect the environment for RMSPROJ
  4. look for a user preference indicating the last-used RMSPROJ
  5. fall back to a preference: DefaultWorkspace

A similar but simpler sequence is used to determine the value of the Slim Session in order to support working on Slim palettes and shaders with or without an active Maya/RfM session.

STAGE - A Workspace Namespace

STAGE represents a namespace within RMSPROJ that can be used to prevent asset-name collisions. A value for STAGE can be derived from any source, but usually it is tied to the name of the currently opened file. RMS implements a variety of asset stage strategies to ensure that site-specific file name versioning practices have the correct namespacing effect. For example, Maya only prevents name collisions within the scope of a single Maya session. To ensure that newly constructed shaders, textures, and RIBs don't collide with those from other Maya scenes, RfM creates assets with pathnames containing $STAGE. Here's a snippet from RMSWorkspace.ini:

SetPref WSSubdir.rfmImages      {renderman/$STAGE/images}
SetPref WSSubdir.rfmData        {renderman/$STAGE/data}
SetPref WSSubdir.rfmShaders     {renderman/$STAGE/shaders}
SetPref WSSubdir.rfmTextures    {renderman/textures}
SetPref WSSubdir.rfmRIBs        {renderman/$STAGE/rib}
SetPref WSSubdir.slimPalettes   {slim/palettes}
SetPref WSSubdir.slimSessions   {slim/sessions}
SetPref WSSubdir.slimShaders    {slim/shaders/$STAGE}
SetPref WSSubdir.slimTextures   {renderman/textures}
SetPref WSSubdir.slimTmps       {slim/tmp}

Out of box, the value of STAGE is equal to the Maya file (sans extension, aka FILEBASE) in which the reference is contained. Slim shaders associated with Woody.ma will be deposited here:

.../slim/shaders/Woody/

This mapping between Maya filename and stage is embodied in a simple TCL procedure, StageStrategyCB_file, located in RMSExpresion.tcl. Additional procedures are provided to demonstrate alternate strategies, including:

  • StageStrategyCB_fileProjRel: produces STAGE by combining an RMSPROJ-relative pathname with FILEBASE.
  • StageStrategyCB_fileStripVersion: produces STAGE by running a regsub expression on FILEBASE.
  • StageStrategyCB_env: produces STAGE from an environment lookup
  • StageStrategyCB_pref: produces STAGE from a user pref
  • StageStrategyCB_proj: produces STAGE by looking only at the RMSPROJ

RMSPROJ_SHARING

Another notion, which may be useful to large-scale productions, is RMSPROJ_SHARING. This concept is only meaningful in ProjWithinProd mode and represents an optional third location in a production filesystem hierachy. The value of this concept lies in its ability to express global-asset overrides at a granularity that is greater than a single project (details below). To obtain a value for RMSPROJ_SHARING we simply search up the filesystem hierarchy, starting at RMSPROJ. As with RMSProduction.ini, RMSSharing.ini both defines the value of RMSPROJ_SHARING and affords the opportunity to express RMS preferences at this level of sharing. This is not always needed in smaller productions. However, in movie production the relationship between shot and sequence maps well onto RMSPROJ_SHARING. Here, each shot within a sequence needs to share assets produced by the sequence supervisor as well as the asset-override decisions required by the sequence.

Putting It All Together

The first requirement for asset sharing is that things just work. When you reference Woody's model file and the gas-station set into your shot file it is expected that the textures and shaders for both can be located and happily cohabitate in the same context. Any scheme that involves fully- qualified filenames to assets can be used to achieve this end. The ProdEqualsProj mode affords a couple of benefits beyond this through abstract asset references (involving the use of $STAGE, etc.).

The real trick in larger-scale productions is to accomodate asset-overrides. This is where ProdWithinProj may be worth considering. Here, production-relative pathnames are the key. In service of our offline-vs-online and asset-override requirements, we employ pathnames that have been relativized to RMSPROD (and equivalently to RMSPROD_GLOBAL). The directory-structure of your production already guarantees uniqueness of asset references, so we leverage this fact to produce the asset references in various RMS tools: Slim palettes, RIB files, etc. In order to facilitate cross-workspace overrides it is extremely helpful to employ a canonical directory structure; RMS tools can only impose a canonical structure on the assets it creates. We accomplish this via per-tool preferences that can be specified in RMSProduction.ini and that reference our well-known scripting variables and workspace settings. For example, the way that Slim produces .slo files is controlled by its DefaultMasterRef preference, typically set to:

$slimShaders/$INSTANCENAME

When Slim writes a .slo file it produces a fully-qualified path name to ensure that the file is deposited in the correct location, without reliance on a current-working directory. But, when the .slo file is referred to in Slim palette files, project-relative paths are implied by the DefaultMasterRef. All RMS applications employ the workspace object to both Globalize as well as to Localize asset references. This object presents itself to RMS scriptors as the workspace command. The three critical methods of the workspace object are LocalizePath, GlobalizePath and ResolvePath. When a Slim shader is added to an RfM session we perform the same relativization scheme, in the same manner. The resulting reference to the Slim shader is a production-relative pathname to the shader as viewed by RfM's current workspace settings.

Now, finally, here's the payoff for this simple collection of operations: this relative-path naming scheme can be coupled with an ordered collection of search locations to achieve our key objectives.

Consider this setting for the RenderMan searchpaths:

$RMSPROJ:$RMPROJ_SHARING:$RMSPROD:$RMSPROD_GLOBAL

The RMSPROD_GLOBAL and RMSPROD settings allow us to locate any asset within the production with precedence given to the offline value. The RMSPROJ searchpath allows us to override any production-wide assets on a per-workspace basis by replicating the production-wide asset structure below the RMSPROJ or RMSPROJ_SHARING directories.

In this example layout, we see that Woody's beltbuckle shader - chars/Woody/slim/shaders/beltbuckle - is overridden for the sequence ts_s00.

- ToyStory3/
  |- RMSProduction.ini
  |- chars/
  |   |- Buzz/
  |   |   |- maya/
  |   |   |- menv/
  |   |   |- renderman/
  |   |   |   |- data/
  |   |   |   `- rib/
  |   |   |- shake/
  |   |   |- slim/
  |   |   |   |- palettes/
  |   |   |   |- sessions/
  |   |   |   `- shaders/
  |   |   `- textures/
  |   `- Woody/
  |       |- maya/
  |       |- menv/
  |       |- renderman/
  |       |   |- data/
  |       |   `- rib/
  |       |- shake/
  |       |- slim/
  |       |   |- palettes/
  |       |   |- sessions/
  |       |   `- shaders/
  |       |       `- beltbuckle.slo
  |       `- textures/
  |- props/
  |   `- gas-station/
  |       |- island/
  |       `- pump/
  |- sequences/
  |   |- seq_00/
  |   |   |- RMSSharing.ini
  |   |   |- chars/
  |   |   |   `- Woody/
  |   |   |       |- slim/
  |   |   |       |   `- shaders/
  |   |   |       |       `- beltbuckle.slo
  |   |   |       `- textures/
  |   |   |- shot_00_00/
  |   |   |   |- maya/
  |   |   |   |- menv/
  |   |   |   |- renderman/
  |   |   |   `- slim/
  |   |   `- shot_00_01/
  |   |       |- maya/
  |   |       |- menv/
  |   |       |- renderman/
  |   |       `- slim/
  |   |- seq_01/
  |   |   |- RMSSharing.ini
  |   |   |- shot_01_00
  |   |   `....
  |   `....
  `- sets/
      |- AndysRoom/
      |- PizzaPlanet/
      `....

Summary

  • RMSPROD, RMSPROD_GLOBAL: a container of workspaces. In small-scale operation, may be equivalent to workspace. The goal of a RenderMan production is to produce pixels. Other assets may be of use in, e.g., game or toy production. To support online-vs-offline operation for staging asset references, we support two production locations.
  • RMSPROJ ('PROJ', 'workspace'): the place where one or more units of work are performed. The results of the work are deposited below/within this area.
  • Workspace File: a file located in RMSPROJ or RMSPROJ_SHARING that defines searchpath settings. You may find it valuable to have multiple workspace files in each workspace. Or you may prefer to disallow user edits of workspace settings. The default workspace file name is RMSworkspace.ws.
  • STAGE: a namespace that can be used to prevent asset collisions.
  • Asset: an input or output file of the production. Input files may reside anywhere on disk. Output files are deposited below the workspace. References to input files may be either relative or absolute. Relative pathnames enable local overrides. Absolute pathnames may be more robust in certain circumstances, more fragile in others. Relative pathnames are subject to searchpaths and these can be changed to give precedence of one asset over another.
  • Work: embodied via the creation of asset files (RIB, sl, slo, tex, bkm, ptc, image, ...) on disk.