Resources Documentation RenderMan :Developers' Guide :The Tools :RfM Dev Guide :Application Notes :

Dynamic Shader Binding

Dynamic Shader Binding

Introduction

Dynamic Binding is a new way to assign shaders to geometry in RMS that simplifies binding for complex assets and provides a powerful but easy-to-use, programmable method for the assignment of shaders and modification of shading parameters in a pipeline-friendly, "just-in-time" way.

Dynamic binding is controlled by the user creating a rule that, when fired, attaches a shader, inserts an arbitrary RIBbox, or edits one or more parameters to Ri requests. The rule's input is primarily the path name of the object. Additionally, any object set membership that has been set up in Maya can be queried, as well as pass-specific information. In more complex situations, multiple rules may impact the final choice of shader and/or the parameter values to the shader.

The new binding mode introduces a new asset file called RenderMan Look File (extension .rlf) that is created automatically in RfM (and/or, potentially, elsewhere) and is used at render time in conjunction with the new RLFInjector RIF to assign shaders to geometry in a "just-in-time" fashion. The Look Files contain a database of which shaders belong where and RIB payloads that will be injected into the RIB stream to assign shaders or modify their parameters. The payloads are injected into the RIB stream at appropriate points marked out by RfM. This means that RIB files on disk no longer contain actual Surface calls, just markers - called nuggets - that are processed by the RLFInjector RIF.

An overview of basic usage is provided in the Pipeline section of the Users Guide. What follows here is a deeper discussion of the implementation for users who wish to understand more about the inner workings.

Note that the existing mode of static binding continues to work in much the same way, but, under the hood, is implemented slightly differently, as mentioned below and summarized in the Appendices. Most importantly, however, dynamic binding can override static binding.

Rules

RfM uses rules to map XPath expressions to payload IDs. There are two kinds of rules:

  • Ri Injection Rule - acts first, deposits one or more Ri Requests into the Ri stream
  • Ri Request Edit Rule - acts last, edits the parameters of an individual Ri request

Execution Model

A renderable scene is comprised of a tree of objects, some of which arrive through lightweight references. You can think of a scene as a tree of nodes, any subtree of which represents a scope that might have an associated rule set. Any individual node can be uniquely identified by its path. We can identify a plurality of nodes with XPath expressions. As leaf nodes (geometry) are delivered to the renderer, we evaluate our rule sets to find payloads to deliver to the renderer. We can think of this as "decorating" our scene; the fact that this happens "just-in-time" empowers production pipelines.

Phase 1: Ri Injections

To decorate a scene, we visit each leaf node in "depth-first" tree order. At each injection point (identified by special nuggets in the Ri stream) we evaluate (and possibly execute) the active list of rules to find one or more payloads to inject. The active list of rules comes about by joining rule sets found along the object path from root to leaf. This ordering, coupled with execution orderings, means that root-level rule sets take precedence over leaf-level rule sets (i.e. shot wins over set or char).

When an XPath expression finds a match, the rule is said to succeed. Upon finding a match, the associated payload is obtained and stored on a stack. At this point execution will either terminate or continue as governed by an attribute associated with each rule. When the rule set is exhausted or a matching rule terminates the rule search, the process moves on to the injection phase, where the stack of payloads is delivered to the renderer. Those payloads encountered last will appear first in the Ri stream; this allows higher-precedence overrides to compose with lower-level defaults, following standard Ri behavior.

If the list of rules is exhausted, either none succeeded or the last matching rule specified "continue", then the current Attribute Identifier Path relative to the current assembly scope is used to determine the injectable payload. This final attempt to find an injectable payload only occurs at the innermost assembly level. This is how static binding works in the absence of any rules.

Phase 2: Ri Request Edits

Whether or not injection payloads are delivered, another phase of rule set execution follows. Here, rules of a different type, Ri Request Edits, are evaluated. The same execution model is followed, but the leaves seen by the XPath expression engine are the actual RenderMan Interface (Ri) requests. Since XPath expressions can match either the leaf-node RiRequest or an individual parameter (attribute) of that node, payloads must either deliver a collection of values or an individual value. Again, according the the rule, we either continue or break execution as a payload is delivered. Payloads are applied in stack order and therefore a class of parameter-value matching expressions will only match the original value, not the modified value. This execution model is motivated by the requirement that top-level rule sets be able to override leaf-level rule sets.

Finally if no edit rules fired, or the last firing rule specified "continue", then the current attribute identifier path is looked up in the static edits table. This is analogous to the static binding injections described above.

XPath

In addition to the object path discussed above, XPath expressions have two additional kinds of information available. Both are are pass-related values that are provided as "attributes" (in the XML sense) of the document that the XPath expression excutes on,. The first appears as attributes on the fixed-named root node /renderpass; the second appears as attributes on the leaf node itself. Currently the second type is limited to any set membership the object has in the scene file. An example use of this attribute information would be to have a rule that overrides all shader assignment in the "Shadow" pass, regardless of its name.

Usage Guidelines

The paths available for XPath matching have two or three components:

/renderpass/nurbsSphere1/Sphere1

or:

/renderpass/nurbsSphere1/Sphere1/RiOption
  • Always start with "/renderpass". This element provides access to the render pass-specific options that are established with RiOption.
  • The Attribute Identifier path for the current XPath match follows.
  • Optionally for Edit Rules, the last element is the Ri request name. Requests that can be filtered are:
    • RiAreaLightSource
    • RiBxdf
    • RiHider
    • RiLightSource
    • RiOption
    • RiPattern
    • RiShadingRate
    • RiShader
    • RiSurface

Additional Information Available to XPath

/renderpass has XML attributes set corresponding to the pass-specific RiOptions:

  • pass_class
  • pass_id
  • pass_phase
  • pass_flavor
  • pass_crew
  • pass_camera_name
  • pass_camera_flavor

For example:

/renderpass[@class = "Final"]//Sphere

or:

/renderpass[@camera_name = "Left_Eye"]//Sphere

Ri request elements have their parameters set as XML attributes using either their name or their position if unnamed. For example, for the RIB specification RiShadingRate 1.0, the following positional parameter query can be used:

//RiShadingRate[@1 = '1.0']

The previous example selected the RiShadingRate call whereas the following example selects the parameter itself using its exact name "specAmount":

//MyLightName/RiAreaLight/@specAmount

Payloads

Payloads are chunks of data that can be inserted into a RIB stream. A payload may represent an actual call to Surface, selected by injection type rules, in which case the data will be one or more lines of RIB, or a payload may be sets of one or more parameter values as selected by edit rules. Parameter override payloads can be stored as a dictionary of parameter name keys and their values, or just a single value. In either case the payload is keyed uniquely by an ID that a rule refers to; this ID may simply be the "Default ID" delivered in the injection nugget, as in the case of static binding.

Edit payloads of type string or string array also undergo a round of simple substitution at the moment the edit occurs. The following set of strings are considered:

  • ${OBJNAME}
  • ${OBJPATH}
  • ${NAMESPACE}

The values correspond to the current attribute identifier name, the accumulated stack of attribute identifier name (ie /renderpass/nurbsSphere/nurbsShape) and the namespace of the current look file (most often the scene name) respectively.

Payload Resolution

When an injection rule fires the result is the name of a payload to be injected. To deal with scene referencing, a namespace name resolution sequence is then applied to the payload name to find it. In a scene file referencing case, the actual name of the payload from the point of view of the main or outermost scene will not be the same as it was when a rule from a referenced scene was authored; the name of the payload will now be prepended by a namespace from the point of view of the main scene file. For each injectable payload a namespace-qualified search is made from the outermost Look File inward. In most cases the first (main scene's) Look File will contain the payload, but the search will still continue down towards the leaf Look File, at each step the name being prepended with one less namespace element.

The RLFInjector Rif

The new binding system requires, whether static or dynamic binding is being employed, a Rif to convert the nuggets deposited by RfM into the final shader calls. The Rif ships as part of RfM and is invoked via a ##rifcontrol call in RfM's RIB files. It is used whether internal or external rendering is selected. Put another way, to render RIB files generated from RfM you need to have the RLFInjector Rif.

RIB "Nuggets"

Nuggets are RIB structural comments inserted into the RIB stream by RfM to defer the moment of binding shaders to geometry to a just-in-time fashion. The RLFInjector rif uses theses nuggets to determine shader binding and possibly edits to Ri Requests.

The following Nuggets used by RfM and RLFInjector:

##RLF ScopeBegin [-rlffilename filename] [-namespace <ns>]  [-stripnamespace 0/1] ...

ScopeBegin indicates the starting point of a new RLF scope in the RIB stream.

-rlffilename <filename>
path name of a new RLF to be pushed on the stack of scopes. If this option is missing the RLFInjector Rif will take the current RIB filename, replace .rib with .rlf, and attempt to open that.
-namespace <ns>
Sets the namespace of the current scope. Note the same RLF can be used in multiple scopes in different namespaces.
-stripnamespace [0|1]
Directs the RLFInjector Rif to either consider the namespaced attribute identifiers or discard them. Default is 1 == strip.

EXAMPLE:

##RLF ScopeBegin -name perspShape_Final -localbinding 1

##RLF ScopeEnd

ScopeEnd ends the current scope and pops the RLF off of the stack.

EXAMPLE:

##RLF ScopeEnd

##RLF Inject SurfaceShading [-defaultPayloadId <payloadid>]
                           [ [-attribute <name>@<value>] ... ]

Inject directs the RLFInjector to execute the insert payload phase. The current attribute identifier path and the current stack of RLF scopes are used to determine if a payload (or payloads) are to be deposited.

-defaultPayloadId <payloadid>
In the absence of finding a dynamic rule use this id to obtain a payload. If no payload is found for the id then the regular attribute path lookup is attempted.
-attribute <name>@<value>
When dynamic rules execute they can access these named values as XML attributes on the leaf element node of path. RfM uses the "sets" name to indicate membership of any shading groups

EXAMPLE:

##RLF Inject SurfaceShading -attribute sets@,initialShadingGroup,

Performance Considerations

Users should be aware of the various trade offs that are made between the execution of dynamic rules and static edits. In general one should keep in mind that each dynamic rule in play is potentially executed for every object in the scene. This can be mitigated in a several ways. Ordering rules that will match most often at the highest point they can be in the rule set could improve performance drastically if there are large numbers of rules. Specifically it would be an obvious speed advantage to have early rules that fire first and have the break flow control semantics.

Static edits also could also dramatically out perform dynamic edit rules. Consider the case where every shader in a scene having thousands of shaders needs a different texture map assigned to a parameter. Static edits of each of these shaders will result in a hash table lookup for each shader. Whereas a matching rule for each shader will require a linear search and execution of an average of half the rule list.

In summary care must be taken in selecting how binding and edits are to happen for the best performance. The RLFInjector plugin collects various timing and count information delivered by the normal RenderMan XML statistics output to aid in this process.

Appendices

Static Binding Details

Shaders that are attached to objects in the scene in the usual manner are also handled by the RLFInjector using a static binding table:

  • If no rules fire then the current Attribute Identifier Path relative to the current Assembly scope is used to determine the injectable payload. This final attempt to find an injectable payload only occurs at the innermost Assembly level.
  • As with dynamic binding, static binding uses a RIF to convert the nuggets deposited by RfM into the final shader calls. The RIF ships with RfM and is invoked via a ##rifcontrol call in the RfM's RIB files. RIB files generated by RfM cannot be rendered without the RLFInjector RIF.

Working With Imported RIB

If you are working with RIB that is not generated by RMS then it is likely that RIB won’t contain nuggets at all. If this is the case, writing dynamic binding rules will not work at all because the only time they are evaluated is when the RLFInjector encounters a nugget, as explained above. This limitation can be removed by putting the RLFInjector into a special mode that executes the rule-matching phase at all geometric primitive sites.

To set up this mode, the Rif must be given the -filtergeom 1 argument as it is being loaded. In RMS this is most easily added in your own .ini file (e.g. RenderMan_for_Maya.ini). The DefaultRifs preference can be changed to include the argument:

SetPref DefaultRifs {
    "-rif RLFInjector -rifargs -filtergeom 1 -rifend"
}

  • Important

    We highly recommend that, rather than editing an .ini file in your installation directory ($RMSTREE/etc), users should create a duplicate file containing your site-specific overrides and place it in a separate directory, which must then be referenced via the $RMS_SCRIPT_PATHS environment variable.

Once that is in place rules can be authored in the usual manner, either in the Dynamic Rules editor panel or by creating RLFs via a Python script.

The XPath expressions will still be run against the paths constructed by the Attribute "identifier" calls, so the imported RIB must contain those in order for this technique to be of use.

It should be noted that rule evaluation can impact performance, so careful consideration of the number of rules and geometric primitives is of supreme importance.

XPath Expressions

For more information on XPath Expressions:

TM & © 1986 - Pixar. All Rights Reserved.