RSL Update Guide

RSL Update Guide

Introduction

This document serves as an overview and a timeline for changes to the RenderMan Shading Language. The focus here is generally on core language features and new functions. Changes to existing functions are generally documented in the release notes, unless they require a detailed description.


PRMan 18

"Plausible" Updates

New Shadeops

  • There is a new shadeop - textureatlas() - which can be used to look up textures comprised of multiple texture maps as produced by paint applications such as Mari, Mudbox, and Zbrush.
  • There is a new diagnostic shadeop - rbug() - which outputs all points, vectors, or normals to the ptviewer utility during a render.

Shadeop Updates

  • rayinfo supports two new queries - "integrationcounter" and "integrationlod" - which offer shaders access to the path tracer and re-renderer's current state.
  • The gather() shadeop now allows fetching "ray:" values along with "lighting:irradianceoutside" in the same call.
  • The rayinfo() shadeop supports a new "shadingintent" query, returning "cache", "shadegrid", or "shaderay".
  • indirectspecular() now supports the "usemotionbias" parameter. Additionally, it now accepts a "position" parameter to specify where rays should be shot from; the default is P.

Additional Changes

  • txmake now creates an SHA1 hash for each output texture. This SHA1 hash is then used by the renderer to only open unique instances of an given texture. Even if the filenames of the textures are different, if they have the same SHA1 hash, the renderer will reuse the texture cache entries from the first unique version of a given texture file.
  • The shader compiler now generates more informative error messages when there are problems with the declaration of a function in the DSO.
  • There is a new, special __computesRefinement parameter, which can be used to tell the renderer whether a shader's refinement method is going to modify the shading rate.

PRMan 17

Adaptive Area Shadows

The areashadow() shadeop now adaptively samples the shadow function, using fewer samples in regions of low variance, for faster area shadows, both ray-traced and not.

RSL Arrays

PRMan's implementation of arrays in the RenderMan Shading Language (RSL) has been overhauled, providing significant speedups and efficiencies. Users can expect to see performance increase by 20%, on average. (Note that these performance improvements will require that shaders be recompiled, which will make them incompatible with earlier releases). RslPlugins that use array iterators will also need to be recompiled to avoid a performance penalty; before recompiling, the plugins will have to be modified to eliminate dereferencing array data directly. See the RSL Plugin API documentation for more information.

Plausible Hair

PRMan now provides improved support for hair shading, with a new plausible hair shader and new sampling tools provided via the stdrsl Library.

stdrsl Additions

The stdrsl Library has been updated with several new header files:

  • Colors.h - provides common color manipulation functions
  • Hair.h - provides physically plausible components for hair
  • HenyeyGreenstein.h - an anisotropic scattering volumetric specular component parameterized by a color and a scattering parameter "g"
  • IsotropicVolume.h - an isotropic scattering volumetric diffuse component parameterized by a diffuse color
  • Logger.h - a simple struct that can be used to log error messages
  • OrenNayar.h - a microfacet diffuse component parameterized by a diffuse color and roughness
  • SampleMgr.h - a component to manage sample reduction up various ray trees

PRMan 16

Physically Plausible Shading

The shading language was extended with new integrator functions, sampling functions, and methods, designed to support "physically plausible" shading. These extensions include:

  • directlighting() - combines the diffuse and specular integrators in a single function.
  • indirectspecular() - provides a means by which the specular response associated with indirect light transport paths can match the response to direct lighting.
  • diffuselighting and specularlighting are optional pipeline methods that provide finer control than the existing lighting method and enable significant renderer optimizations.
  • generateSamplesAS(), evaluateSamplesAS(), generateSamplesEnv(), and evaluateSamplesEnv() are sample-generating functions designed to interoperate with the directlighting() and indirectspecular() integrator functions.

Additionally, the PRMan distribution includes a stdrsl Library and stdrsl Shaders to help integrate these new shading techniques in users' pipelines. These files can be found in $RMANTREE/lib/rsl/.

For more information, please consult the Physically Plausible Shading in RSL application note.

areashadow() and the areashadow Method

The areashadow() function supports ray-traced shadows and traceable deep shadow maps. Traceable deep shadow maps can be produced via the areashadow method. Further details can be found in the documentation for areashadow() and the Area Shadowing application note.

getpoints()

The getpoints() function reads point data from a point cloud, frame cache, or grid cache, and returns the number of points found. More information can be found in the Baking 3D Textures and the getpoints() documentation.


PRMan 15

Filter Regions

Filter regions are a new type supported by the shading language for accessing the texture system. A filterregion can be created from texture coordinates in a couple different ways, scaled, queried to determine size, and passed to the texture(), shadow(), environment(), ptexture(), and gather() calls. A filterregion acts much like a struct. For more information on usage, please consult the Filter Regions application note.

ptexture()

The ptexture() shadeop has been introduced as part of PRMan's support for Per-Face Textures (PTex). The Per-Face Textures application note provides details of shaders and utility programs that can be used bake shaded results into Ptex textures.

Additional technical information about Ptex textures and filtering can be found at http://www.disneyanimation.com/library/ptex

gridmin(), gridmax()

Two new functions that operate over a grid of floats and return a uniform result have been added. The usage is as follows:

uniform float maxP = gridmax(varying float myVal);
uniform float minP = gridmin(varying float myVal);

These functions will find the max/min of all the values of myVal over the surface of the grid currently being shaded. The result can be varying, however the same value will be returned at all points in the varying result value.

Struct Inheritance and Member Functions

RSL structs may now declare member functions that operate on their members, facilitating code factoring that was previously more difficult to achieve. Structs may also be composed via inheritance, which gives access to the base class' members and member functions. For more information, please see the Structs in RSL application note.


PRMan 14

Structs

Support for "struct" types was added to the shading language in PRMan 14. Structs can contain uniform and varying data of any type, including arrays, nested structs, etc. Struct members are operated upon by reference, so adopting structs has little performance impact.

For more information, including the the use of structs in plugins and various limitations of the current implementation, see the Structs in RSL application note.

Shader Diagnostics

PRMan includes source file and line number information with most shader error messages, as well as (optionally) a complete call stack. This is helpful when diagnosing errors in library routines that are called in many different locations. This feature incurs a slight performance penalty, so it is controlled by a "shading debug level" option.

Option "shading" "debug" [2]   # Enable extra diagnostics

Call stacks are enabled at debug level two (or higher).

For more information, please consult the Shader Profiling application note.

Resizable Arrays in Plugins

Resizable arrays are fully supported in shader plugins. (Prior to PRMan 14 it was possible to pass resizable arrays to plugin functions, but they could not be resized.) For example, this code pushes a value onto a uniform resizable array:

RslResizer* resizer = argv[1]->GetResizer();
unsigned int n = resizer->GetLength();
resizer->Resize(n+1);

RslFloatArrayIter array(argv[1]);
RslFloatIter val(argv[2]);
array[n] = *val;

See the RSL Plugin API Reference for more information.


PRMan 13.5

Nested returns

Return statements may appear anywhere in a function, rather than being restricted to the end of the function. For example:

float positive(float x) {
    if (x > 0)
        return 1;
    else
        return 0;
}

Uniform values returned from varying conditionals are promoted to varying. For example, the function above returns a varying value if x is varying.

Short-circuiting Logical Operators

The logical operators && and || correctly "short circuit", whereas previously they would evaluate both their arguments. This is crucial for correctness, since logical operators might be used to "guard" potentially unsafe operations:

if (i < arraylength(nums) && nums[i] != 0) {
    // This used to test nums[i] with an out-of-range index.
}

Shader Parameter Scope and Externs

A shader parameter can be used anywhere inside the body of the shader definition without requiring an extern declaration. (This change was motivated by the addition of member variables, which have similarly broad scope.) To avoid confusion, we recommend that a naming convention be used to distinguish between shader parameters and local variables.

Extern declarations are now required only when a nested function refers to variables defined in an enclosing function (e.g. local variables or function parameters). Extern declarations may still be used, even if they are unnecessary (e.g. for documentation purposes).

Resizable Arrays

A resizable array can be obtained by declaring a local array variable with an unspecified or non-constant length. Initialization is optional. Only arrays declared with an unspecified or non-constant length are resizable.

For complete details, please see the Arrays section of the RSL reference documentation.

Class Definitions, Member Variables, and Methods

A shader can now be written as a class definition with member variables and methods. The class definition specifies shader parameters, and the methods accept additional parameters.

The values of member variables are preserved between method calls, e.g. allowing state to be shared between displacement and surface shading. Uniform member variables can also be preserved across multiple executions of the same shader, allowing expensive calculations to be performed less frequently.

Shaders can call methods and read member variables of other shaders.

For more information, see the Shader Objects and Co-Shaders application note.

Co-shaders

Scene descriptions can include co-shaders, which allow custom shading pipelines to be expressed in the shading language itself. For example, layers of a surface appearance can be expressed as separate shader instances, which are combined by a shader that calls the displacement, opacity, and surface methods of each layer.

Co-shaders are simply shader objects with no explicitly specified purpose; they are not executed directly by the renderer, but rather are called from other shaders.

For more information, see the Shader Objects and Co-Shaders application note.

Illuminance Caching Control

In release 12.5 a "lightcache" argument was added to the illuminance function to allow light caching to be disabled (previously this was accomplished by setting "P = P"):

illuminance(P, ..., "lightcache", "refresh") {
    ...
}

It's now possible to specify illuminance caching behavior on a per-light basis, rather than this kind of all-or-nothing basis. A light shader can define a string parameter called __lightcache. If the value is "refresh", the illuminance function will ignore any previously cached results and re-execute the light; otherwise the usual caching rules apply. This is useful for lights that employ stateful plugins.

Note that the __lightcache parameter is ignored if it equals "reuse"; that does not force cache reuse. Also note that the value of the __lightcache parameter can be specified in an RiLightSource call (or even as a vertex variable).


PRMan 13

Output Argument Initialization Warning

It is risky to rely on the input value of a shader output parameter. The situation was improved slightly in release 13.0: varying output parameters always have correct input values now, but uniform output parameters sometimes do not. (Fixing this behavior is difficult because the cost of guaranteeing validity is surprisingly high.)

The shader compiler now reports a warning when a shader relies on the input value of a uniform output parameter. The most reliable way to work around this issue is to have the shader explicitly copy the value from an input argument to an output argument.

Alternatively, in 13.5 a member variable can be used instead. Member variable initialization is well defined and easy to control (e.g. using a constant initializer or an assignment in a construct or begin method). For more information, see the Shader Objects and Co-Shaders application note.

New Shader Plugin API

Starting in PRMan 13.0, shader plugins (DSOs) must be thread safe (unless the renderer is run in single-threaded mode). This is facilitated by a new API that provides per-thread storage management, thread-safe global storage, and other useful features.

The new API also provides higher performance by allowing a plugin function to operate on an entire batch of data, rather than being called once for each point shaded. For example, here is a plugin function that computes "a = b + c" for floating-point values:

int add(RslContext* rslContext, int argc, const RslArg** argv)
{
    RslFloatIter a(argv[0]);        // an iterator is like a pointer
    RslFloatIter b(argv[1]);
    RslFloatIter c(argv[2]);

    int n = argv[0]->NumValues();   // number of values in this batch
    for (int i = 0; i < n; ++i) {
        *a = *b + *c;
        ++a; ++b; ++c;              // advance iterators
    }
    return 0;
}

The API is described in detail here:

Additional changes include the following:

  • Shader plugin function arguments and results can now be arrays of unspecified size. Shader plugin functions can also accept an arbitrary number of arguments, e.g. point findmax(point, ...).
  • An RSL plugin function can now receive a uniform variable as a varying output parameter (unlike in RSL functions). This eliminates the need to create overloaded variants of a function to accept both uniform and varying outputs.

Shader Profiling

Use the following option to enable shader profiling:

Option "statistics" "shaderprofile" ["profile.xml"]

Then load the XML file into your Web brower to view the results.

Shader profiling identifies hotspots in shaders, taking the guesswork out of optimization. Shader hotspots include call stacks, allowing a detailed understanding of the context in which heavily used library routines are called. Each hotspot is linked to its source code, simplifying browsing and analysis.

See the Shader Profiling application note for more information.

Shader Meta Data

Comments in shader source code can now specify arbitrary meta data that is recorded in compiled shaders. The meta data can be accessed via the PRMan SDK, and scripts can use sloinfo to fetch meta data as XML.

The syntax is as follows:

/* This is a comment.
   <meta id="name">data</meta>
*/

The data is arbitrary; it can span multiple lines in block comments (but not in "//" comments). The identifier is a key that can be used to fetch the meta data. Meta data identifiers can have the same names as identifiers in shader code; they reside in a separate namespace.

For more information, see the Shader Meta Data developer note.

wnoise() - Wavelet Noise

The wavelet noise function has the following prototype:

float wnoise( point pt, float filterWidth, [parameterlist] )

It produces one or more bands of noise constructed with wavelets, and is based on the 2005 SIGGRAPH paper by Cook and DeRose. Wavelet noise has a similar appearance to Perlin noise but can be band-limited more effectively for the sake of antialiasing. Unlike the noise function, which returns values between 0 and 1, wnoise, by default, returns values between -1 and 1. The distribution and range of these values can be changed using the parameters described in the RenderMan Shading Language reference.

transform() Generalization

The RSL transform() function has been generalized. It now has the following prototypes, where T = point, vector, normal, or matrix:

T transform([string fromspace,] string tospace, T src);
T transform([string fromspace,] matrix m, T src);

The fromspace is optional and defaults to "current". Calling transform on a point works as before. When called with a vector or normal it is equivalent to a vtransform or ntransform call. When called with a matrix, it returns the from/to transformation matrix times the given matrix.

Miscellaneous Improvements

For more details about additional RSL enhancements in PRMan 13, please consult the RPS 13.0 release notes.