2.1   pxGridMax.cpp

#include <stdlib.h>
#include "RslPlugin.h"

extern "C" {

RSLEXPORT int pxMaxGridFloat(RslContext* rslContext,
                             int argc, const RslArg* argv[])
{
    RslFloatIter arg   (argv[1]);
    RslFloatIter result(argv[0]);

    *result = *arg;                     // Use first value as initial max.
    int numVals = RslArg::NumValues(argc, argv);
    for (int i = 0; i < numVals; ++i) {
        if (*arg > *result)
            *result = *arg;
         ++arg; ++result;
    }
    return 0;
}

RSLEXPORT int pxMaxGridTriple(RslContext* rslContext,
                              int argc, const RslArg* argv[])
{
    RslPointIter arg   (argv[1]);    // Points and colors are both just float[3]
    RslPointIter result(argv[0]);

    // Use first value as initial max.
    (*result)[0] = (*arg)[0]; (*result)[1] = (*arg)[1]; (*result)[2] = (*arg)[2];
    int numVals = RslArg::NumValues(argc, argv);
    for (int i = 0; i < numVals; ++i) {
        if ((*arg)[0] > (*result)[0])
            (*result)[0] = (*arg)[0];
        if ((*arg)[1] > (*result)[1])
            (*result)[1] = (*arg)[1];
        if ((*arg)[2] > (*result)[2])
            (*result)[2] = (*arg)[2];
        ++arg; ++result;
    }
    return 0;
}

static RslFunction myFunctions[] =
{
    { "uniform float pxGridMax (float)", pxMaxGridFloat,  NULL, NULL, NULL, NULL },
    { "uniform point pxGridMax (point)", pxMaxGridTriple, NULL, NULL, NULL, NULL },
    { "uniform color pxGridMax (color)", pxMaxGridTriple, NULL, NULL, NULL, NULL },
    { NULL }
};
RSLEXPORT RslFunctionTable RslPublicFunctions = myFunctions;

}; /* extern "C" */

2.2   gridmax.sl

// The plugin directive tells the compiler which files
// to search for plugin functions.
plugin "pxGridMax";
surface gridmax ()
{
    uniform float ymax = 0;
    uniform point pmax = (0,0,0);

    ymax = pxGridMax(ycomp(N));
    pmax = pxGridMax(P);

    printf ("y = %f, pxGridMax(y) = %f\n", ycomp(N), ymax);
    printf ("P = %p, pxGridMax(P) = %p\n", P, pmax);
}

3.1   The C++ File

Plugins must include the provided header, RslPlugin.h, which contains class definitions and macros that are used for writing your own plugin functions. You must compile your plugins with a C++ compiler, but you must use C linkage. To accomplish this, use extern "C" around the tables and functions to guarantee C-style linkage for the plugin loading system.

3.2   The RslPublicFunctions Table

The C++ file contains a table describing the functions contained in the plugin - essentially mapping the RSL functions (with arguments) to the C++ implementations. The functions may be polymorphic; in other words, one may have separate functions of the same name that are distinguished by the types of the arguments passed to them or by the type of the value returned by the function.

To give a more concrete example, here is an example table that describes implementations of a squaring function for several types of arguments:

static RslFunction myFunctions[] =
{
    { "float  sqr (float)",   sqr_float,  NULL, NULL, NULL, NULL },
    { "point  sqr (point)",   sqr_triple, NULL, NULL, NULL, NULL },
    { "vector sqr (vector)",  sqr_triple, NULL, NULL, NULL, NULL },
    { "normal sqr (normal)",  sqr_triple, NULL, NULL, NULL, NULL },
    { "color  sqr (color)",   sqr_triple, NULL, NULL, NULL, NULL },
    { "float  sqr (...)",     sqr_vararg, NULL, NULL, NULL, NULL },
    { NULL }
};
RSLEXPORT RslFunctionTable RslPublicFunctions = myFunctions;

Note the following properties of the table declaration:

1. A static array of RslFunction structs must be used to construct the table, which must be called RslPublicFunctions.

2. The end of the table is denoted by a NULL entry.

3. The table contains one entry for each polymorphic version of a plugin function. In this example, there are versions that square floats, points, vectors, normals, and colors.

4. The file can contain any number of functions, as long as they are all defined in the RslFunctionTable.

5. The first string in each RslFunction struct is the declaration of the function as it will be called from a shader.

   • Detail (uniform or varying) can be specified in the function declaration (it is varying by default). Argument detail can be queried at run time by the plugin function (using IsUniform) if the plugin author wishes to avoid the overhead of iterating over uniform values passed as varying parameters.
   • The output keyword should be specified in the function declaration if the programmer is going to change the value of an argument. It is up to the programmer to ensure that an input-only argument does not have data assigned to it.
   • The function declaration may contain as the last argument, '...', which denotes varargs. A variable number of arguments can then be passed to the plugin function from the shader.

6. The second argument is the name of the C++ function that will execute the plugin function.

7. The third and fourth arguments are the names of the C++ functions that will run when the plugin function first executes after being loaded and at the end of a rendering pass.

8. The fifth and sixth arguments are the names of the C++ functions that will run immediately before and after each render. During a regular prman render, these occur during RiWorldEnd(). During re-rendering these occur during RiEditEnd() and sometines during RiEditBegin. See the Managing user-generated data section of the Re-rendering application note for details.

3.3   Plugin Function Definitions

Plugin functions are declared as follows:

RSLEXPORT int myfunction(RslContext *rslContext,
                         int argc, const RslArg *argv[]) {
   ...
}

The function takes three arguments: rslContext is a pointer to the thread-specific state in which the plugin function is running, argc is the number of arguments to the function (including one for the return value), and argv is an array of argument pointers (each of which is a const RslArg*). The argument array starts with the return value, if any (argv[0] should not be used if the RSL return type is void). The plugin function returns an integer return value: 0 indicates no error and a nonzero return value indicates an error. The RSLEXPORT macro is an OS-specific declaration used to ensure the function is visible to the program loading the plugin. Be sure to enclose all the plugin functions in an extern "C" { ... } block, as shown in the example above.

The RslArgs have a number of useful methods used to determine attributes of a given argument. IsFloat(), IsString(), IsPoint(), etc. can be used to determine if an argument is a given type. IsArray() can be used to determine if an argument is an array or not. If the argument is an array, GetArrayLength() can be used to determine the number of elements of an array argument. See the RSL Plugin API Reference for a description of other advanced methods.

3.4   Iterators

In PRMan a plugin function will operate over a collection of points at the same time. However, not all of the points in the collection will be active at any given time (due to the nature of loops or conditional statements in the shader). In order to perform operations on only those points that are active the plugin interface provides iterators. The data for each active point of a given argument is accessed through an RslIter object of a given type. The example from Section 2 demonstrates how iterators are used:

// Get iterators for result and argument.  The result is always uniform.
RslFloatIter result(argv[2]);
RslFloatIter    arg(argv[1]);

// Copy the first value (the initial maximum) from arg to result.
*result = *arg;

// Query for the number of iteration values from the argument.
// It will be one if the argument is uniform.
int numVals = argv[0]->NumValues();

// Copy the maximum argument value to the result.
for (int i = 0; i < numVals; ++i) {
    if (*arg > *result)
        *result = *arg;

     // Increment the argument iterator.  Result is uniform,
     // so it does not require incrementing (but doing so wouldn't hurt).
     ++arg;
}

An iterator is constructed from a given argument:

RslFloatIter myfloatArg(argv[2]);

The iterator can then be treated like a pointer to the given type, so one can access the data for a given argument at a given point by simply dereferencing it:

*myfloatArg

returns the value of the first active float. The iterator is advanced by simply incrementing it:

++myfloatArg``

The pre-increment operator should be preferred, since it is more efficient than the post-increment operator. Assignment to arguments is equally simple:

*myfloatArg = 42.0

Note that any argument (not just argv[0]) can be used as an output argument. It is up to the programmer to ensure that plugins only modify assignable values that have been specified with the output modifier on the arguments.

In this new interface, it is crucial to know how many iterations must be performed by the plugin function. The IsVarying() method of the RslIter class can be used to determine if an argument is varying or uniform. For varying arguments, the number of iterations will be equivalent to the number of active points on the grid; for uniform arguments, only one iteration should be performed. The number of iterations to perform can be obtained from the NumValues() method of the RslArg. Functions that return a value in the result argument should use argv[0]->NumValues() as the number of iterations, since the shader compiler guarantees that the detail of the result argument matches the detail of all the other arguments. Functions that return void or return any results in output arguments must examine all their arguments to determine the correct mix of varying and uniform argument iterations. There is a static convenience function in RslArg to simplify this: int numVals = RslArg::NumValues(argc, argv). It should be passed the argc and argv that were passed into the plugin function. As an example, a plugin function that adds its arguments would iterate as follows:

int numVals = argv[0]->NumValues();
for (int i = 0; i < numVals; ++i) {
    *result = *a + *b;
    ++result; ++a; ++b;
}

If the result is varying, the loop will run once for each active point. If the result is uniform, the loop will run just once. If the result argument is varying and either or both arguments are uniform, the loop will still run over all the active points for the result. If the result argument is uniform and either argument is varying, the shader compiler will flag the detail mismatch as an error. An example that uses output arguments, pxGridMax.cpp, is discussed above.

3.5   Strings and Arrays

String and array arguments can present interesting issues when iterating and assigning. We will start with an example that appends '.tx' to a string argument and returns the result.

RSLEXPORT int appendTxLeak(RslContext* rslContext,
                           int argc, const RslArg* argv[])
{
    RslStringIter outString(argv[0]);
    RslStringIter inString (argv[1]);
    int inLen = strlen(*inString);
    // WARNING -- THIS WILL LEAK MEMORY!
    char *tempString = new char[inLen+4];

    strcpy(tempString, *inString);   // This code assumes the strings have
                                     // uniform detail
    tempString[inLen] = '.';
    tempString[inLen+1] = 't';
    tempString[inLen+2] = 'x';
    tempString[inLen+3] = '\0';
    *outString = tempString;

    return 0;
}

static RslFunction myFunctions[] = {
    { "string appendTxLeak(string)", appendTxLeak, NULL, NULL},
    { NULL }
};
RSLEXPORT RslFunctionTable RslPublicFunctions = myFunctions;

Dereferencing a string iterator gives a string pointer (char*). Strings are assigned by pointer assignment (e.g. *outString = tempString). However, the contents of an RSL string must never be modified; doing so could result in a crash. Also, the renderer does not free strings that are allocated in plugins, so the above example would cause memory to remain in use for the result until the end of render. One should never free memory associated with a plugin argument; doing so will result in a crash. Storage management is discussed below.

Arrays are accessed through a separate class of iterators, RslArrayIter. Array iterators for all of the types in RSL are provided. Here is an example that searches an array of vectors for a negative z component and returns an array of zeroes and ones based on the results. Notice that we don't dereference the data to the array iterator directly as that would lead to runtime errors if the representation of the array changed inside the renderer.

RSLEXPORT int findNegZ(RslContext* rslContext,
                       int argc, const RslArg* argv[])
{
    RslFloatArrayIter results(argv[0]);
    RslVectorArrayIter   vecs(argv[1]);

    int resultSize = argv[0]->GetArrayLength();

    // Walk over all the active shading points
    int numVals = argv[0]->NumValues();
    for (int i = 0; i < numVals; i++) {
        // Search the array for -z at this active point
        for (int j = 0; j < resultSize; j++) {
            if (vecs[j][2] < 0) {
                results[j] = 1.0;
            } else {
                results[j] = 0.0;
            }
        }
        ++results; ++vecs;
    }
    return 0;
}

static RslFunction myFunctions[] = {
    { "float[] findNegZ(vector[])", findNegZ, NULL, NULL},
     { NULL }
};
RSLEXPORT RslFunctionTable RslPublicFunctions = myFunctions;

For the most part, resizable arrays can be used just like ordinary arrays, using RslArrayIter. They can be resized using an RslResizer object, which is obtained from the array's RslArg. For example, this code demonstrates how to push a value onto a uniform resizable array:

RslResizer* resizer = argv[1]->GetResizer();
unsigned int n = resizer->GetLength();
resizer->Resize(n+1);

RslFloatArrayIter array(argv[1]);
RslFloatIter val(argv[2]);
array[n] = *val;

3.6   Threading and Data Lifetime

As mentioned in the string example above, memory allocated in a plugin method is not freed by the renderer; the plugin developer must perform their own memory management. There are five different ways this can be done: per frame plugin initialization, per frame function intialization, global store allocation, thread-local allocation, and shader-local allocation. The first three are not thread safe and the second two are. For memory management that is not thread safe, the developer must place synchronization primitives around memory accesses to ensure correctness.

Per frame initialization is an easy way to allocate memory for data tables that need to be created exactly once for the life of a plugin; the memory allocated by per frame initialization will persist for the duration of a frame. At the end of the frame, a cleanup routine can be called to free any allocated resources. An example that uses per frame initialization is shown here, and also in the noise example, below.

static char* tempString;

initStringBuffer(RixContext *ctx) { tempString = new char[1024]; }

deleteStringBuffer(RixContext *ctx) { delete[] tempString; }

RSLEXPORT int appendTx(RslContext* rslContext,
                       int argc, const RslArg* argv[])
{
    RslStringIter outString(argv[0]);
    RslStringIter inString (argv[1]);
    int inLen = strlen(*inString);

    // WARNING -- WITHOUT LOCKS, THIS WILL NOT WORK WITH MULTIPLE THREADS!
    // Need to lock access to tempString here
    strcpy(tempString, *inString);
    tempString[inLen] = '.';
    tempString[inLen+1] = 't';
    tempString[inLen+2] = 'x';
    tempString[inLen+3] = '\0';
    // Need to unlock access to tempString here
    *outString = tempString;

    return 0;
}

static RslFunction myFunctions[] = {
    { "string appendTx(string)", appendTx, NULL, NULL },
    { NULL }
};
RSLEXPORT RslFunctionTable RslPublicFunctions(myFunctions,
                                              initStringBuffer,
                                              deleteStringBuffer);

This version of the plugin does not leak memory on each execution, but, as noted in the comments, it has a major problem: it is not thread safe. The next example shows the use of per function initialization and uses RixInterfaces to create lock objects that ensure thread safety. However, in the example above, the string from one thread of execution will most likely be copied to strings from other threads, potentially causing an error that would be hard to debug. In addition, this version is not ideal because it assumes the strings coming into the function will never exceed 1024 characters in length.

The example below uses per function initialization to initialize global memory and locks for the given appendTx plugin function. Note however, there are two caveats with per function initialization, the first is that an initialization function will be run for every plugin function it is associated with. So if the same init function is associated with multiple plugin functions, it will be executed multiple times. The second is that cleanup functions are only run on plugin functions that have defined init functions, NULL init function pointers will prevent cleanup functions from executing.

The initializers in this example use RixInterfaces to obtain a factory object called RixThreadUtils. Once we have the RixThreadUtils factory object, we then create an RixMutex object that can be used to lock and unlock access to the shared global data in the plugin. Ideally one would never use locks in a plugin, since they reduce the paralellism that can be achieved by the multithreaded renderer.

static char* tempString;
RixMutex *s_stringLock = NULL;

initStringBuffer(RixContext *ctx)
{
    RixThreadUtils *lockFactory =
       (RixThreadUtils *)ctx->GetRixInterface(k_RixThreadUtils);
    s_stringLock = lockFactory->NewMutex();
    tempString = new char[1024];
}

deleteStringBuffer(RixContext *ctx)
{
    delete[] tempString;
    if (s_stringLock) delete s_stringLock;
}

RSLEXPORT int appendTx(RslContext* rslContext,
                       int argc, const RslArg* argv[])
{
    RslStringIter outString(argv[0]);
    RslStringIter inString (argv[1]);
    int inLen = strlen(*inString);

    // Need to lock access to tempString here
    s_stringLock->Lock();
    strcpy(tempString, *inString);
    tempString[inLen] = '.';
    tempString[inLen+1] = 't';
    tempString[inLen+2] = 'x';
    tempString[inLen+3] = '\0';
    // Need to unlock access to tempString here
    s_stringLock->Unlock();
    *outString = tempString;

    return 0;
}

static RslFunction myFunctions[] = {
    { "string appendTx(string)", appendTx, initStringBuffer, deleteStringBuffer },
    { NULL }
};
RSLEXPORT RslFunctionTable RslPublicFunctions = myFunctions;

The final example describes the use of RixInterface objects that can be used to create memory stores that can be shared across plugin functions as well as across plugin files and across plugin types. We now revisit the global appendTx example one last time:

static RixMutex *s_stringLock = NULL;

// Called automatically at end of frame
void destructGlobals(RixContext *ctx, void *buffer)
{
    delete[] (char *)buffer;
    if (s_stringLock) delete s_stringLock;
}

RSLEXPORT int appendTx(RslContext* rslContext,
                       int argc, const RslArg* argv[])
{
    RslStringIter outString(argv[0]);
    RslStringIter inString (argv[1]);
    int inLen = strlen(*inString);
    RixStorage* globalStorage = rslContext->GetGlobalStorage();

    // We must lock a global store before operating on it to be thread safe
    globalStorage->Lock();
    char *tempString = (char *)globalStorage->Get("tempString");
    if (!tempString) {
        tempString = new char[1024];
        RixThreadUtils *lockFactory =
           (RixThreadUtils *)rslContext->GetRixInterface(k_RixThreadUtils);
        s_stringLock = lockFactory->NewMutex();
        globalStorage->Set("tempString", (void *)tempString, destructGlobals);
    }
    globalStorage->Unlock();

    // Need to lock access to tempString here
    s_stringLock->Lock();
    strcpy(tempString, *inString);
    tempString[inLen] = '.';
    tempString[inLen+1] = 't';
    tempString[inLen+2] = 'x';
    tempString[inLen+3] = '\0';
    // Need to unlock access to tempString here
    s_stringLock->Unlock();
    *outString = tempString;

    return 0;
}

This example is a little more complex, because it attempts to maintain thread safety. The GetGlobalStorage method of the RslContext allows one to retrieve a global storage object. Since this is a global memory store, one has to lock it before performing operations on it with the Lock method. Then memory associated with an arbitrary key can be obtained with the Get method. If no data has been stored for this key, a NULL value is returned. If one detects that no data has been stored with this key, it can be created and put back into the global memory store with the Set method.

A full description of the interfaces available via the GetRixInterface method can be found here: RixInterface Reference.

An much simpler and more efficient, thread safe way to write this function involves using the RslContext methods GetThreadData and SetThreadData to manage a memory buffer for each thread of execution. The code to do this is shown here:

// Called automatically when a thread terminates
void destructBuffer(RixContext *ctx, void *buffer)
{
    delete[] (char *)buffer;
}

RSLEXPORT int appendTx(RslContext* rslContext,
                       int argc, const RslArg* argv[])
{
    RslStringIter outString(argv[0]);
    RslStringIter inString (argv[1]);
    int inLen = strlen(*inString);

    char *tempString = (char *)rslContext->GetThreadData();

    // If the string is NULL allocate a buffer for this thread
    if (!tempString) {
      tempString = new char[1024];
        rslContext->SetThreadData((void *)tempString, destructBuffer);
    }

    strcpy(tempString, *inString);
    tempString[inLen] = '.';
    tempString[inLen+1] = 't';
    tempString[inLen+2] = 'x';
    tempString[inLen+3] = '\0';
    *outString = tempString;

    return 0;
}

static RslFunction myFunctions[] = {
    { "string appendTx(string)", appendTx, NULL, NULL},
    { NULL }
};
RSLEXPORT RslFunctionTable RslPublicFunctions = myFunctions;

This version of our string appending function works well - it is thread safe and efficient. However the code still expects the strings will never exceed 1024 characters in length. A final revision to this function involves the use of SetLocalData and GetLocalData. These are very similar to the SetThreadData and GetThreadData methods, but the allocated memory persists only for the time that it takes to shade the current collection of points being operated on by the shader. This makes it more efficient in terms of peak memory utilization, but it can be less efficient in terms of the time utilized to call memory allocation and deallocation routines for every collection of points, rather than once per thread. The code for this version of the function is shown below:

// In this version, the destructBuffer function is called
// at the end of the shader that uses this function
RSLEXPORT int appendTx(RslContext* rslContext,
                       int argc, const RslArg* argv[])
{
    RslStringIter outString(argv[0]);
    RslStringIter inString (argv[1]);
    int inLen = strlen(*inString);

    char *tempString = (char *)rslContext->GetLocalData();

    // If the string is NULL allocate a buffer for this grid
    if (!tempString) {
      tempString = new char[inLen + 4];
        rslContext->SetLocalData((void *)tempString, destructBuffer);
    }

    strcpy(tempString, *inString);
    tempString[inLen] = '.';
    tempString[inLen+1] = 't';
    tempString[inLen+2] = 'x';
    tempString[inLen+3] = '\0';
    *outString = tempString;

    return 0;
}

3.7   Compilation Issues

Compiling and using a new plugin requires three steps:

1. Compiling the C++ file that contains your plugin functions.
2. Compiling the shader that uses your functions.
3. Rendering a frame.

Compiling your C++ file is straightforward: just use the standard C++ compiler to generate an object (.o/.obj) file, then generate a shared object (.so/.dll) file from the object file. Remember that, though using C++, you must use C-style linkage. You also must ensure that your C++ compiler and libraries are compatible with the compiler and runtime libraries used by PRMan (gcc for Linux and OS X and Microsoft Visual C for Windows). Here are example commands for building a plugin on several architectures:

Linux:

g++ -fPIC -I$RMANTREE/include -c myfunc.cpp
g++ -shared myfunc.o -o myfunc.so

Mac OS X:

g++ -I$RMANTREE/include -c myfunc.cpp
setenv MACOSX_DEPLOYMENT_TARGET 10.3
g++ -bundle -undefined dynamic_lookup myfunc.o -o myfunc.so

Windows:

cl -nologo -MT -I%RMANTREE%\include -c myfunc.cpp
link -nologo -DLL -out:myfunc.dll myfunc.obj %RMANTREE%\lib\prman.lib

The resulting file myfunc.so or myfunc.dll is the plugin that implements your new function. It is not important that the filename matches the name of the function.

When compiling your shader, if the RSL compiler comes across a function reference that is neither a known built-in nor a function defined in RSL, it will search all plugins defined with the plugin directive until it finds one within the exported RslPublicFunctions table. Note that the plugin file must be specified with the plugin directive. (The .so or .dll extension is not required.) The path to plugins can be relative and can be modified with the -I command line option of the RSL compiler. The RSL compiler will typecheck your function call and issue a warning if you pass arguments that do not match any of the entries in the RslPublicFunctions table of your plugin.

Once your shader is successfully compiled, you are ready to render. The renderer requires the plugin to be in one of the directories that it searches for shaders. In other words, the searchpath specified with the option Option "searchpath" "shader" also specifies the searchpath for the RSL function plugins.