Plug-in Rif Filters
Plug-in Rif Filters
May 2004 (Updated Nov 2010)
Introduction
Release 12 of Pixar's PhotoRealistic RenderMan introduced a new RIB pipeline feature. We have found that pipeline designers appreciate programmability, so we identified and implemented a new contact-point where pipeline designers and TDs can "do their dirty work." This app note introduces a new class of RenderMan plugin that offers new opportunities to improve the flexibility, performance, and encapsulation of a RenderMan-based production pipeline.
Plug-in Ri Filters
Many of the RIB fixup utilities found in RenderMan production pipelines can accurately be described as RenderMan Interface filters. They parse a RIB stream, search for patterns that match some criteria, and replace these patterns with a more preferable, possibly empty, pattern. It is often convenient to construct chains of RIB filters and the result is a powerful back-end RIB processing system that can solve production pipeline problems in a unique way, due largely to its position in the pipeline at the "mouth" of the renderer. To the extent that these filtering systems process data from one on-disk representation to another, it can be far from optimal - it is easy to construct a system that spends more time filtering RIB than rendering. To eliminate the disk I/O overhead and the burdens associated with parsing RIB, we have introduced plug-in Ri filters.
The figure above depicts the contract between the RenderMan renderer and the various plugin types currently defined. The Ri calls are at the heart of the system and allow procedural primitives and plug-in filters to fundamentally affect the RI stream driving the renderer. The Rx library is provided to allow plugins to query rendering state and is available to all plugin types. For example, RxOption and RxAttribute can be used to look up values of rendering state variables. The new Rif interface is provided to allow plug-in Ri filters to query and affect the state of the active filter chain. The names in boxes represent the entry points required of the various plugin types by the renderer. Ri filter plugins must implement the RifPluginManufacture method, and this must construct a C++ object whose class is a derivative of the virtual base-class, RifPlugin. The GetFilter method is required to return a dispatch table of Ri functions and it is through this table that an Ri Filter plugin can do its work. By implementing a subset of the procedures in the RifFilter structure, the plugin establishes callbacks that will be invoked as an individual Ri request works its way through the filter chain.
Plugins can control whether Ri procedures not implemented by the plugin result in pass-through or terminate behavior. If a plugin only implements the RiSphere request, the default filtering mode would determine whether the result was a RIB stream of spheres or everything but spheres.
Implementation Example
A trivial filter that converts all spheres to cones would set the default filtering mode to pass-through and would implement a procedure to be called when spheres are encountered. There it would simply invoke RiCone. Because an individual filter has access, via the Rif interface, to the state of the Ri filter chain, it can choose to invoke new procedures at the beginning, current, or next point in the filter chain. Here are some snippets from a useful Ri filter plug-in that applies regexp pattern matching to a RIB stream. RIB requests that are bound to an identifier matching the user-provided regular expression are filtered.
First we define our subclass of the RifPlugin virtual base class:
#include "ri.h"
#include "RifPlugin.h"
#include <stdio.h>
#include <regex.h>
#include <stack>
#include <string>
class matcher : public RifPlugin
{
public:
matcher(const char *expr, bool invert);
virtual ~matcher();
virtual RifFilter &GetFilter(); // Returns my filled in dispatch table
private:
RifFilter m_rifFilter;
// Here's my dispatch table we fill it in at
// construction
void setName(const char *);
// This will be called whenever Attribute
// "identifier" "name" changes.
std::stack<bool> m_validityStack;
// Tracks AttributeBegin/AttributeEnd
// contains bool representing whether the current
// Attribute "identifier" "name" matches the
// expression.
std::string m_expression;
// Here's the expression provided by the user
regex_t m_regexp;
// A compiled version of the user's expression
bool m_invert;
// Should we invert the sense of the test?
// Here are the class methods which will be plugged into the RifFilter
// dispatch table. They will need to access the internals of the current
// matcher object when invoked.
static RtVoid myAttributeBegin();
static RtVoid myAttributeEnd();
static RtVoid myAttributeV(RtToken nm, RtInt n, RtToken tok[],
RtPointer val[]);
};
Next we implement the entrypoint that the RifPlugin machinery will invoke to construct an instance of our matcher class. Note that instantiation arguments are provided at the time of construction. For brevity, we omit error checking.
RifPlugin *
RifPluginManufacture(int argc, char **argv)
{
char *pattern = 0L;
bool invert = false;
// here we parse incoming arguments looking for -pat regexp and -inv
for (; argc > 0 && argv[0][0] == '-'; argc--, argv++)
{
if( !strncmp("-pat", argv[0], 4) && argc > 1 )
{
argc--;
argv++;
pattern = argv[0];
}
else
if( !strcmp("-inv", argv[0]) )
invert = true;
}
if(!pattern) return 0L;
// all okay here, so construct an instance of matcher
matcher *s = new matcher(pattern, invert);
return s;
}
Next, our constructor initializes the members of the filter table with our overrides, and does some book-keeping.
matcher::matcher(const char *expr, bool invert) :
m_invert(invert), m_expression(expr)
{
// First we initialize our RifFilter structs with the (static) class
// methods
m_rifFilter.AttributeBegin = myAttributeBegin;
m_rifFilter.AttributeEnd = myAttributeEnd;
m_rifFilter.AttributeV = myAttributeV;
// here we initialize other state and compile the regular expression
m_validityStack.push(!m_invert);
int err = regcomp(&m_regexp, expr, REG_EXTENDED|REG_NOSUB);
}
matcher::~matcher() {
regfree(&m_regexp);
}
RifFilter & matcher::GetFilter()
{
return m_rifFilter;
}
Finally, we define our Ri filter override procedures. Note that, since the callback functions are static class members, we recover the per-object context via a call to RifGetCurrentPlugin. The key is to look for changes to the RenderMan attribute "identifier:name". These come about when the attribute stack is popped or when the attribute is set. These procedures manage the validity stack and apply the pattern matcher, regexec, when needed. Whenever the validity state changes the filter sets its filtering mode accordingly, and since we have not implemented any other Ri requests, they are subject to this filtering mode.
RtVoid
matcher::myAttributeBegin()
{
// find matcher object
matcher *obj = static_cast<matcher *> (RifGetCurrentPlugin());
// push the current validity
obj->m_validityStack.push(obj->m_validityStack.top());
RiAttributeBegin(); // pass the RiAttribute call down the chain
}
RtVoid
matcher::myAttributeEnd()
{
matcher *obj = static_cast<matcher *> (RifGetCurrentPlugin());
obj->m_validityStack.pop();
if (obj->m_validityStack.top()) // the validity may have changed
obj->m_rifFilter.Filtering = RifFilter::k_Continue;
else
obj->m_rifFilter.Filtering = RifFilter::k_Terminate;
RiAttributeEnd(); // pass the call down the chain
}
RtVoid
matcher::myAttributeV(RtToken nm, RtInt n, RtToken tokens[],
RtPointer parms[])
{
if( !strcmp(nm, "identifier") && !strcmp(tokens[0], "name") )
{
matcher *obj = static_cast<matcher *> (RifGetCurrentPlugin());
obj->setName( ((RtString *) parms[0])[0] );
}
RiAttributeV(nm, n, tokens, parms); // pass the call down the chain
}
void
matcher::setName(const char *nm)
{
// name has changed, check for a pattern match
bool match;
if( 0 == regexec( &m_regexp, nm, 1, 0, 0) )
match = true;
else
match = false;
m_validityStack.top() = m_invert ? !match : match;
if(m_validityStack.top())
m_rifFilter.Filtering = RifFilter::k_Continue;
else
m_rifFilter.Filtering = RifFilter::k_Terminate;
}
Using Ri Filters
Now we are ready to run our filter. Rif chains can be described via prman command-line options.
% prman -rif matcher.so -rifargs -pat ^her -rifend example.rib
This invocation identifies the location of the Ri filter plugin and establishes the instantiation arguments for this run of the filter. Here we invoke a single Ri filter but we can easily chain them by supplying multiple -rif blocks. The ordering of the -rif blocks on the command line determines the placement of each filter in the filter chain. Depending on the behavior of your collection of filters, dramatically different results can be obtained simply by reordering the filter chain.
Compared to ad-hoc or home-grown solutions for RIB filtering, this new plugin type offers a number of advantages. First, the RIB parsing problem is eliminated. The native RIB parser built into the renderer does all the heavy lifting. More importantly, because the filters are expressed as plugins, there is no need to put the results onto disk. This virtually eliminates the run-time and disk space costs associated with most RIB filters.
There are times when it is desireable to store the filtered RIB stream rather than render it. This turns out to be simple.
% catrib -o filtered.rib -rif matcher.so -rifargs \
-pat ^her -rifend example.rib
Now the filtered RIB will magically appear in filtered.rib, and will not be rendered. For this version of example.rib:
FrameBegin 1
Display "test.tif" "tiff" "rgba"
Format 512 512 1
Projection "perspective" "fov" [45]
Translate 0 0 10
WorldBegin
AttributeBegin
Attribute "identifier" "name" "my sphere"
Sphere .5 -.5 .5 360
"constant float foo" [.1]
"varying float bar" [0 1 2 3]
"varying float[2] bar2" [0 0 1 1 2 2 3 3]
Attribute "identifier" "name" "his cones"
Cone 1 .5 360
AttributeBegin
Attribute "identifier" "name" "her cylinder and cone"
Cylinder .5 -1 1 360
Cone 2 .5 360
AttributeEnd
Cone 3 .5 360
AttributeEnd
Cone 4 .5 360
Polygon "P" [0 0 0 1 1 1 2 2 2]
WorldEnd
FrameEnd
This is the result of the above filtering:
##RenderMan RIB version 3.04 FrameBegin 1 Display "test.tif" "tiff" "rgba" Format 512 512 1 Projection "perspective" "fov" [45] Translate 0 0 10 WorldBegin AttributeBegin Attribute "identifier" "name" ["my sphere"] Attribute "identifier" "name" ["his cones"] AttributeBegin Attribute "identifier" "name" ["her cylinder and cone"] Cylinder 0.5 -1 1 360 Cone 2 0.5 360 AttributeEnd AttributeEnd Cone 4 .5 360 Polygon "P" [0 0 0 1 1 1 2 2 2] WorldEnd FrameEnd
Note the disappearance of "my sphere" and "his cones" because they did not match the pattern. The matcher plugin is a useful utility that has many of the properties desirable of Ri filter plugins. First, it is a very compact plugin with a tiny, easily maintained codebase. Second, its functionality is easy to understand and thus its presence in a chain of filters should offer no surprising side effects. You can even use multiple instances of matcher in a pipeline. As an exercise, run the following snippet to see how many geometric primitives result.
% catrib -rif matcher.so -rifargs -pat cone -rifend \
-rif matcher.so -rifargs -pat ^her -inv -rifend
In case your imagination hasn't already run wild, here are a few potential Ri filter applications to consider:
- convert all polygons to subdivision surfaces
- generate level of detail representations of heavy primitives
- fixup texture or shader references
- flatten the transform hierachy
The Joy of Rx/Rix
The example above, while compact, has to keep track of its own AttributeBegin and AttributeEnd stack and note when an Attribute "identifier:name" matches. Fortunately RenderMan is already keeping track of the Attribute state for us. We can simply use the RxAttribute() call to query the "identifier:name" of a piece of geometry and determine its fate based on a match.
Let's say we instead want a filter that simply removes all spheres that have a name that matches the given pattern. We could re-write the matcher plugin above and remove call backs to the Attribute functions and instead just add a single call back for RiSphere. We also set the default pass through mode to k_Continue.
matcher::matcher(const char *expr, bool invert) :
m_invert(invert),
m_expression(expr)
{
// initialize our RifFilter structs with the (static) class methods
m_rifFilter.SphereV = mySphereV;
m_rifFilter.Filtering = RifFilter::k_Continue;
// here we initialize other state and compile the regular expression
int err = regcomp(&m_regexp, expr, REG_EXTENDED|REG_NOSUB);
}
Now we just need an implementation for our Sphere callback.
RtVoid
matcher::mySphereV(RtFloat radius, RtFloat zmin, RtFloat zmax, RtFloat tmax,
RtInt n, RtToken tokens[], RtPointer parms[])
{
char ``sphereName = (char ``)alloca(1);
RxInfoType_t type;
RtInt resultcnt, size;
matcher *obj = static_cast<matcher *>( RifGetCurrentPlugin() );
// Ask prman for the identifier:name
size = RxAttribute("identifier:name", sphereName, sizeof(char*),
&type, &resultcnt);
if (!size && (0 == regexec (&obj->m_regexp, *sphereName, 1, 0, 0)))
RiSphereV(radius, zmin, zmax, tmax, n, tokens, parms);
}
If we were to run this example on the RIB stream below, only "his sphere" would render and "her sphere" would be removed.
FrameBegin 1
Display "test.tif" "tiff" "rgba"
Format 512 512 1
Projection "perspective" "fov" [45]
Translate 0 0 10
WorldBegin
AttributeBegin
Attribute "identifier" "name" "his sphere"
Color [0 0 1]
Sphere .5 -.5 .5 360
"constant float foo" [.1]
"varying float bar" [0 1 2 3]
"varying float[2] bar2" [0 0 1 1 2 2 3 3]
AttributeEnd
AttributeBegin
Attribute "identifier" "name" "her sphere"
Color [1 0 0]
Sphere .9 -.9 .9 360
AttributeEnd
WorldEnd
FrameEnd
However, if we want the result of this filter written to a file, we cannot use the catrib utility anymore. Instead we have to now use the -catrib flag to the prman executable, because only prman can keep track of the graphics state. The following is the result of running:
% prman -catrib filtered.rib -rif matcher.so \
-rifargs -pat ^his -rifend example.rib
Once again, the filtered RIB will magically appear in filtered.rib and not be rendered, even though it will have maintained the correct graphics state:
##RenderMan RIB
version 3.04
FrameBegin 1
Display "test.tif" "tiff" "rgba"
Format 512 512 1
Projection "perspective" "fov" [45]
Translate 0 0 10
WorldBegin
AttributeBegin
Attribute "identifier" "name" "his sphere"
Color [0 0 1]
Sphere .5 -.5 .5 360
"constant float foo" [.1]
"varying float bar" [0 1 2 3]
"varying float[2] bar2" [0 0 1 1 2 2 3 3]
AttributeEnd
AttributeBegin
Attribute "identifier" "name" "her sphere"
Color [1 0 0]
AttributeEnd
WorldEnd
FrameEnd
Parsing Parameters
One thing developers of Ri filter plugins may want is to alter the parameter list of an Ri call or modify the behavior of the current filter based on the contents of a parameter list. Fortunately there is a nice helper function that can be used to do this, RifGetDeclaration(). As an example, the Sphere matcher could examine the additional arguments to the Sphere call as follows:
RtVoid
matcher::mySphereV(RtFloat radius, RtFloat zmin, RtFloat zmax, RtFloat tmax,
RtInt n, RtToken tokens[], RtPointer parms[])
{
static char* types[] = {
"float",
"point",
"color",
"integer",
"string",
"vector",
"normal",
"hpoint",
"matrix",
"mpoint" };
static char* details[] = {
"constant",
"uniform",
"varying",
"vertex",
"varying"};
/* Echo parameter list as a comment into the RIB stream*/
for(int i=0;i<n;i++)
{
RifTokenType type;
RifTokenDetail detail;
int arraylen;
if(0 == RifGetDeclaration(tokens[i], &type, &detail, &arraylen))
{
/* GetDeclaration always returns 1, even for non-array tokens */
if (arraylen > 1 || strrchr((const char *)tokens[i], '['))
{
RiArchiveRecord(RI_COMMENT, "\t%s: %s %s[%d]",
tokens[i], details[(int) detail],
types[(int) type], arraylen);
}
else
{
RiArchiveRecord(RI_COMMENT, "\t%s: %s %s",
tokens[i], details[(int) detail],
types[(int) type]);
}
}
}
}
Archives and Procedurals
One of the nice features about the new Ri filters is that they work even if your RIB stream is composed of Procedurals and Archives, be they file-based or in-line. In other words, all RIB that passes through to the renderer will also pass through the Ri filter pipeline.
Using our sphere matcher example above, the following RIB below:
FrameBegin 1
Display "test.tif" "tiff" "rgba"
Format 512 512 1
Projection "perspective" "fov" [45]
Translate 0 0 10
ArchiveBegin "thisworks"
Sphere .5 -.5 .5 360
"constant float foo" [.1]
"varying float bar" [0 1 2 3]
"varying float[2] bar2" [0 0 1 1 2 2 3 3]
ArchiveEnd
WorldBegin
AttributeBegin
Attribute "identifier" "name" "his sphere"
Color [0 0 1]
ReadArchive "thisworks"
AttributeEnd
AttributeBegin
Attribute "identifier" "name" "her sphere"
Color [1 0 0]
Sphere .9 -.9 .9 360
AttributeEnd
WorldEnd
FrameEnd
will become this resulting RIB:
##RenderMan RIB
version 3.04
FrameBegin 1
Display "test.tif" "tiff" "rgba"
Format 512 512 1
Projection "perspective" "fov" [45]
Translate 0 0 10
WorldBegin
AttributeBegin
Attribute "identifier" "name" "his sphere"
Color [0 0 1]
#RenderMan RIB
Sphere .5 -.5 .5 360
"constant float foo" [.1]
"varying float bar" [0 1 2 3]
"varying float[2] bar2" [0 0 1 1 2 2 3 3]
AttributeEnd
AttributeBegin
Attribute "identifier" "name" "her sphere"
Color [1 0 0]
AttributeEnd
WorldEnd
FrameEnd