# The Importance of Importance

November, 2010 (Revised September, 2011)

### Introduction

This application note discusses the importance of using importance to guide accuracy in RenderMan shaders, and aims to show how shaders can be written with efficient ray tracing in mind. In the course of this discussion, we will also examine controlling the number of samples used for area lights.

The importance of a ray represents how much that ray ultimately will contribute to the image. Using this knowledge can be exploited to make many calculations more efficient.

### Reflections and Refractions

First of all, let's look at a simple scene with ray tracing. We're going to employ a ray tracing glass shader to model the reflections and refractions that occur when light passes through glass.

surface
glass1(float Kr = 1.0;   // coefficient for ideal (mirror) reflection
float Kt = 1.0;   // coefficient for ideal refraction
float ior = 1.5;  // index of refraction
float Ks = 1.0;   // specular reflection coefficient
float shinyness = 20.0;   // Phong exponent
float samples = 4)        // number of rays (for antialiasing)
{
color ci, hitci;
normal Nn = normalize(N);
vector In = normalize(I);
normal Nf = faceforward(Nn, In);
vector V = -In;   // view direction
vector reflDir, refrDir;
float eta = (In.Nn < 0) ? 1/ior : ior;   // relative index of refraction
float kr, kt;

Ci = 0;

// Compute specular highlight
if (Ks > 0) {
Ci += Ks * specular(Nn, V, 1/shinyness);
}

// Compute Fresnel reflection and refraction coefficients (kr and kt)
// and reflection and refraction directions.  If there is total
// internal reflection kt is set to 0 and refrDir is set to (0,0,0).
fresnel(In, Nf, eta, kr, kt, reflDir, refrDir);
kt = 1 - kr;

// Shoot reflection rays
if (Kr * kr > 0) {
ci = 0; hitci = 0;
gather("illuminance", P, reflDir, 0, samples,
"surface:Ci", hitci) {
ci += hitci;
}
Ci += Kr * kr * ci / samples;
}

// Shoot refraction rays
if (Kt * kt > 0) {
ci = 0; hitci = 0;
gather("illuminance", P, refrDir, 0, samples,
"surface:Ci", hitci) {
ci += hitci;
}
Ci += Kt * kt * ci / samples;
}

Oi = 1;
}

Simple enough! What could possibly go wrong? Well, typically, refractive effects require a large ray depth (Attribute "trace" "maxspeculardepth" 5 or higher). At each ray level we will fire four rays for reflection, and four more for refraction. Each of those might hit the glass again and fire 8 more in turn. Even with relatively small max specular depths, this means we're firing a lot of rays, and it gets costly very quickly.

To illustrate this point, using the glass1 shader to compute the following image of a simple test scene with three glass objects results in 1.1 billion rays fired (13 minutes runtime on an 8-core machine).

(This image was rendered with importance culling turned off, ie. with Attribute "trace" "float importancethreshold" 0.0. More on this attribute below.)

#### What to do?

Firstly, and traditionally, the sample count can be limited using ray depth as a guide. For example, we could do the following:

// Reduce the number of rays at deep levels
uniform float raydepth, sam;
rayinfo("depth", raydepth);
sam = (raydepth <= 2) ? samples : 1;   // only 1 ray for deep refr/refl!

That's OK. Now at ray depths greater than 2, we will only fire one reflection ray and one refraction ray. This can trim the number of rays we fire significantly: for the scene in the image above, the number of rays is now 153 million (1.5 minutes runtime) while the image is nearly identical. (Another variation is to shoot more rays at ray depth 0 and only 1 reflection ray and 1 refraction ray at all depths greater than 0.)

However, the metric is coarse and we can do better.

#### Importance thresholding

In both of the gather constructs above, the results are multiplied by a weight based on the input parameters Kr and Kt and the Fresnel coefficients kr and kt. This is multiplicative down the ray tree. If the result of a given gather() call is going to be multiplied by something small, then potentially the fidelity of the result is less important. The potentially comes from cases where a low-weighted ray hits a very bright surface. Nevertheless, if a low-weighted ray results in shading that fires another low-weighted ray, we can rapidly land in a situation where the net result of those deeper rays is not visually significant to the final shaded result.

If we were able to pass to the hit shader the weight by which a given set of rays will be multiplied, that shader might be able to modify its behavior so that it in turn only fires a number of rays that is appropriate for the weight the result will have in the final picture.

Fortunately, PRMan has always allowed you to pass a parameter to the shader invoked by a ray, using gather() and its "send:surface" mechanism. Unfortunately, that requires all shaders to be authored to accept the parameter for the system to be fully effective.

PRman also has a built-in mechanism for trimming ray counts using importance weighting. The gather() call accepts a parameter called weight, of type varying color. The default is color(1). When a shader fires rays using gather(), the current importance is multiplied by the weights passed to gather(). This means that the importance down the ray tree is calculated properly (low weight rays that invoke further low weight rays are deemed less important than those with higher weights).

If the accumulated importance is at or below Attribute "trace" "float importancethreshold" the gather() and indirectspecular() calls only fire some of the relevant rays, but will give the remaining rays higher weight using the Monte Carlo technique known as Russian roulette.

Here's an example with an extension of the refraction block of code from above:

// Shoot refraction rays
if (Kt * kt > 0) {
color weight = color(Kt * kt);   // NEW LINE HERE
ci = 0; hitci = 0;
gather("illuminance", P, refrDir, 0, sam,
"weight", weight,   // NEW LINE HERE
"surface:Ci", hitci) {
ci += hitci;
}
Ci += Kt * kt * ci / sam;
}

(Similar change in the reflection block.)

No other interaction is required from the shader other than supplying the weights. Contrast this to using "send:surface:XYZ", where the hit shader would have to check the accumulated importance and multiply it by additional weights when firing further rays.

The importance cut-off can be tweaked by setting the importancethreshold attribute or passing it as a parameter to gather() or indirectspecular(). (As mentioned above, the default value is 0.001.)

Doing this trims the ray tree and prevents it becoming to deep or too bushy in areas where the overall contribution is not high enough to make a visually salient difference. We are still, however, presented with a potentially large number of rays that must be fired because they might make an important contribution to the picture.

#### Going further

One possible approach, rather than relying on the Russian roulette decisions made by the importancethreshold attribute but still leveraging PRMan's importance sampling support, is to modify the number of samples based on the computed importance. This can be done like so:

varying float maxImportance = 1;
rayinfo("importance", maxImportance);   // max of (r,g,b) importance
sam = samples * maxImportance;

or via a similar approach. In this way, we can limit the number of samples down the ray tree in a more fine-grained fashion. The number of reflection and refraction rays is still tied together, though, and we are not exploiting that the weight of reflections may be different than the weight of refractions.

We can instead make the number of reflection and refraction rays independent of each other. Furthermore, we can implement the Russian roulette technique explicitly in the shader (instead of using PRMan's built-in Russian roulette) to probabilistically decide whether to shoot a ray or not when the importance is low. Here's a shader snippet illustrating this:

// Reduce the number of rays when importance is low.  Use Russian
// roulette when the computed number of rays is between 0 and 1.
float maxImportance;
rayinfo("importance", maxImportance);   // max of (r,g,b) importance
float reflSam = Kr * kr * samples * maxImportance;
float refrSam = Kt * kt * samples * maxImportance;

...

// Shoot refraction rays
if (refrSam > RRthreshold) {   // shoot 1 or more rays
color weight = color(Kt * kt);
float sam = max(round(refrSam), 1);
ci = 0; hitci = 0;
gather("illuminance", P, refrDir, 0, sam,
"weight", weight,
"surface:Ci", hitci) {
ci += hitci;
}
Ci += Kt * kt * ci / sam;
} else if (refrSam/RRthreshold > random()) {   // Russian roulette: 0/1 ray
ci = 0; hitci = 0;
gather("illuminance", P, refrDir, 0, 1,
"surface:Ci", hitci) {
ci += hitci;
}
Ci += ci;   // no mult by Kt * kt to increase weight of surviving rays
}

What we're doing here is determining when the importance dictates that a fractional number of rays be fired, and probabilistically choosing to fire 0 or 1 ray (independently for reflection or refraction). Note that we avoid multiplying by the Kt and kt coefficients because we've effectively weighted by them already in choosing whether to shoot the ray or not.

When using Russian roulette in the surface shaders, the importancethreshold attribute should be set to 0.0.

Here's an example of a Russian roulette glass shader that puts it all together:

surface
glassrr(float Kr = 1.0;   // coefficient for ideal (mirror) reflection
float Kt = 1.0;   // coefficient for ideal refraction
float ior = 1.5;  // index of refraction
float Ks = 1.0;   // specular reflection coefficient
float shinyness = 20.0;   // Phong exponent
float samples = 4;        // number of rays (for antialiasing)
float RRthreshold = 0.001) // Russian roulette threshold
{
color ci, hitci;
normal Nn = normalize(N);
vector In = normalize(I);
normal Nf = faceforward(Nn, In);
vector V = -In;   // view direction
vector reflDir, refrDir;
float eta = (In.Nn < 0) ? 1/ior : ior;   // relative index of refraction
float kr, kt;

Ci = 0;

// Compute specular highlight
if (Ks > 0) {
Ci += Ks * specular(Nn, V, 1/shinyness);
}

// Compute Fresnel reflection and refraction coefficients (kr and kt)
// and reflection and refraction directions.  If there is total
// internal reflection kt is set to 0 and refrDir is set to (0,0,0).
fresnel(In, Nf, eta, kr, kt, reflDir, refrDir);
kt = 1 - kr;

// Reduce the number of rays when importance is low.  Use Russian
// roulette when the computed number of rays is between 0 and 1.
float maxImportance;
rayinfo("importance", maxImportance);   // max of (r,g,b) importance
float reflSam = Kr * kr * samples * maxImportance;
float refrSam = Kt * kt * samples * maxImportance;

// Shoot reflection rays
if (reflSam > RRthreshold) {   // shoot 1 or more rays
color weight = color(Kr * kr);
float sam = max(round(reflSam), 1);
ci = 0; hitci = 0;
gather("illuminance", P, reflDir, 0, sam,
"weight", weight,
"surface:Ci", hitci) {
ci += hitci;
}
Ci += Kr * kr * ci / sam;
} else if (reflSam/RRthreshold > random()) {   // Russian roulette: 0/1 ray
ci = 0; hitci = 0;
gather("illuminance", P, reflDir, 0, 1,
"surface:Ci", hitci) {
ci += hitci;
}
Ci += ci;   // no mult by Kr * kr to increase weight of surviving rays
}

// Shoot refraction rays
if (refrSam > RRthreshold) {   // shoot 1 or more rays
color weight = color(Kt * kt);
float sam = max(round(refrSam), 1);
ci = 0; hitci = 0;
gather("illuminance", P, refrDir, 0, sam,
"weight", weight,
"surface:Ci", hitci) {
ci += hitci;
}
Ci += Kt * kt * ci / sam;
} else if (refrSam/RRthreshold > random()) {   // Russian roulette: 0/1 ray
ci = 0; hitci = 0;
gather("illuminance", P, refrDir, 0, 1,
"surface:Ci", hitci) {
ci += hitci;
}
Ci += ci;   // no mult by Kt * kt to increase weight of surviving rays
}

Oi = 1;
}

This revamped shader is considerably faster than the original one: it renders the following image using only 8.7 million rays (6 seconds runtime).

This image is nearly identical to the first glass image.

#### Under the hood

How is importance actually implemented "under the hood"?

PRMan takes the parent ray's importance and multiplies it by the weights provided by the shader in the gather() call. Importance (1,1,1) is used in Reyes rendering for directly visible objects that don't have a parent ray. When using the raytrace hider in PRMan 18, the importance of each camera ray is set to 1. As of PRMan 19, when using the raytrace hider the importance of each camera ray will be set to 1/(psx*psy), where psx and psy are the PixelSamples values.) Weights of (1,1,1) are used if no weights are provided. If this product is at or below importancethreshold (0.001 by default) in all three color bands then Russian Roulette is used to determine whether or not to trace the new ray.

For a ray that survives this "early out", this product is divided by the number of rays. This is the importance assigned to the new ray.

With this calculation, the importance carried with each ray represents how much the color at the ray's hit point will ultimatively contribute to the image.

#### Other uses of the importance function

Importance can be used in other shaders than just glass. For example, if importance is propagated by all shaders in a scene, shaders can check the importance of the result they are about to compute and make approximations if the importance is low: coarser sampling of shadows, blurrier texture lookups, etc.

Potential pitfall: It is important that the switch from accurate to approximate calculation isn't abrupt. For example, it would be wrong for the shader to do full texture lookups if the importance is above a certain threshold, but use a constant color if it is below. Here's why: with such a shader, if the shader that shoots the rays that hit it increases the number of rays that it shoots a little bit, the importance of each ray is decreased a bit, and may suddenly fall below the importance threshold - and the shader will suddenly compute colors that are too different. Instead, the approximations used at low importance must be gradually phased in: the lower the importance, the more approximate the calculation can be.

The observant reader will notice that in PRMan versions prior to 16.2 we actually fell into this pitfall by implementing an automatic "early out" for all rays with low importance . Opting to simply not trace all rays with importance below 0.001 can produce undesirable results. It is more correct to use Russian roulette if the target number of rays is between 0 and 1, as we do in PRMan 16.2 and higher.

There is one potentially confusing issue with importance and radiosity caching.

When we run the diffuselighting() shader method the result is going to be stored in the radiosity cache. It would be dangerous to use the importance of the first ray that hits a patch to simplify shading calculations since another ray with higher importance might later hit the same patch and reuse the shading result. So importance should not be used in diffuselighting() methods. To safeguard against unintended mistakes, rayinfo() explicitly returns importance 1 when called from diffuselighting().

In summary: importance is a great source of optimizations, but not in diffuselighting() methods. One should not use importance there. However, even though importance can't be used to optimize calculations in diffuselighting() methods, one can still use ray depth etc. to simplify calculations.

### BRDF Importance Sampling

Another use of importance is for importance sampling of the BRDF function.

The surface BRDF will result in some samples from some angles being of lower importance than others. Carefully controlling the number of samples shot using the expected BRDF weight will help keep renders fast. This is at the core of the techniques described in the PRMan application note Physically Plausible Shading in RSL.

### Solid Angle Sample Reduction

The overall weight of an area light on a surface is maximally 2 PI. Samples subtend an angle with the surface. Each sample must be weighted such that its form factor projects appropriately onto the hemisphere. For each area light sample we should weight that area by N_light.L. It may be helpful to express the maximal number of samples as the number required to accurately sample the hemisphere. The actual number used may be lower and we could, for example, use:

samples = maxSamples*areaOfLight/(2*PI)

### Summary

Leveraging the importance mechanism in PRMan can significantly improve the speed of ray-traced renders. When writing shaders to be used with ray tracing, keep in mind the likely number of ray bounces and the ray depths required to produce the quality of picture you need. Especially for shaders like glass, which need higher ray depths to produce production-quality pictures, being aware of the likely explosion of ray counts down the tree is important. Taking care to ensure the budget of rays you fire are spent only on visually important portions of the final result can drastically improve the performance of your renders.

Using the solid angle subtended by an area light to compute the required sampling density can also help control the number of rays/area light samples required, and improve the performance of scenes that employ area lighting.

Russian Roulette is a standard method in Monte Carlo simulations in physics and applied mathematics. It was introduced to computer graphics in the following paper:

• James Arvo and David Kirk", "Particle transport and image synthesis", Proc. SIGGRAPH 1990, pp. 63-66.

A general overview of the use of importance for speeding up rendering can be found in this article:

• Per Christensen, "Adjoints and Importance in Rendering: an Overview", IEEE Trans. Visualization and Computer Graphics, 9(3), pp. 329-340, 2003.

Many details of importance sampling and multiple importance sampling are described in this paper:

• Eric Veach and Leonidas Guibas, "Optimally combining sampling techniques for Monte Carlo rendering", Proc. SIGGRAPH 1995, pp. 419-428.