# The Importance of Importance

# The Importance of Importance

*November, 2010 (Revised September, 2011)*

### Introduction

This application note discusses the importance of using importance to guide accuracy in RenderMan shaders, and aims to show how shaders can be written with efficient ray tracing in mind. In the course of this discussion, we will also examine controlling the number of samples used for area lights.

The importance of a ray represents how much that ray ultimately will contribute to the image. Using this knowledge can be exploited to make many calculations more efficient.

### Reflections and Refractions

First of all, let's look at a simple scene with ray tracing. We're going to employ a ray tracing glass shader to model the reflections and refractions that occur when light passes through glass.

The following glass shader is one such shader we could use:

surface glass1(float Kr = 1.0; // coefficient for ideal (mirror) reflection float Kt = 1.0; // coefficient for ideal refraction float ior = 1.5; // index of refraction float Ks = 1.0; // specular reflection coefficient float shinyness = 20.0; // Phong exponent float samples = 4) // number of rays (for antialiasing) { color ci, hitci; normal Nn = normalize(N); vector In = normalize(I); normal Nf = faceforward(Nn, In); vector V = -In; // view direction vector reflDir, refrDir; float eta = (In.Nn < 0) ? 1/ior : ior; // relative index of refraction float kr, kt; Ci = 0; // Compute specular highlight if (Ks > 0) { Ci += Ks * specular(Nn, V, 1/shinyness); } // Compute Fresnel reflection and refraction coefficients (kr and kt) // and reflection and refraction directions. If there is total // internal reflection kt is set to 0 and refrDir is set to (0,0,0). fresnel(In, Nf, eta, kr, kt, reflDir, refrDir); kt = 1 - kr; // Shoot reflection rays if (Kr * kr > 0) { ci = 0; hitci = 0; gather("illuminance", P, reflDir, 0, samples, "surface:Ci", hitci) { ci += hitci; } Ci += Kr * kr * ci / samples; } // Shoot refraction rays if (Kt * kt > 0) { ci = 0; hitci = 0; gather("illuminance", P, refrDir, 0, samples, "surface:Ci", hitci) { ci += hitci; } Ci += Kt * kt * ci / samples; } Oi = 1; }

Simple enough! What could possibly go wrong? Well, typically,
refractive effects require a large ray depth (`Attribute "trace"
"maxspeculardepth" 5` or higher). At each ray level we will fire four
rays for reflection, and four more for refraction. Each of those might
hit the glass again and fire 8 more in turn. Even with relatively
small max specular depths, this means we're firing a lot of rays, and
it gets costly very quickly.

To illustrate this point, using the `glass1` shader to compute the
following image of a simple test scene with three glass objects results in
1.1 billion rays fired (13 minutes runtime on an 8-core machine).

(This image was rendered with importance culling turned off, ie. with
`Attribute "trace" "float importancethreshold" 0.0`. More on this
attribute below.)

#### What to do?

Firstly, and traditionally, the sample count can be limited using ray depth as a guide. For example, we could do the following:

// Reduce the number of rays at deep levels uniform float raydepth, sam; rayinfo("depth", raydepth); sam = (raydepth <= 2) ? samples : 1; // only 1 ray for deep refr/refl!

That's OK. Now at ray depths greater than 2, we will only fire one reflection ray and one refraction ray. This can trim the number of rays we fire significantly: for the scene in the image above, the number of rays is now 153 million (1.5 minutes runtime) while the image is nearly identical. (Another variation is to shoot more rays at ray depth 0 and only 1 reflection ray and 1 refraction ray at all depths greater than 0.)

However, the metric is coarse and we can do better.

#### Importance thresholding

In both of the gather constructs above, the results are multiplied by a
weight based on the input parameters `Kr` and `Kt` and the Fresnel
coefficients `kr` and `kt`. This is
multiplicative down the ray tree. If the result of a given
`gather()` call is going to be multiplied by something small, then
potentially the fidelity of the result is less important. The *potentially*
comes from cases where a low-weighted ray hits a very bright surface.
Nevertheless, if a low-weighted ray results in shading that fires another
low-weighted ray, we can rapidly land in a situation where the net result of
those deeper rays is not visually significant to the final shaded
result.

If we were able to pass to the hit shader the weight by which a given set of rays will be multiplied, that shader might be able to modify its behavior so that it in turn only fires a number of rays that is appropriate for the weight the result will have in the final picture.

Fortunately, PRMan has always allowed you to pass a parameter to the shader
invoked by a ray, using `gather()` and its "`send:surface`" mechanism.
*Un*fortunately, that requires all shaders to be authored to accept the
parameter for the system to be fully effective.

PRman also has a built-in mechanism for trimming ray counts using importance
weighting. The `gather()` call accepts a parameter called `weight`, of
type `varying color`. The default is `color(1)`. When a shader fires
rays using `gather()`, the current importance is multiplied by the weights
passed to `gather()`. This means that the importance down the ray tree is
calculated properly (low weight rays that invoke further low weight rays are
deemed less important than those with higher weights).

If the accumulated importance is at or below
`Attribute "trace" "float importancethreshold"` the `gather()` and
`indirectspecular()` calls only fire some of the relevant rays, but will
give the remaining rays higher weight using the Monte Carlo technique known
as *Russian roulette*.

Here's an example with an extension of the refraction block of code from above:

// Shoot refraction rays if (Kt * kt > 0) { color weight = color(Kt * kt); // NEW LINE HERE ci = 0; hitci = 0; gather("illuminance", P, refrDir, 0, sam, "weight", weight, // NEW LINE HERE "surface:Ci", hitci) { ci += hitci; } Ci += Kt * kt * ci / sam; }

(Similar change in the reflection block.)

No other interaction is required from the shader other than supplying the
weights. Contrast this to using "`send:surface:XYZ`", where the hit shader
would have to check the accumulated importance and multiply it by additional
weights when firing further rays.

The importance cut-off can be tweaked by setting the `importancethreshold`
attribute or passing it as a parameter to `gather()` or
`indirectspecular()`. (As mentioned above, the default value is 0.001.)

Doing this trims the ray tree and prevents it becoming to deep or too bushy in
areas where the overall contribution is not high enough to make a visually
salient difference. We are still, however, presented with a potentially large
number of rays that must be fired because they *might* make an important
contribution to the picture.

#### Going further

One possible approach, rather than relying on the Russian roulette decisions
made by the `importancethreshold` attribute but still leveraging PRMan's
importance sampling support, is to modify the number of samples based on the
computed importance. This can be done like so:

varying float maxImportance = 1; rayinfo("importance", maxImportance); // max of (r,g,b) importance sam = samples * maxImportance;

or via a similar approach. In this way, we can limit the number of samples down the ray tree in a more fine-grained fashion. The number of reflection and refraction rays is still tied together, though, and we are not exploiting that the weight of reflections may be different than the weight of refractions.

We can instead make the number of reflection and refraction rays
independent of each other. Furthermore, we can implement the *Russian
roulette* technique explicitly in the shader (instead of using PRMan's
built-in Russian roulette) to probabilistically decide whether to
shoot a ray or not when the importance is low. Here's a shader
snippet illustrating this:

// Reduce the number of rays when importance is low. Use Russian // roulette when the computed number of rays is between 0 and 1. float maxImportance; rayinfo("importance", maxImportance); // max of (r,g,b) importance float reflSam = Kr * kr * samples * maxImportance; float refrSam = Kt * kt * samples * maxImportance; ... // Shoot refraction rays if (refrSam > RRthreshold) { // shoot 1 or more rays color weight = color(Kt * kt); float sam = max(round(refrSam), 1); ci = 0; hitci = 0; gather("illuminance", P, refrDir, 0, sam, "weight", weight, "surface:Ci", hitci) { ci += hitci; } Ci += Kt * kt * ci / sam; } else if (refrSam/RRthreshold > random()) { // Russian roulette: 0/1 ray ci = 0; hitci = 0; gather("illuminance", P, refrDir, 0, 1, "surface:Ci", hitci) { ci += hitci; } Ci += ci; // no mult by Kt * kt to increase weight of surviving rays }

What we're doing here is determining when the importance dictates that a fractional number of rays be fired, and probabilistically choosing to fire 0 or 1 ray (independently for reflection or refraction). Note that we avoid multiplying by the Kt and kt coefficients because we've effectively weighted by them already in choosing whether to shoot the ray or not.

When using Russian roulette in the surface shaders, the
`importancethreshold` attribute should be set to 0.0.

Here's an example of a Russian roulette glass shader that puts it all together:

surface glassrr(float Kr = 1.0; // coefficient for ideal (mirror) reflection float Kt = 1.0; // coefficient for ideal refraction float ior = 1.5; // index of refraction float Ks = 1.0; // specular reflection coefficient float shinyness = 20.0; // Phong exponent float samples = 4; // number of rays (for antialiasing) float RRthreshold = 0.001) // Russian roulette threshold { color ci, hitci; normal Nn = normalize(N); vector In = normalize(I); normal Nf = faceforward(Nn, In); vector V = -In; // view direction vector reflDir, refrDir; float eta = (In.Nn < 0) ? 1/ior : ior; // relative index of refraction float kr, kt; Ci = 0; // Compute specular highlight if (Ks > 0) { Ci += Ks * specular(Nn, V, 1/shinyness); } // Compute Fresnel reflection and refraction coefficients (kr and kt) // and reflection and refraction directions. If there is total // internal reflection kt is set to 0 and refrDir is set to (0,0,0). fresnel(In, Nf, eta, kr, kt, reflDir, refrDir); kt = 1 - kr; // Reduce the number of rays when importance is low. Use Russian // roulette when the computed number of rays is between 0 and 1. float maxImportance; rayinfo("importance", maxImportance); // max of (r,g,b) importance float reflSam = Kr * kr * samples * maxImportance; float refrSam = Kt * kt * samples * maxImportance; // Shoot reflection rays if (reflSam > RRthreshold) { // shoot 1 or more rays color weight = color(Kr * kr); float sam = max(round(reflSam), 1); ci = 0; hitci = 0; gather("illuminance", P, reflDir, 0, sam, "weight", weight, "surface:Ci", hitci) { ci += hitci; } Ci += Kr * kr * ci / sam; } else if (reflSam/RRthreshold > random()) { // Russian roulette: 0/1 ray ci = 0; hitci = 0; gather("illuminance", P, reflDir, 0, 1, "surface:Ci", hitci) { ci += hitci; } Ci += ci; // no mult by Kr * kr to increase weight of surviving rays } // Shoot refraction rays if (refrSam > RRthreshold) { // shoot 1 or more rays color weight = color(Kt * kt); float sam = max(round(refrSam), 1); ci = 0; hitci = 0; gather("illuminance", P, refrDir, 0, sam, "weight", weight, "surface:Ci", hitci) { ci += hitci; } Ci += Kt * kt * ci / sam; } else if (refrSam/RRthreshold > random()) { // Russian roulette: 0/1 ray ci = 0; hitci = 0; gather("illuminance", P, refrDir, 0, 1, "surface:Ci", hitci) { ci += hitci; } Ci += ci; // no mult by Kt * kt to increase weight of surviving rays } Oi = 1; }

This revamped shader is considerably faster than the original one: it renders the following image using only 8.7 million rays (6 seconds runtime).

This image is nearly identical to the first glass image.

#### Under the hood

How is importance actually implemented "under the hood"?

PRMan takes the parent ray's importance and multiplies it by the weights
provided by the shader in the `gather()` call. `Importance (1,1,1)` is
used in Reyes rendering for directly visible objects that don't have a parent
ray. When using the raytrace hider in PRMan 18, the importance
of each camera ray is set to 1. As of PRMan 19, when using the raytrace hider
the importance of each camera ray will be set to `1/(psx*psy)`, where `psx`
and `psy` are the PixelSamples values.) Weights of (1,1,1) are used if no
weights are provided. If this product is at or below `importancethreshold`
(`0.001` by default) in all three color bands then Russian Roulette is used
to determine whether or not to trace the new ray.

For a ray that survives this "early out", this product is divided by the number of rays. This is the importance assigned to the new ray.

With this calculation, the importance carried with each ray represents how much the color at the ray's hit point will ultimatively contribute to the image.

#### Other uses of the importance function

Importance can be used in other shaders than just glass. For example, if importance is propagated by all shaders in a scene, shaders can check the importance of the result they are about to compute and make approximations if the importance is low: coarser sampling of shadows, blurrier texture lookups, etc.

Potential pitfall: It is important that the switch from accurate to approximate calculation isn't abrupt. For example, it would be wrong for the shader to do full texture lookups if the importance is above a certain threshold, but use a constant color if it is below. Here's why: with such a shader, if the shader that shoots the rays that hit it increases the number of rays that it shoots a little bit, the importance of each ray is decreased a bit, and may suddenly fall below the importance threshold - and the shader will suddenly compute colors that are too different. Instead, the approximations used at low importance must be gradually phased in: the lower the importance, the more approximate the calculation can be.

The observant reader will notice that in PRMan versions prior to 16.2 we actually fell into this pitfall by implementing an automatic "early out" for all rays with low importance . Opting to simply not trace all rays with importance below 0.001 can produce undesirable results. It is more correct to use Russian roulette if the target number of rays is between 0 and 1, as we do in PRMan 16.2 and higher.

#### Importance and radiosity caching

There is one potentially confusing issue with importance and radiosity caching.

When we run the `diffuselighting()` shader method the result is going to
be stored in the radiosity cache. It would be dangerous to use the
importance of the first ray that hits a patch to simplify shading
calculations since another ray with higher importance might later hit
the same patch and reuse the shading result. So importance should not
be used in `diffuselighting()` methods. To safeguard against unintended
mistakes, `rayinfo()` explicitly returns importance 1 when called from
`diffuselighting()`.

In summary: importance is a great source of optimizations, but not in
`diffuselighting()` methods. One should not use importance there.
However, even though importance can't be used to optimize calculations
in `diffuselighting()` methods, one can still use ray depth etc. to
simplify calculations.

### BRDF Importance Sampling

Another use of importance is for importance sampling of the BRDF function.

The surface BRDF will result in some samples from some angles being of lower importance than others. Carefully controlling the number of samples shot using the expected BRDF weight will help keep renders fast. This is at the core of the techniques described in the PRMan application note Physically Plausible Shading in RSL.

### Solid Angle Sample Reduction

The overall weight of an area light on a surface is maximally `2 PI`.
Samples subtend an angle with the surface. Each sample must be weighted such
that its form factor projects appropriately onto the hemisphere. For each
area light sample we should weight that area by `N_light.L`. It may be
helpful to express the maximal number of samples as the number required to
accurately sample the hemisphere. The actual number used may be lower and we
could, for example, use:

samples = maxSamples*areaOfLight/(2*PI)

### Summary

Leveraging the importance mechanism in PRMan can significantly improve the speed of ray-traced renders. When writing shaders to be used with ray tracing, keep in mind the likely number of ray bounces and the ray depths required to produce the quality of picture you need. Especially for shaders like glass, which need higher ray depths to produce production-quality pictures, being aware of the likely explosion of ray counts down the tree is important. Taking care to ensure the budget of rays you fire are spent only on visually important portions of the final result can drastically improve the performance of your renders.

Using the solid angle subtended by an area light to compute the required sampling density can also help control the number of rays/area light samples required, and improve the performance of scenes that employ area lighting.

### More Information

Russian Roulette is a standard method in Monte Carlo simulations in physics and applied mathematics. It was introduced to computer graphics in the following paper:

- James Arvo and David Kirk", "Particle transport and image synthesis", Proc. SIGGRAPH 1990, pp. 63-66.

A general overview of the use of importance for speeding up rendering can be found in this article:

- Per Christensen, "Adjoints and Importance in Rendering: an Overview", IEEE Trans. Visualization and Computer Graphics, 9(3), pp. 329-340, 2003.

Many details of importance sampling and multiple importance sampling are described in this paper:

- Eric Veach and Leonidas Guibas, "Optimally combining sampling techniques for Monte Carlo rendering", Proc. SIGGRAPH 1995, pp. 419-428.