# Running Average Encoding - Why It Works

In the previous post I introduced an as-far-as-I-know novel method for performing progressive least squares optimisation with spherical basis functions. Here, I’ll go into more detail about how it works, and also derive my original, approximate method from the corrected version.

Many thanks to Peter-Pike Sloan for providing the first part of this derivation.

We’ll be dealing with spherical integrals for the sake of this post, but everything is equally applicable to hemispheres by restricting the integration domain. For example:

will be used as shorthand for ‘the integral over the sphere of the function , where is a direction vector. All integrals will be done in respect to .

is taken to mean the value of the function we’re trying to fit in direction ; this value will usually be obtained using Monte Carlo sampling.

We’ll also assuming fixed basis functions parameterised only by their direction, such that is the value of the *i*th basis function in direction . The basis functions will be evaluated by multiplying with a per-basis amplitude and summing, such that the result in direction is given by:

In the case of spherical Gaussians, , and , or the value of the *i*th lobe evaluated in direction .

Our goal is to minimise the squared difference between and so that the fit matches the original function as closely as possible. Mathematically, that can be expressed as:

To minimise, we differentiate the function with respect to each unknown and then set the derivative to 0.

Let . Therefore, for each .

Therefore, by setting ,

At this step, we now have a method for producing a Monte Carlo estimate of the raw moments : as each sample comes in, multiply it by each basis function and add it to the estimate for each lobe. This is in fact what was done for the naïve projection used in The Order: 1886. To reconstruct the lobe amplitudes we need to multiply by the inverse of :

This is a perfectly valid method of performing least squares without storing all of the samples at every step, although it can be noisier than if all samples were used to perform the fit. However, it does require a large matrix multiplication to reconstruct the amplitudes, which is unsuitable for progressive rendering.

In the ‘running average’ algorithm, we want to reconstruct the amplitudes as every sample comes in so that the results can be displayed at every iteration. There are therefore a few more steps we need to perform.

Let’s rearrange the above equation to solve for a single .

We can bring the entire right hand side under the same integral due to the linearity of integration.

Finally, we end up with the following equation for :

The two spherical integrals here which can be computed in tandem using Monte-Carlo integration. The estimate for given a single sample in direction with a value (where is an estimate of ) is given by:

The average value of across all samples will tend towards the true least-squares value.

Likewise, the estimator for is given by averaging .

To solve this for , we need to know the amplitudes for all where . We can approximate this by using the values solved for in the previous iteration of the algorithm. As the number of samples increases, the vector will gradually converge to the true value. The convergence could potentially be improved by seeding the vector with the estimate from a low-sample-count run rather than with the 0 vector; in practice, the error seems to disappear fairly quickly.

Similarly, since is often only an estimator for the function value and not the true value, high variance in its estimate can cause errors in the vector. One possible strategy to counter this is to gradually increase the sample weights over time (e.g. with ); however, in my implementation I haven’t found this to be necessary.

In this running average method, the integral in the denominator is calculated using Monte Carlo integration in the same way that is. In fact, it turns out that computing both of them in lockstep improves the accuracy of the algorithm since any sampling bias in the numerator will be partially balanced out by the bias in the denominator. However, it’s also true that the integral may be wildly inaccurate at small sample counts and end up amplifying small values; therefore, to balance that out, I recommend clamping the estimator for the integral to at least the true integral. Alternatively, it’s possible to always use the precomputed true integral on the denominator and only estimate the vector, although this results in slightly increased error.

My original algorithm was created by experimentation. I thought it would be worth going through why it worked and the approximations it made. Note that none of this is necessary to understand the corrected equation – it’s purely for curiosity and interest!

Effectively, at each step, it solved the following equation:

If we rearrange that to get into a form vaguely resembling our proper solution above:

For the spherical integral of a spherical Gaussian basis function with itself, , since and . Therefore,

This is very close to our ‘correct’ equation above. In fact, it becomes equal when

We can rearrange that a little further:

Since is an estimator for and we assume that, as the fit converges, , we’re left with:

In other words, using the original algorithm for a given sample, the error is mostly determined by how close is to . Since the influence of samples with higher basis weights is greater anyway, this turned out to be a reasonable approximation. However, given the option, I’d still recommend using the corrected algorithm!