So, I hit my wall against the head for two days… err my head against the wall, as I couldn’t solve a design problem with shadow mapping. I kept fixing an issue, cleaning what shouldn’t be necessary, then the compiler suddenly complains of yet another problem.

For some reason I’ve always been suspicious of this little bit of code, and now I know why as I discovered a very obscure bug:

// build scene bounding box
const VisibleObjectsBoundsInfo& visInfo = sm->getVisibleObjectsBoundsInfo(texCam);
AxisAlignedBox sceneBB = visInfo.aabb;
AxisAlignedBox receiverAABB = sm->getVisibleObjectsBoundsInfo(cam).receiverAabb;
sceneBB.merge(receiverAABB);
sceneBB.merge(cam->getDerivedPosition());

/* ... some other code here ... */

// calculate the intersection body B
mPointListBodyB.reset();
calculateB(*sm, *cam, *light, sceneBB, receiverAABB, &mPointListBodyB);

This is the code from both Focused and LispSM shadow maps. I’ve never been pleasant by Ogre’s shadow mapping quality (and I don’t know if this has something to do with it) and that bit of code always looked weird to me. Honestly I thought Ogre overestimated or wrongly calculated the aabb of either the receiver’s or caster’s. But that was just a hunch

What looked weird is that why would we merge the aabb from culled casters with the aabb from receivers (I still wonder if that’s right!). Neither Light Space Perspective Shadow Maps nor Robust Shadow Maps for Large Environments seem to talk about it.

However my most current problem was that I hit a wall during the refactor, because sm->getVisibleObjectsBoundsInfo(texCam) was asking for the AABB enclosing all the casters and I haven’t calculated it yet. And then it struct me!!!

The shadow code asks for the aabb enclosing all casters (frustum culled by the shadow’s camera), to set the frustum of the shadow’s camera!

It’s a chicken-and-egg problem, a closed loop, zero, nada!

There’s no solution to this problem. You can’t frustum cull without the frustum, but you need to frustum cull to calculate the frustum. Got it? That little bastard mentally blocked me for 2 days.

But it worked in Ogre 1.x…

Well you’ll see, I always thought it was a miracle that bit of code produced right shadows at all,  but that’s another topic. The math is just too advanced for mere mortals.

Anyway, the thing is that the rendering loop in Ogre works like this:

1. During normal render, an aabb receiver-only entities would be calculated.
2. getShadowCamera -> setup the shadow camera and its frustum
3. Cull all objects. While culling, the shadow built the aabbs made of caster-only entities.
4. Repeat (in the next frame)

Basically, after rendering one frame, getShadowCamera will have something to work with: the results from the previous frame. It’s working with out-of-date data. Step 2 works with the data from step 3 (and step 3 can’t be performed without doing step 2 first). Often that usually works. But it can lead to ugly popping artifacts specially if stuff is moving fast or the camera is. Sometimes more than one frame may be required to converge into the correct result.

In fact, I checked with a debugger and during the first frame and getShadowCamera made an early out because it thought the shadow camera had nothing to render and set a default frustum. Shadows are incorrectly drawn on the first frame. Caught in the act!

The next frame the single cube I was rendering fell into the aabb from that default frustum that was used and getShadowCamera started making the right calculations.

But mere happy coincidence doesn’t change the fact it’s wrong and artifacts may arise. In fact if not all objects fell inside that default frustum, it could theoretically take more than one frame until the shadow mapping camera accommodates to fit them all.

Going for simplicity this time

It was a nice idea to have an AABB to clip the “intersection body B” being built only from casters that are in the frustum. Theoretically it would improve shadow mapping quality because the near and far plane are tighter.

Unfortunately, it’s flawed. Even if it weren’t, it’s very difficult to design because it messes up the rendering sequence. For each light we would need to run a custom rendering sequence:

1. For each light/shadow map:
   1. Calculate rough (conservative) estimate for Frustum
   2. Frustum cull all objects and calculate the Aabb
   3. Put the culled objects list on hold. Now calculate the final Frustum.
   4. Frustum cull again (optional)
   5. Render the objects.

This may look easy, but take in mind this is embedded inside the rendering process as an RTT (render to texture target). The actual rendering sequence would look like this:

1. Frustum cull all objects and calculate the Aabb from receivers.
2. Put the culled objects list on hold. Now start with the shadow map rendering process
3. For each light/shadow map:
   1. Calculate rough (conservative) estimate for Frustum
   2. Frustum cull all objects and calculate the Aabb
   3. Put the culled objects list on hold. Now calculate the final Frustum.
   4. Frustum cull again (optional)
   5. (Optional) repeat steps 2 to 4 N times until it converges well enough
   6. Render the objects.
4. Resume normal rendering process. Render the list of objects calculated in step 2. Don’t forget to setup the materials to use the shadow map bindings from step 3

And I haven’t included the fact that shadows may want to render Render Queues 0 to 4, while the normal rendering process may be rendering in two steps: first RQs 0 to 2, then RQs 3 to 4. That means we have to check in step 3 whether we have the aabb data from all RQs. It can get really nasty.

And I’m a simple guy. I love simplicity in design. So it’s clear now that we should do the same as the source code demo from the original authors: build the aabbs without bloody frustum culling them.

It was a little pointless anyway

If you take a look at the original papers, the reason of these Aabbs is to clip the frustum to avoid very large far and near planes. This is even shown in CascadedShadowMaps11 sample. Ogre tried to go further by trying to make the aabb as small as possible. However even in not-so-complex scenes, this was pointless.

One could argue that achieving “ideal far plane” gives a lot more precision. But if you start thinking in real applications, any scene other than those with a couple objects and very specific lighting directions actually meet this criteria. Most cases there’s always going to be some object that pushes the far plane further until it intersects either the eye’s frustum or the receiver’s aabb.

The whole scene’s caster aabb still helps when the viewer’s frustum is very large, but it’s not supposed to be formed of already culled objects (though we will still cull the receiver’s aabb, as this is simple & easy because it’s done against the viewer’s camera).

Furthermore, even Ogre 1.x did this for quality, take in mind that if an object jumps in and out of the shadow camera’s culling (remember, it was one frame behind, but it still managed work) then the shadow quality is going to flicker. A more conservative approach should produce more stable results.

What do we do now then?

In Ogre 2.0 we still create an AABB based on culled receiver-only objects, because that’s done during the normal pass. The clear separation between culling & rendering makes this easier.

However for the AABB that encloses all casters, we enclose all casters. No frustum culled performed (we still filter by RQs and visbility flags, otherwise we would be doing something the user doesn’t want).

Also now I’ve updated how the body B is calculated, because it didn’t make sense. And the results so far have been really good.