Deferred Shading and Anti-Aliasing

When lots of dynamic lighting is necessary for your project (or just plain cool) then you may look to deferred shading over forward rendering. It has many benefits and some drawbacks that are all well documented. One of the downsides is the lack of hardware anti-aliasing support. There are various solutions to help alleviate the problem.

In my latest game project I wanted to attempt implementing my own anti-aliasing. I chose to use a screen space solution as part of my post processing. The first thing that needs to be done is determine where the edges are that may be aliased. Edge detection algorithms use a filter like Sobel to give weighted values at each pixel to show the likeliness that it lies on an edge. Common data to use as edge indicators are depth and normal of surrounding pixels. Applying the filter will point out discontinuities in these values.

Once the edges are found a blur can be applied based on the weight of the edge value. This is a fairly simple approach that can work quite well. In my project I found a quick and naive implementation was slow and had other downsides. There are a few things to watch out for.

The shader code has many texture lookups and many texture coordinate offsets. These offset texture coordinates should be computed in the vertex shader. Then they will automatically be interpolated and accessed in the pixel shader just like the non-offset texture coordinates to greatly minimize computations. Also, with a consistent filter size you will loose detail as objects move away from the eye. As an object takes up less space on the screen the blur will effect more than just the edges of the object. This can be helped by changing the sample offsets of the blur based on depth, but again this will increase computation that will have to be done per pixel.

This screen shot shows comparisons between aliased and anti-aliased as well as output from my edge detection.

Additional information on deferred shading and anti-aliasing approaches can be found at the following references:
Deferred Shading in STALKER by Oles Shishkovtsov
Deferred Shading Tutorial by Fabio Policarpo and Francisco Fonseca