VFX of Custom Depth Masking


VFX of Custom Depth Masking

A lot of post processing tutorials use the custom depth buffer. One thing that most tutorials don’t show is how to mask off effects using the buffer. A great example of this is when creating an outline shader. If you don’t mask off the object that is on the Custom Depth Buffer then the outline will overlay on top of anything in front of it. In order to keep this from happening we need to mask it off using the scene depth buffer.
Warning: Currently this only works from a set camera distance.

Brainstorming
The biggest issue was figuring out how to tell if there’s an object in front of the one used on the Custom Depth Buffer. I didn't want to put more stress on the pipeline. I looked at what was already given to me and found out a handy way to use the regular depth buffer.
Let's get started!



Materials



In 4 steps objects can be masked off within the Custom Depth buffer. These 4 steps basically check if the scene depth color of the object is darker than the Custom Buffer. If the object's scene depth color is darker then there’s an object in front of it.



  1. InputGet both the Custom and Scene depth and mask out the R channel. One channel is only needed from the depth because every channel holds the same values. This is why the depth channel is a grayscale from 0->255.

  2. Depth Detail - In order to utilize the depth buffer it needs to be divided by a large number (or multiply by a very small number). This is done because the returned value isn’t precise enough to tell the difference between objects on the depth buffer.



Dividing the regular depth by 1000 and the custom depth by 10000000 gives enough detail to work with.


  1. Masking - Saturate the value returned from the custom depth value, which gives a strong silhouette of the object. Then invert the value to use as a mask. The inverted value is then multiplied by the scene depth. What's left is the mask of an object on the custom depth buffer within the scene buffer.
  1. Threshold Check - Plug the masked value into an If statement. If the masked pixel is greater than the OutlineDepthThreshold (in this case .08) then that tells there’s nothing in front of it. If the value is less than .08 then there’s something in front of it.




DONE!





Final Thoughts

Things I would do different….
  • The masking threshold feels a little more hardcoded than I’d like. Instead, I’d come up with a threshold that utilizes the camera far plane, distance, and angle relative to the player.

Comments

Popular Posts