Introduction to the problem :
-The first thing that any graphics programmer will start cursing is the depth buffer, and this is because of it's precision and the leak of understanding on how it works.
-Setting the bit depth is a straight forward process,select the biggest depth buffer available that feats in your video memory plane.
-Creating the projection matrix,specifically setting the near and far planes is a little more challenging,because the leak of math education + some leek of knowledge on how computer works makes a noob programmer to set the near planes at 0.000001f and far planes at 9999999,this is a bad thing,because the z-buffer advertises as something that it is not; when you are setting the near and far plane you are not just setting your convenient frustum but also it's compression, I will talk more about this later.
-The other element that is hard for beginners to understand is that in computer programming like in any real application made in this universe the real number is a pipe dream,floats and double look like they have a place in
continues mathematics but they are just convinces over
discreet mathematics.
The core of the problem :
-First let's look at the bit depth and it's meaning, depth buffer come in many flavors, in recent video cards the selection was reduced but we steal have a few :
+15bit, that's 32768 discrete values(as in the 16bit depth buffer with 1 bit stencil).
+16bit, that's 65536 discrete values(as in the standard 16bit depth buffer and the 24bit depth buffer with 8bit stencil).
+24bit, that's 16777216 discrete values(as in the standard 24bit depth buffer and the 32bit depth buffer with 8bit stencil).
+32bit, that's 4294967296 discrete values.
When you see this values you will probably say yawn that's a lot of discreet values,but there is another catch, the depth buffer DOES NOT HAVE A LINEAR DISTRIBUTION, almost all the values are in the first 10% of the interval, it has almost a hyperbolic distribution.
Lets see some examples :
-For a 32 bit depth with the z-near at 0.1f and the z-far at 100 we will have
at z-buffer value error
0.25 2579559936 0.00000000014537363
0.2505 2582992440 0.000000000145955692
0.26 2645702437 0.000000000157236113
1.00 3869339904 2.325978E-9
4.00 4191784896 0.00000003721565
10.00 4256273894 0.000000232597813
20.00 4277770227 0.0000009303912
80.00 4293892476 0.0000148862573
90.00 4294489597 0.00001884042
99.00 4294923866 0.0000227969067
99.0005 4294923888 0.0000227971377
100.00 4294967293 0.0000232597758
Here is a graph representing the error as it grows (at the bottom of the page there is a link for a depth error calculator that I made):
Can you spot the problem, the difference between 0.25 and 0.2505 is of 0.0005 units in the 3d world but the difference in the depth buffer is of 3432504 units, now let's see
between 99 and 99.0005 in 3d space logic dictates that it will also be 0.0005 units but in the z-buffer the difference is only of 22.
-For a 16 bit depth buffer the situation is a lot more tragic :
at z-buffer value error
0.25 39360 0.000009526843
0.2505 39413 0.00000956498752
99 65535 1.47180724
99.0005 65535 1.471822
-For 15 bits I won't even show you the numbers(but you can calculate them yourself with the calculator that you can download at the bottom of this page):
As you can see near the z-far we can't even distinguish values.
And I hope you noticed that at 0.25 we stored 39360 and our full precision is 65563 more then half of our precision is in the first 0.25 units of our depth !!!!
Why we have the problem
-After the projection matrix transforms the coordinates, the XYZ-vertex values are divided by their clip coordinate W value, which results in normalized device coordinates. This step is known as the perspective divide. The clip coordinate W value represents the distance from the eye. As the distance from the eye
increases, 1/W approaches 0. Therefore, X/W and Y/W also approach zero, causing the rendered primitives
to occupy less screen space and appear smaller.This is how computers simulate a perspective view.
As part of the perspective divide, Z is also divided by W. For objects that are already close to the back of the view volume, a change in distance of one coordinate unit has less impact on Z/W than if the object is near the front of the view volume.What you should realize is that because of this the z is not linear in precision and it's precision is proportional with the reciprocal of (zFar / (zFar - zNear) + zFar * zNear / (zNear - zFar)) / z as a result we have a but load of precision near the camera and almost no precision at the far plane.The perspective divide, by its nature, causes more Z precision close to the front of the view volume than near the back.Half of the resolution of the z-buffer is "wasted"(read used) for the first 5% of the interval between the near and far planes.
This is not such a bad thing if you think about it,because you need more precision where the action lies near the camera, but this is something that you have to understand and work in your design.
-The depth buffer compression, when the depth is stored, it must be feted in 15,16 or 32 bits so :
int storeZ = Math.Floor((1 << resolution) * depth);
where the resolution is how many bits we have to spare for the precision.
-In Directx you don't have direct access to the depth-buffer as a result video card manufacturers use unicorn magic in the compression algorithms because of this the precision may be slightly smaller then you expect or the distribution may have a different shape then you expect on some devices so always use a hysteresis.
Possible solutions
-Understand that there is no magic bullet here(some people that work in the game industry still think there is)
-You can't have a depth buffer that will draw with high precision objects near the camera and objects far away,you must compromise.
-For huge distances you can use a logarithmic depth buffer, a technology made by the SGI people, where basically you want to simulate a logarithmic curve in your depth buffer, this is a god technique as it spreads the precision a little bit better(I will have a post about it),as an side-effect of the logarithmic evolution you will start having problems with the polygons that are near the camera especially if you have big polygons.
-The best thing you can do is to keep you near plane as far as possible from the camera and your far plane as close as possible, this is because the z-buffer distribution is almost hyperbolic, as a result if you compress it, it will become more linear, example of the error curve :
For 16bit depth buffer from 0.1 to 100
For 16bit depth buffer from 0.5 to 100
As you can see the error was reduced dramatically just by taking the near plane closer to the camera.
Remember
-The clip coordinate W after perspective transform represents the distance from the eye.
-The clip coordinate Z represents the distance from the eye corrected by the perspective transform.
-The z-buffer contrary to it's name doesn't hold the z value but the z/w value.
-Near plane as far as possible without compromising the quality of your game.
-Far plane as close as possible without compromising the quality of your game.
-The z-buffer is discreet, you don't have an unlimited number of values, it can only grow with fixed increments of approximately 1/(2^number_of_bits - 1).
-Try not to use stencil buffers, every bit counts.
-If you need to render depth in to a texture don't save the z component save the z/w because in the depth buffer we need normalized device coordinates, that's coordinates that can be feted in to axis aligned bounding box with the min in [-1,-1,0] and max in [1,1,1],also don't do the division operation in the vertex shader, do it in the pixel shader because z and w don't change proportionally, if you don't have a very tessellated mesh you will encounter problems.
-Perspectiv ecuations :
z' = z*(zf/zf-zn)-((zn*zf)/zf-zn)
w' = z
where z',w' are clip space coordinates, z is a camera space coordinate,zn is the near plane and zf is the far plane.
-In Directx you don't have direct access to the depth-buffer as a result video card manufacturers use unicorn magic in the compression algorithms because of this the precision may be slightly smaller then you expect or the distribution may have a different shape then you expect so always use a hysteresis.
-As a bonus because you used your precious time to read this article I uploaded a depth-buffer error calculator so you can calculate your near and far planes with more ease :
It's pretty straight forward to use, simply type in your near plane, your far plane, the biggest error that you are interested in and your bit depth and you will get a graph that you can sample presenting the error as you approach the far plane.
Here is the download link it's on Microsoft SkyDrive so if you have an account and you are not logged you may not see it, it was written in c# .net 4.0 and you will need at least .net 4.0 client framework installed, you also have the source there :
If you want to know more about the history of the depth buffer, see
this article on Wikipedia and know that the guy who invented it was
Edwin Catmull, the guy who also invented the Catmul-Rom spline and the Catmul-Clark subdivision also known as Mesh-Smooth,Sub-D or NURMS.
Thanks for reading,hope it was a useful experience for you, leave a comment if you think something is not write, or if you enjoyed it and good coding :)
