Maybe you heard that ATI GPUs starting from 2*** have hardware tessellation unit. Maybe you have seen it working in GPU Mesh Mapper and Render Monkey 1.81. But there isn't any info on how to use it yourself. So with the power of reverse engineering I have finally written my first application that uses hardware tessellation. And I will gladly share the info with you.
To enable tessellator, simply write a few lines of code.
dev->SetRenderState(D3DRS_MAXTESSELLATIONLEVEL, *(DWORD*)(&floatTesselationFactor)); //from 1.0f to 15.0f
There is however, one issue. If you are changing the primitive type of the geometry, you need to disable tessellation:
dev->SetRenderState(D3DRS_POINTSIZE, 0x7FA03000); //Other values can do the same
And after that, enable it again.
Before you try this out, you also need to change you vertex shader. Now you will get not a single vertex data, but the data of all three vertices of a triangle and barycentric coordinates inside it.
So, if you have something like:
float4 position: POSITION0,
float2 texcoord: TEXCOORD0
as your vertex shader input, change it to:
float3 barycentric: BLENDWEIGHT0,
float4 position1: POSITION0,
float2 texcoord1: TEXCOORD0,
float4 position2: POSITION4,
float2 texcoord2: TEXCOORD4,
float4 position3: POSITION8,
float2 texcoord3: TEXCOORD8
Now, to find parameters for current vertex, use:
float4 position = position1 * barycentric.x + position2 * barycentric.y + position3 * barycentric.z;
float2 texcoord = texcoord1 * barycentric.x + texcoord2 * barycentric.y + texcoord3 * barycentric.z;
Have fun with it!