Monday, 14 July 2008

Using ATI hardware tessellation in DX9.

Maybe you heard that ATI GPUs starting from 2*** have hardware tessellation unit. Maybe you have seen it working in GPU Mesh Mapper and Render Monkey 1.81. But there isn't any info on how to use it yourself. So with the power of reverse engineering I have finally written my first application that uses hardware tessellation. And I will gladly share the info with you.

To enable tessellator, simply write a few lines of code.
dev->SetRenderState(D3DRS_POINTSIZE, 0x7FA03001);
dev->SetRenderState(D3DRS_MAXTESSELLATIONLEVEL, *(DWORD*)(&floatTesselationFactor)); //from 1.0f to 15.0f

There is however, one issue. If you are changing the primitive type of the geometry, you need to disable tessellation:
dev->SetRenderState(D3DRS_POINTSIZE, 0x7FA03000); //Other values can do the same

And after that, enable it again.

Before you try this out, you also need to change you vertex shader. Now you will get not a single vertex data, but the data of all three vertices of a triangle and barycentric coordinates inside it.

So, if you have something like:
struct VS_INPUT
float4 position: POSITION0,
float2 texcoord: TEXCOORD0

as your vertex shader input, change it to:

struct VS_INPUT
float3 barycentric: BLENDWEIGHT0,

//First vertex
float4 position1: POSITION0,
float2 texcoord1: TEXCOORD0,

//Second vertex
float4 position2: POSITION4,
float2 texcoord2: TEXCOORD4,

//Third vertex
float4 position3: POSITION8,
float2 texcoord3: TEXCOORD8

Now, to find parameters for current vertex, use:

float4 position = position1 * barycentric.x + position2 * barycentric.y + position3 * barycentric.z;
float2 texcoord = texcoord1 * barycentric.x + texcoord2 * barycentric.y + texcoord3 * barycentric.z;

Have fun with it!


SJ said...

Is there anyway you can share your code with me? I'm looking for a full example.

NULL_PTR said...

I've written a simple, but working example. You can get it here: 15kb
It draws 2x2 quad grid, but the result is much more detailed: 21kb

NULL_PTR said...

And one more thing, you can press 'W' to see wireframe, 'N' to set tesselation level to 1.0 (still, every triangle is divided in six pieces).
Also, you can press them both at the same time.

SJ said...

Thanks, but it doesn't seem to work. When I start the program all I see is a black screen.

I have an ATI Radeon 2400 HD Pro which supposedly supports the tessellator. I am running Vista 32-bit on a Quad Core Q6600.

I may get to try it on a 2900 XT tomorrow so I'll let you know how that goes. What care are you using?

NULL_PTR said...

I have ATI 2600XT and 4850 on Vista 64-bit and Catalyst 8.8

Have you tried moving your mouse to rotate camera?

SJ said...

Ok it works on the 2900 XT. It seems like the 2400 HD Pro doesn't support tessellation.

Thanks for the help.

Budi said...

Why not use the official Tessellation SDK?

NULL_PTR said...

This post was written more than a year ago, when there was no official SDK available.

But a part of it about vertex shader input structure and how you average parameters is still true, so this post couldn't be discarded altogether.