Need help with fog emulation

I'm trying to implement fog, using fog coordinate instead of Q.
I use linear fog table, created as guFogGenerateLinear (rdp.fog, 0, fog_multiplier + fog_offset),
where fog_multiplier and fog_offset are values set by moveword command. fog_offset is negative, and fog_multiplier + fog_offset ~ 255
Fog coordinate is calculated as follow:
v.fog = 1.0f/max(0.01f, v.z/v.w* fog_multiplier + fog_offset), where v.z and v.w are coordinates in homogeneous space.
I’ve got correct fog thickness with this method, but I have one serious problem. When values of fog_multiplier and fog_offset are large (~20000), fog is not smooth. Look at the screen shots. First screen, fog is quite correct. Second screen – I moved camera to the right just a bit, and got right wall fogged wrongly. It happens because 2 0f 3 vertices have fog coordinate set to minimal. When fog_multiplier and fog_offset are large, v.z/v.w must be very close to 1 to (v.z/v.w* fog_multiplier + fog_offset) be positive. Vertices had fog coordinate ~100, camera was moved, and fog coordinates became minimal -> fog became discontinuous. I don’t know, how to fix it. Afaik, other plugins use similar methods for fog emulation, why only I have this problem?

Orkin said:

Try taking out the "1.0f /", I don't have it in mine, and it seems to work well.

Click to expand...

It will not work. v.fog is used instead of v.q in the fog equation, and q = 1/w

If you're doing it in homogeneous space, you need to do it either after clipping, or make sure the vertex isn't outside the near clipping plane (z >= -w), otherwise you'll get incorrect results (took me awhile to track down this problem in glN64).

Click to expand...

I have this problem even when all vertices are inside (as wrong polygons on my second screen shot). May be fog coordinate must be clipped in some special way? I have tried to assign fog as z/w before clipping and use clipped value in the equation. It does not work well too. I have tried to clip z (not z/w) and w separately, and calculate fog after clipping - result is awful. Is it possible to see in your plugin the fog coordinate before and after clipping? It could help me to understand, what happens here.

Orkin, let me be sure that I understand correctly your algorithm:

0) Create linear fog table. FogStart = 0, FogEnd = 255

1) load modeling coordinate vertices (x, y, z)

2) multiply them by modelview and projection matrices and get (x, y, z, w) coordinates in homogeneous space.

3) Calculate fog coordinate as:

if (z < -w)
v.fog = 0;
else
v.fog = max(0, z/w * fog_multiplier + fog_offset);

Since z/w is in range [-1, 1], fog is in range [0, fog_multiplier + fog_offset]

4) clip triangles

5) draw triangles

If you do it exactly as this, well, it’s probably does not correctly supported by glide