The Mystery of the SM64 Portrait

[Clements]

Super Mario 64 is emulated well by every emulator and practically every plugin. Jabo's, Rice's, glN64() and TR64 do a fantastic job, with glN64() even doing the transport effect.

BUT!!!

One effect in SM64 to my knowledge has never been emulated, and that is the SM64 Bowser/Peach Portrait behind the 1st star door. So I went and did some investigative work with the plugins I have to see what I could come up with.

As you know, in the real game the portrait starts as Peach, but as you get closer the portrait changes to that of Bowser. I have caught the process in three stages with three different plugins:

-1964 OpenGL Graphics build 4.5.0

-Jabo's D3D6 1.5.1

-glN64() [and in actual fact most plugins, see later]

1. The first (top pic) is 1964 OpenGL Graphics build 4.5.0 which clearly shows Peach in the portrait, but it doesn't change to Bowser at all - It is 'fully' Peach.

2. Second up showing the next stage of the effect is Jabo's plugin. It shows a weird mix of both Peach AND Bowser - but no transition between them.

3. Third and finally glN64() [and many others] show the portrait of just Bowser. It is 'fully' Bowser, there is no Peach there.

So, what do you make of this guys? Will this effect be cracked? I've shown the effect in three stages by three different plugins, so how about an author stringing those together somehow? Any thoughts on this? hone:

 

[Trotterwatch]

Funnily enough I was checking this effect out a few nights back, and got the same things found out as you. I hope this can be emulated sometime soon - no big deal, just one of those little things that would surely make Mario 100% perfect emulated?

 

[Clements]

Wow, we think alike! It would be great to see this one solved, and it would indeed be 100% emulated. The effect itself is quite clever and interesting, can't think of a similar effect to it in any other N64 game. It might be a framebuffer effect that no plugin has emulated correctly since it involves a texture changing into another... or it might be something else.

When playing on a real N64 you have no idea how unique effects like this are. That's one of the things I like about N64 emulation. You appreciate the complexity of some of the stuff the N64 can do.

 

[blight]

I think Hacktarux' LLE graphics plugin can do it

 

[vleespet]

Actually, in my old memories, I remember that I once got it to work properly. I think it was on Corn using an Ati Rage Pro (my geforce didn't work fine). I'm not 100% sure that it was on Corn, but I'm sure I once managed to get it to work in a right way. I hope to post some screenies soon.

 

[Trotterwatch]

I've seen it before so that it shows Peach and then at the last moment it shows Bowser (when you fall in the hole - I am near certain about that) but the effect definately wasn't shown correctly.

 

[Knuckles]

I noticed that but I didn't really take care of it.

/me will ask gonetz

 

[Orkin]

The reason it doesn't work is because SM64 is using the LOD factor as a blending factor between the Peach and Bowser pictures. Kinda like trilinear mip-mapping, but with the same size mip-maps, with different contents.

What makes this hard to emulate, is that there really isn't, AFAIK, a way to get the LOD bias into the blender of the existing PC graphics APIs. Also, AFAIK, you can't specify a mip-map that's the same size as its parent in OGL, or D3D.

It might be possible to emulate this effect using pixel shaders, or maybe by computing the LOD factor on the CPU per-vertex instead of per-pixel. Either way probably wouldn't be easy, but it should be possible.

Orkin

 

[revl8er]

It's not that big of a deal, it's just one of those things that would make a game 100% working. That's one of those problems like in mario kart with the screen on luigi's raceway, or does it work? I don't know because I never got it to work.

 

[NeoNight]

[OT] hehe that reminds me of that goemon's adventure game.. there is one lvl in that game that hasen't been emulated. Well the effect thats taking place in the background hasn't. Other then that most (haven't checked rice's latest yet) seems to emulate it damn near perfectly.

 

[Allnatural]

I recall Jabo mentioning long ago that he had it working, but it was such a nasty hack he removed it.

 

[Gonetz]

This effect does not work with Glide64 because this effect does not use standart constant LOD factor, which is set by gsDPSetPrimColor, it uses per-pixel calculated LOD, which does not supported by Glide64 and other plugins too. But of course, it can be emulated, and (may be) without using pixel shaders.

 

[Knuckles]

still the king Gonetz

btw, you should update your system specs

 

[Gonetz]

still the king Gonetz

I did not told that this effect works or will work with Glide64. The screens are just a proof that it could be done.

 

[The Khan Artist]

Would it be possible to emulate this effect by detecting when two mipmaps are the same size, then upsampling one of them and let trilinear filtering do the rest?

 

[vleespet]

I'd say that it should be a framebuffer effect with multitexturing, but then I shouldn't be able to see it on Corn in a right way with my Ati rage pro (I'm still 99% sure that it worked), so then it should be a software emulated blending mode.

 

[Gonetz]

Would it be possible to emulate this effect by detecting when two mipmaps are the same size, then upsampling one of them and let trilinear filtering do the rest?

It's not neccessary. I think it will be enough to

computing the LOD factor on the CPU per-vertex instead of per-pixel.

I made this effect working by calculation LOD per triangle. What is LOD?

level of detail@pixel (texels/pixel), derived per pixel

That is it's changed with triangle's size. When you come closer to the portrait, it becomes larger, LOD becomes smaler. So, you must somehow calculate texels/pixel dependency. I used simple method: texture width/triangle width. I got this effect fully working, but my method is obviously incorrect and non universal, that is it's a hack. Texture area/triangle area must work better, but I'm not sure in which space I should calculate triangle area to get correct result.

BTW, floating LOD is used not only in Mario. It's basic mechanism to implement mip-mapping on N64. Many effects work incorrect because it's not supported.

 

[Orkin]

I believe it should be done in screen space. You could do it by taking the difference in texture coordinates between vertices in the triangle, and dividing that by the size of the triangle in screen pixels. You would have to take into account the texel/pixel ratio for each edge of the triangle, I guess taking each one and averaging it would work?

Sorta like this:

lodFactor = 0.0f;
for (int i = 0; i < 3; i++)
{
    j = (i < 2) ? i + 1 : 0;

    deltaS = vertex[j].s - vertex[i].s;
    deltaX = vertex[j].x - vertex[i].x;
    lodFactor += deltaS / deltaX;

    deltaT = vertex[j].t - vertex[i].t;
    deltaY = vertex[j].y - vertex[i].y;
    lodFactor += deltaT / deltaY;
}

// Divide by 6 (2 directions * 3 edges) to find average
lodFactor = max( lodFactor / 6.0f, 1.0f );

You may also be able to get closer to the original by computing per-vertex insetad of per-triangle. Just average the two edges that use that vertex, instead of all three.

Somewhere in the docs it discusses how this value is computed on the N64, but I never quite fully understood it...

 

[Orkin]

I just got to thinking...instead of averaging directions, it would be better to get the distance between the texture coords, and the vertices.

More like this:

lodFactor = 0.0f;
for (int i = 0; i < 3; i++)
{
j = (i < 2) ? i + 1 : 0;

deltaS = vertex[j].s - vertex[i].s;
deltaT = vertex[j].t - vertex[i].t;
deltaTexels = sqrt( deltaS * deltaS + deltaT * deltaT );

deltaX = vertex[j].x - vertex[i].x;
deltaY = vertex[j].y - vertex[i].y;
deltaPixels = sqrt( deltaX * deltaX + deltaY * deltaY );

lodFactor += deltaTexels / deltaPixels;
}

// Divide by 3 (3 edges) to find average
lodFactor = max( lodFactor / 3.0f, 1.0f );

 

[Gonetz]

I've thought about this way. It may work for this effect, but will not work as general mechanism.
Have you read chapter 13.7 "Tile Selection" of the programming manual? LOD is used for mip-mapping, for tile selection. I thought that it's never used in games - most of games use 1-2 tiles. However, it is used in Perfect Dark, end somewhere else for sure. To select correct tile, you need to know exactly, how much texels are used for one pixel. Suppose, we have a quad and 32x32 texture. We far from this quad, and it's screen size is 15.9x15.9 If we will use distances, we will get LOD = 2, but actual LOD is 4 because area of the quad is 4 time smaler then area of the texture. That's why I think that area must be used for LOD calculation. But with the screen size we have a problem. It will work, if the quad is in front of our view. When camera angle will change, quad's area will reduced - LOD will be changed too. Example: if you stop Mario in the middle of the corridor, camera will move and the portrait will change - it's bad since the distance from Mario to the portrait is not changed.
What do you think?