The Mystery of the SM64 Portrait

[Clements]

Super Mario 64 is emulated well by every emulator and practically every plugin. Jabo's, Rice's, glN64() and TR64 do a fantastic job, with glN64() even doing the transport effect.

BUT!!!

One effect in SM64 to my knowledge has never been emulated, and that is the SM64 Bowser/Peach Portrait behind the 1st star door. So I went and did some investigative work with the plugins I have to see what I could come up with.

As you know, in the real game the portrait starts as Peach, but as you get closer the portrait changes to that of Bowser. I have caught the process in three stages with three different plugins:

-1964 OpenGL Graphics build 4.5.0

-Jabo's D3D6 1.5.1

-glN64() [and in actual fact most plugins, see later]

1. The first (top pic) is 1964 OpenGL Graphics build 4.5.0 which clearly shows Peach in the portrait, but it doesn't change to Bowser at all - It is 'fully' Peach.

2. Second up showing the next stage of the effect is Jabo's plugin. It shows a weird mix of both Peach AND Bowser - but no transition between them.

3. Third and finally glN64() [and many others] show the portrait of just Bowser. It is 'fully' Bowser, there is no Peach there.

So, what do you make of this guys? Will this effect be cracked? I've shown the effect in three stages by three different plugins, so how about an author stringing those together somehow? Any thoughts on this? :phone:

[Trotterwatch]

Funnily enough I was checking this effect out a few nights back, and got the same things found out as you. I hope this can be emulated sometime soon - no big deal, just one of those little things that would surely make Mario 100% perfect emulated?

[Clements]

Wow, we think alike! It would be great to see this one solved, and it would indeed be 100% emulated. The effect itself is quite clever and interesting, can't think of a similar effect to it in any other N64 game. It might be a framebuffer effect that no plugin has emulated correctly since it involves a texture changing into another... or it might be something else.

When playing on a real N64 you have no idea how unique effects like this are. That's one of the things I like about N64 emulation. You appreciate the complexity of some of the stuff the N64 can do.

[blight]

I think Hacktarux' LLE graphics plugin can do it ;) :P

[vleespet]

Actually, in my old memories, I remember that I once got it to work properly. I think it was on Corn using an Ati Rage Pro (my geforce didn't work fine). I'm not 100% sure that it was on Corn, but I'm sure I once managed to get it to work in a right way. I hope to post some screenies soon.

[Trotterwatch]

I've seen it before so that it shows Peach and then at the last moment it shows Bowser (when you fall in the hole - I am near certain about that) but the effect definately wasn't shown correctly.

[Knuckles]

I noticed that but I didn't really take care of it.

/me will ask gonetz ;)

[Orkin]

The reason it doesn't work is because SM64 is using the LOD factor as a blending factor between the Peach and Bowser pictures. Kinda like trilinear mip-mapping, but with the same size mip-maps, with different contents.

What makes this hard to emulate, is that there really isn't, AFAIK, a way to get the LOD bias into the blender of the existing PC graphics APIs. Also, AFAIK, you can't specify a mip-map that's the same size as its parent in OGL, or D3D.

It might be possible to emulate this effect using pixel shaders, or maybe by computing the LOD factor on the CPU per-vertex instead of per-pixel. Either way probably wouldn't be easy, but it should be possible.

Orkin

[revl8er]

It's not that big of a deal, it's just one of those things that would make a game 100% working. That's one of those problems like in mario kart with the screen on luigi's raceway, or does it work? I don't know because I never got it to work.

[NeoNight]

[OT] hehe that reminds me of that goemon's adventure game.. there is one lvl in that game that hasen't been emulated. Well the effect thats taking place in the background hasn't. Other then that most (haven't checked rice's latest yet) seems to emulate it damn near perfectly.

[Allnatural]

I recall Jabo mentioning long ago that he had it working, but it was such a nasty hack he removed it.

[Gonetz]

This effect does not work with Glide64 because this effect does not use standart constant LOD factor, which is set by gsDPSetPrimColor, it uses per-pixel calculated LOD, which does not supported by Glide64 and other plugins too. But of course, it can be emulated, and (may be) without using pixel shaders.

[Knuckles]

still the king Gonetz ;)

btw, you should update your system specs :P

[Gonetz]

still the king Gonetz
I did not told that this effect works or will work with Glide64. The screens are just a proof that it could be done.

[The Khan Artist]

Would it be possible to emulate this effect by detecting when two mipmaps are the same size, then upsampling one of them and let trilinear filtering do the rest?

[vleespet]

I'd say that it should be a framebuffer effect with multitexturing, but then I shouldn't be able to see it on Corn in a right way with my Ati rage pro (I'm still 99% sure that it worked), so then it should be a software emulated blending mode.

[Gonetz]

Would it be possible to emulate this effect by detecting when two mipmaps are the same size, then upsampling one of them and let trilinear filtering do the rest?
It's not neccessary. I think it will be enough to

computing the LOD factor on the CPU per-vertex instead of per-pixel.
I made this effect working by calculation LOD per triangle. What is LOD?

level of detail@pixel (texels/pixel), derived per pixel
That is it's changed with triangle's size. When you come closer to the portrait, it becomes larger, LOD becomes smaler. So, you must somehow calculate texels/pixel dependency. I used simple method: texture width/triangle width. I got this effect fully working, but my method is obviously incorrect and non universal, that is it's a hack. Texture area/triangle area must work better, but I'm not sure in which space I should calculate triangle area to get correct result.

BTW, floating LOD is used not only in Mario. It's basic mechanism to implement mip-mapping on N64. Many effects work incorrect because it's not supported.

[Orkin]

I believe it should be done in screen space. You could do it by taking the difference in texture coordinates between vertices in the triangle, and dividing that by the size of the triangle in screen pixels. You would have to take into account the texel/pixel ratio for each edge of the triangle, I guess taking each one and averaging it would work?

Sorta like this:




lodFactor = 0.0f;
for (int i = 0; i < 3; i++)
{
j = (i < 2) ? i + 1 : 0;

deltaS = vertex[j].s - vertex[i].s;
deltaX = vertex[j].x - vertex[i].x;
lodFactor += deltaS / deltaX;

deltaT = vertex[j].t - vertex[i].t;
deltaY = vertex[j].y - vertex[i].y;
lodFactor += deltaT / deltaY;
}

// Divide by 6 (2 directions * 3 edges) to find average
lodFactor = max( lodFactor / 6.0f, 1.0f );



You may also be able to get closer to the original by computing per-vertex insetad of per-triangle. Just average the two edges that use that vertex, instead of all three.

Somewhere in the docs it discusses how this value is computed on the N64, but I never quite fully understood it...

[Orkin]

I just got to thinking...instead of averaging directions, it would be better to get the distance between the texture coords, and the vertices.

More like this:

lodFactor = 0.0f;
for (int i = 0; i < 3; i++)
{
j = (i < 2) ? i + 1 : 0;

deltaS = vertex[j].s - vertex[i].s;
deltaT = vertex[j].t - vertex[i].t;
deltaTexels = sqrt( deltaS * deltaS + deltaT * deltaT );

deltaX = vertex[j].x - vertex[i].x;
deltaY = vertex[j].y - vertex[i].y;
deltaPixels = sqrt( deltaX * deltaX + deltaY * deltaY );

lodFactor += deltaTexels / deltaPixels;
}

// Divide by 3 (3 edges) to find average
lodFactor = max( lodFactor / 3.0f, 1.0f );

[Gonetz]

I've thought about this way. It may work for this effect, but will not work as general mechanism.
Have you read chapter 13.7 "Tile Selection" of the programming manual? LOD is used for mip-mapping, for tile selection. I thought that it's never used in games - most of games use 1-2 tiles. However, it is used in Perfect Dark, end somewhere else for sure. To select correct tile, you need to know exactly, how much texels are used for one pixel. Suppose, we have a quad and 32x32 texture. We far from this quad, and it's screen size is 15.9x15.9 If we will use distances, we will get LOD = 2, but actual LOD is 4 because area of the quad is 4 time smaler then area of the texture. That's why I think that area must be used for LOD calculation. But with the screen size we have a problem. It will work, if the quad is in front of our view. When camera angle will change, quad's area will reduced - LOD will be changed too. Example: if you stop Mario in the middle of the corridor, camera will move and the portrait will change - it's bad since the distance from Mario to the portrait is not changed.
What do you think?

[Orkin]

I've re-read the documentation, and I think I've finally figured it out!

Now, it may at first seem that you need to use area to calculate the LOD, but in fact you don't, I'll use the following as an example:

Say you have mip maps set up like this:
Tile 0: 32x32
Tile 1: 16x16
Tile 2: 8x8
Tile 3: 4x4
Tile 4: 2x2
Tile 5: 1x1

And you want to render a primitive (say a rectangle for now since the math is easier), with the following sizes:
Texture size = 32x32
On-screen size = 8x8

Let's first try calculating the LOD using area:

Texture area = 32 * 32 = 1024
Screen area = 8 * 8 = 64

LOD = Texture area / Screen area = 1024 / 64 = 16

And we select the tile to use based on the LOD:
LOD_tile = log2( (int)LOD ) = log2( 16 ) = 4

If the LOD_tile is 4, then tile 4 is used (assuming prim_tile is 0), but that is only 2x2, not the 8x8 we need.


Now let's try again by calculating the ratio between size, instead of area:
LOD = Texture size / Screen size = 32 / 8 = 4

LOD_tile = log2( (int)LOD ) = log2( 4 ) = 2

Now, with an LOD_tile of 2, we use tile 2 which is 8x8, the size we needed.

The LOD_fraction is computed like this:
LOD_frac = fraction( LOD / 2^LOD_tile ) // With fraction() returning the fractional part of the result

You may at first look at this and think "wait a second, since LOD_tile is log2( LOD ), the LOD_fraction will always end up being 0". Well, it is 0 if the screen size is exactly the same size as the mip map level, but, since the LOD_tile is truncated to an int, if the LOD is fractional, you will end up with a value that gives the amount to blend between the texture described by the tile at prim_tile + lod_tile, and the one described by the tile at prim_tile + lod_tile + 1.

I hope I explained it clearly enough, this is all based on what is stated in the N64 programming manual. There are other things to take into account such as which tiles to use when detail or sharpen modes are enabled. Plus, when sharpen mode is enabled, the LOD_fraction is negated so you end up doing extrapolation instead of interpolation (now that'll be fun to do in a combiner).

Orkin

[Gonetz]

It seems that you have understood this chapter better then I :)
Your explanation looks very logical. It's time to try it :)

[Zilla]

So in the SM64 ROM, is the picture/frame made of one or many polygons/vertexes, each with it's own texture-map?

[/out-my-depth...] :p

[The Khan Artist]

So in the SM64 ROM, is the picture/frame made of one or many polygons/vertexes, each with it's own texture-map?

[/out-my-depth...] :p

*Simplified mode engaged

Normally, each texture has several different sized versions, called mipmaps. Depending on the size of the area the texture will be displayed on, different mipmap levels are used. This was, if you are close to an object, a high-res image will be used, but when you are far from an object, a lower-res version will be used. This saves processing time. If trilinear filtering is enabled, in-between version will be created.

With the Bowser/Peach portrait, instead of two mipmap levels being simply different sizes, they are in fact different pictures. Peach is the higher (lower-res) mipmap level, and Bowser is the lower mipmap level. As you move closer to the portrait, Bowser slowly fades in.

[Shin_Gouki]

Gonetz..Orkin may i ask :D
how is it going, implementaion?
(errh i mean can we expect the effect working on your next plugin-releases?)
did i mentioned that it was awesome to see here quite "live" the growth of an idea :)
i love the net for such possibilities ..its just great ^^
wbr Shin Gouki

[FAR]

I'm not sure but as far as I can remember UltraHle emulated this effect...

[gandalf]

Gonetz..Orkin may i ask :D
how is it going, implementaion?
(errh i mean can we expect the effect working on your next plugin-releases?)
did i mentioned that it was awesome to see here quite "live" the growth of an idea :)
i love the net for such possibilities ..its just great ^^
wbr Shin Gouki

hehehe,i like to see full frame buffers support in both plug-ins!!!

Specially for Ridge Racer 64 (yeah,GLN64 makes perfect emulation of motion blur in this game!!!!!!!,only is missing the board like Mario Kart)

[ggab]

and now with your great Direct64 (where this effect works :-) ),
is posible to implement it, in another plugin, like Rice & Gonetz ones?

PS: if you want, please send the source implementation to them (at least only the SM64 Portrait effect).
Thanks!!!