N64 Texture Image Extraction
- Thread starter BGNG
- Start date
OP
BGNG
New member
- Thread Starter
- #62
I am well familiar with both.
If you don't feel like converting the code to C++, I'll do that for you. I was just making sure you didn't have any unjust bias against VB like so many do. I personally know quite a number of programming languages including variations of C, BASIC, Java, and Assembly. BASIC is probably the easiest to use, and it's adequate for most of my purposes, so I use that one the most.
__________
For videogame programming, each resource has exactly one (1) function. In the case of these images, they are used solely as textures for one part of the game. So if you were the programmer, it should be fairly obvious that you know exactly where you'll be using said images; meaning you'll also know offhand the dimensions and pixel depth of each of them.
F-Zero X is no exception to that rule. Each image's dimensions are hard-coded into the game since the images are raw binary data. It's an old-fashioned technique, but it's nonetheless faster than deriving the dimensions from the resource itself.
Like I mentioned in my previous post, these are the actual offsets of the track texture images in the F-Zero X ROM:
All textures for any given track set are included in one file. The pixels for these images are stored left-to-right, top-to-bottom in 16-bit RGBA format. These particular resources have 10 images at 128×64 pixels, and 1 image at 256×64 pixels. In the order that they appear in the file: there are 7 128×64 images, then the 256×64 image, then three more 128×64 images.
The 256×64 image, for example, is the texture for track that has high walls.
If you don't feel like converting the code to C++, I'll do that for you. I was just making sure you didn't have any unjust bias against VB like so many do. I personally know quite a number of programming languages including variations of C, BASIC, Java, and Assembly. BASIC is probably the easiest to use, and it's adequate for most of my purposes, so I use that one the most.
__________
For videogame programming, each resource has exactly one (1) function. In the case of these images, they are used solely as textures for one part of the game. So if you were the programmer, it should be fairly obvious that you know exactly where you'll be using said images; meaning you'll also know offhand the dimensions and pixel depth of each of them.
F-Zero X is no exception to that rule. Each image's dimensions are hard-coded into the game since the images are raw binary data. It's an old-fashioned technique, but it's nonetheless faster than deriving the dimensions from the resource itself.
Like I mentioned in my previous post, these are the actual offsets of the track texture images in the F-Zero X ROM:
Code:
0x235130 //Mute City
0x239A80 //Port Town
0x23EC50 //Big Blue
0x243D90 //Sand Ocean
0x24A270 //Devil's Forest
0x2507F0 //White Land
0x255100 //Sector
0x259600 //Red Canyon
0x25F360 //Fire Field
0x266C20 //Silence
0x26D780 //EndingThe 256×64 image, for example, is the texture for track that has high walls.
Last edited:
I'm getting less data when I decompress than that amount.
0xBFC0 is decompressed size using M0CK of at least two level sets (only tried 2).
128 x 64 = 0x2000 in hex. Two bytes per pixel, = 0x4000
256 x 64 = 0x4000 in hex. Two bytes per pixel, = 0x8000
0xBFC0 is what M0CK gives me
Anything I'm doing wrong?
Beginning ones look like 64 x 32...decompress identically to your before.
Using that hypothesis, and seeing your 128 x 16, works great.
Ahh I see, you gave me the dimensions in bytes, which is counter-intuitive a bit. Got it. One more image after this, just didn't fit into 10 attachment limit.
OK now time to work on the decompressor and then finally compressor now that I can test my results.
Next edit: My decompressor implemented...work on compressor starting, after lunch.
0xBFC0 is decompressed size using M0CK of at least two level sets (only tried 2).
128 x 64 = 0x2000 in hex. Two bytes per pixel, = 0x4000
256 x 64 = 0x4000 in hex. Two bytes per pixel, = 0x8000
0xBFC0 is what M0CK gives me
Anything I'm doing wrong?
Beginning ones look like 64 x 32...decompress identically to your before.
Using that hypothesis, and seeing your 128 x 16, works great.
Ahh I see, you gave me the dimensions in bytes, which is counter-intuitive a bit. Got it. One more image after this, just didn't fit into 10 attachment limit.
OK now time to work on the decompressor and then finally compressor now that I can test my results.
Next edit: My decompressor implemented...work on compressor starting, after lunch.
Last edited:
OP
BGNG
New member
- Thread Starter
- #64
Woah. Talk about a memory lapse... Yeah, I was all wrong with those figures. The images are of 64×32 pixels (times 10) and 128×16 pixels (times one)... I have no idea where I got those other numbers. (I'm not as think as you confused I am)
Also, at the end of those files, you'll notice miniature versions (I think 16×8 pixels) of the larger images. Those are used in the Edit Course feature, which I believe to be hard-coded into F-Zero X, but cannot be accessed without the F-Zero X Expansion Kit via N64DD.
Edit:
I have documentation on the MIPS Assembly opcodes (binary data and execution) for the R4300 RISC processor that the Nintendo 64 uses. If you would like said documentation, you would probably be able to write a disassembler and crack open a few games (like Mario Kart 64) and take the compressor code out of there...
...if you feel like it.
Also, at the end of those files, you'll notice miniature versions (I think 16×8 pixels) of the larger images. Those are used in the Edit Course feature, which I believe to be hard-coded into F-Zero X, but cannot be accessed without the F-Zero X Expansion Kit via N64DD.
Edit:
I have documentation on the MIPS Assembly opcodes (binary data and execution) for the R4300 RISC processor that the Nintendo 64 uses. If you would like said documentation, you would probably be able to write a disassembler and crack open a few games (like Mario Kart 64) and take the compressor code out of there...
...if you feel like it.
Last edited:
Compression works nicely.
Compression works nicely. I don't know why yours isn't working. I haven't tried putting in a user image, but this recompresses the textures from big blue, and I put them into the fire level. I guess I had a lot of experience with the ridiculous amounts of hours I put into successfully getting Goldeneye's compression. As to your question, I have the documentation on MIPS; I'm quite an experienced gameshark hacker. I have all the tools I need, Nemu is great for breakpointing and was instrumental in some of the Goldeneye stuff.
Since this is one chunk of data, have to write some stuff that will properly allow insertion of user images. I had no idea how simple F-Zero was, barely uses any textures at all.
Oh, and my compression was about 0x200 worse than theirs, not a huge deal, and especially since user images you can turn the colors down and you'll be fine.
Compression works nicely. I don't know why yours isn't working. I haven't tried putting in a user image, but this recompresses the textures from big blue, and I put them into the fire level. I guess I had a lot of experience with the ridiculous amounts of hours I put into successfully getting Goldeneye's compression. As to your question, I have the documentation on MIPS; I'm quite an experienced gameshark hacker. I have all the tools I need, Nemu is great for breakpointing and was instrumental in some of the Goldeneye stuff.
Since this is one chunk of data, have to write some stuff that will properly allow insertion of user images. I had no idea how simple F-Zero was, barely uses any textures at all.
Oh, and my compression was about 0x200 worse than theirs, not a huge deal, and especially since user images you can turn the colors down and you'll be fine.
OP
BGNG
New member
- Thread Starter
- #66
Wow... A delightful conversation... I don't get too many of these.
Well, that's the same problem I was having; my compressor was generating larger files than those used in the ROM. That means that (obviously) it wasn't compressing as well as it could. To this day I can't figure out what Nintendo did differently, but it works better. It drove all the sense out of me, so I have since given up.
__________
You've probably noticed in your hacking up there that even though the track textures were Big Blue, the bottom (the lava stuff) was still Fire Field. The images for the bottoms of the worlds are NOT MIO0 data. They are uncompressed RGBA starting at offset 0x004CFE50 in the ROM... or it might be 0x004CFE4E... Tile Molester's offset display is confusing. It's just a few bytes from those locations if it isn't either of them.
Also... Like I mentioned in my very first post in this thread, I've completely cracked every byte of the course data resources in the ROM. I've documented the file format and I've attatched the specification to this post.
Look through that document a bit. The course data starts at offset 0x002AD1E0 in the ROM, and all courses are exactly the same size. They are all in order, from Mute City - Figure 8 through Big Hand - Dangerous Curves. There are two more after Big Hand: Death Race and Ending, in that order.
After looking through that document, you should be able to see that changing byte 0x002B3062 (Z64 format) from 08 to 02 will change the internal track set of the Fire Field course from the Fire Field textures to the Big Blue textures.
Well, that's the same problem I was having; my compressor was generating larger files than those used in the ROM. That means that (obviously) it wasn't compressing as well as it could. To this day I can't figure out what Nintendo did differently, but it works better. It drove all the sense out of me, so I have since given up.
__________
You've probably noticed in your hacking up there that even though the track textures were Big Blue, the bottom (the lava stuff) was still Fire Field. The images for the bottoms of the worlds are NOT MIO0 data. They are uncompressed RGBA starting at offset 0x004CFE50 in the ROM... or it might be 0x004CFE4E... Tile Molester's offset display is confusing. It's just a few bytes from those locations if it isn't either of them.
Also... Like I mentioned in my very first post in this thread, I've completely cracked every byte of the course data resources in the ROM. I've documented the file format and I've attatched the specification to this post.
Look through that document a bit. The course data starts at offset 0x002AD1E0 in the ROM, and all courses are exactly the same size. They are all in order, from Mute City - Figure 8 through Big Hand - Dangerous Curves. There are two more after Big Hand: Death Race and Ending, in that order.
After looking through that document, you should be able to see that changing byte 0x002B3062 (Z64 format) from 08 to 02 will change the internal track set of the Fire Field course from the Fire Field textures to the Big Blue textures.
Took a look at the file, looks like you've mapped it out nicely. I still can't get over the shock of how simple it is, the whole thing, compared to other games. This is your project though, I just wanted to get MIO0 going, still have to just finish up the little compression thing, but that's it, then back to my other projects. The hitch with these textures is that they're in the sets of 11+, as you found, which is a huge nuisance for editing textures, especially since our compression algorithms are not as efficient. When they're individual, it's a lot better.
Just for reference, the MIO0 in other games, such as mario or mario kart, are they similar in that they use a lot of textures in one bank?
It would be cool to see a track editor though. Heh, I never really played this game much back in the day, it's kind of funny how every level is essentially the same, just recolor the texture bank and change the level a bit.
Just for reference, the MIO0 in other games, such as mario or mario kart, are they similar in that they use a lot of textures in one bank?
It would be cool to see a track editor though. Heh, I never really played this game much back in the day, it's kind of funny how every level is essentially the same, just recolor the texture bank and change the level a bit.
OP
BGNG
New member
- Thread Starter
- #68
Actually, there's a bit of an update you may want to know about. gottaX (Anton's affiliate) actually DID reply to me the other day. One of the members (SHEDEVR) in his crew was able to create a "flexible parsing" functionality for my MIO0 compression code. He has succeeded in making a compressor which makes files SMALLER than those seen in Nintendo's ROMs. So whatever Nintendo did to make a compressor better than mine, SHEDEVR did it better than them.
I have not been able to obtain the program he created, however. I will keep on that and keep you updated.
__________
I know in Mario Kart 64 that the ghost saves for a memory pak are stored as compressed MIO0 data. So I know that Mario Kart 64 has a compression algorithm in it somewhere. As for course textures or what-not, I really don't know. But chances are, they are individual. F-Zero X is the only place I've seen "libraries" of textures in that manner.
I have not been able to obtain the program he created, however. I will keep on that and keep you updated.
__________
I know in Mario Kart 64 that the ghost saves for a memory pak are stored as compressed MIO0 data. So I know that Mario Kart 64 has a compression algorithm in it somewhere. As for course textures or what-not, I really don't know. But chances are, they are individual. F-Zero X is the only place I've seen "libraries" of textures in that manner.
I have some corrections 
My nickname is not SHEDEVR Here you can find source and binary files for encoder/decoder... Since previous version I've improved compression ratio for some bytes more. But sources has been used for some research, it have some unnecessary code and rectilinear algorithm... It need some improvements for faster packing like binary- or suffix three applying. Enjoy
http://cah4e3.deruspsx.com/upload/mio0pack.rar
My nickname is not SHEDEVR Here you can find source and binary files for encoder/decoder... Since previous version I've improved compression ratio for some bytes more. But sources has been used for some research, it have some unnecessary code and rectilinear algorithm... It need some improvements for faster packing like binary- or suffix three applying. Enjoy http://cah4e3.deruspsx.com/upload/mio0pack.rar
CaH4e3 said:I have some corrections
http://cah4e3.deruspsx.com/upload/mio0pack.rar
Do you mind explaining how your algorithm for LZ encoding works?
HyperHacker
Raving Lunatic
o_o
Sorry to bring up an old thread like this, but wow. I just coded one of these myself a few days ago, and got it so close to having as small output as Nintendo's... (Source code here; includes some info on Mario Kart's compression code.) How did you get it to work? I figured out most of the optimization but I couldn't get that one last part right...
Sorry to bring up an old thread like this, but wow. I just coded one of these myself a few days ago, and got it so close to having as small output as Nintendo's... (Source code here; includes some info on Mario Kart's compression code.) How did you get it to work? I figured out most of the optimization but I couldn't get that one last part right...
HyperHacker
Raving Lunatic
Hmm, I looked at the code, didn't help much without any form of comments or documentation. <_< It actually doesn't seem to be compressing as well as Nintendo's in all cases, though. For example, a file at 0x641F70 in Mario Kart 64 (US) is compressed to 908 bytes in the ROM, but extracting it and compressing it with this, it comes out to around 910.