What's new

Rice's Plugin Source <discussion>

Well, i'm not saying you should make every texture huge (say 1024x1024), just one or two where needed, shouldn't be that bad right?
Besides, those really really small textures like say 16x16, with 4x, 64x64 is many times not enough, i don't remember what game i was messing around with i realized that but i found it very annoying.
 
Last edited:

Cyberman

Moderator
Moderator
Well, i'm not saying you should make every texture huge (say 1024x1024), just one or two where needed, shouldn't be that bad right?
Besides, those really really small textures like say 16x16, with 4x, 64x64 is many times not enough, i don't remember what game i was messing around with i realized that but i found it very annoying.
This first depends on texture management of the game and the card. I can't really see anything useful beyond 4x4 even then the poly count on the models is so low no amount of texturing will improve things much more ;)
More seriously, judicious use of large textures is fine, the limit is not 1024x1024. I think you should take a dose of reality here. That is a MONSTEROUS texture. Most shadow buffers are 1024x1024 MAX. On NWN2 they use a 2048x2048 (simple math makes this 16M) but they turn that off if you have a 128M card. You shouldn't need anything beyond 512x512 on a retextured tile. Even on a grassy plain that looked pixelized at 64x64 you still are running at 8x the original texture and likely pushing the pixel to pixel ratio from texture to display. (IE you might have to zoom up on it to take advantage of the higher resolution).

mudlord It's possible to use your own bilinear filter to stretch the texture to an odd size (or LOAD the texture as a odd size). The biggest issue you will have is spanning accross the multiple textures you are using. It's a LOT of work. However it wold allow 3x 5x 6x and 7x textures. It's something to toy with latter, because implementation now likely would break things. (maybe a branch to play with on it).
 

squall_leonhart

The Great Gunblade Wielder
This first depends on texture management of the game and the card. I can't really see anything useful beyond 4x4 even then the poly count on the models is so low no amount of texturing will improve things much more ;)
More seriously, judicious use of large textures is fine, the limit is not 1024x1024. I think you should take a dose of reality here. That is a MONSTEROUS texture. Most shadow buffers are 1024x1024 MAX. On NWN2 they use a 2048x2048 (simple math makes this 16M) but they turn that off if you have a 128M card. You shouldn't need anything beyond 512x512 on a retextured tile. Even on a grassy plain that looked pixelized at 64x64 you still are running at 8x the original texture and likely pushing the pixel to pixel ratio from texture to display. (IE you might have to zoom up on it to take advantage of the higher resolution).

mudlord It's possible to use your own bilinear filter to stretch the texture to an odd size (or LOAD the texture as a odd size). The biggest issue you will have is spanning accross the multiple textures you are using. It's a LOT of work. However it wold allow 3x 5x 6x and 7x textures. It's something to toy with latter, because implementation now likely would break things. (maybe a branch to play with on it).


actually, its not turned off, depending on the framebuffer, the game has a list of cards, that it enabled/disables options for.

the geforce FX and radeon 9500+ both are capable of rendering up to 4096x4096 texture sizes

most shadows these days. are not stencil, so they do not have texture limits, they are rendered via the shader pipeline and applied directly onto textures already in the core.

...and instance of example, is the fact that 4x4 Supersampling can only work at 1024x768 and lower, as it renders the texture at a higher resolution (4096x3072) and scales it back (which is why 4x4 is such a performance killer)

this is why i believe Rice's plugins can only scale the texture up to 4x
 
Why deny the people wich has the VRAM capabilities to use it because people with less VRAM can't handle it?
I honestly don't get it.
Sure it might be a very few huge textures but if you have say 512 MB VRAM it sould take quite a few of those huge ones to get it to whine, sure not too many have that much and neither do i but i think it's more or less safe to say that the VRAM standard today is 128 MB wich would probably be able to handle more than 4x where needed in most games.

I take it depends alot on the games too, there's a huge difference in the amount of textues in say Mario Kart 64 compared to Conker BFD so yeah, i've said it before, it's up to the texturers to not overdo it.
I guess that would add extra work as to optimizing it and scaling down and up testures on small and big areas, a small sacrifice for more freedom and better results if you ask me.

That being said i'm no graphics card wizzie, i just know what i missed from the first day i tried it out.

And i'm NOT saying we should make every texture in every pack 2048x2048, i just say i'd like the possibilty if i was one of the few with 512 MB VRAM, i myself would never go above 1024x1024 just to make it more compatible and provide a lower res pack scaling down the biggest textures for those who would be having trouble handling it.

You could perhaps add the ability to choose what vram you have and allow the texturer to set the minimum requirement in an config file or something.

I'm sure this came out messy but i'm tired so i'll just leave it as it is.
 

Cyberman

Moderator
Moderator
Why deny the people wich has the VRAM capabilities to use it because people with less VRAM can't handle it?
I honestly don't get it.
Sure it might be a very few huge textures but if you have say 512 MB VRAM it sould take quite a few of those huge ones to get it to whine, sure not too many have that much and neither do i but i think it's more or less safe to say that the VRAM standard today is 128 MB wich would probably be able to handle more than 4x where needed in most games.

I take it depends alot on the games too, there's a huge difference in the amount of textues in say Mario Kart 64 compared to Conker BFD so yeah, i've said it before, it's up to the texturers to not overdo it.
I guess that would add extra work as to optimizing it and scaling down and up testures on small and big areas, a small sacrifice for more freedom and better results if you ask me.

That being said i'm no graphics card wizzie, i just know what i missed from the first day i tried it out.

And i'm NOT saying we should make every texture in every pack 2048x2048, i just say i'd like the possibilty if i was one of the few with 512 MB VRAM, i myself would never go above 1024x1024 just to make it more compatible and provide a lower res pack scaling down the biggest textures for those who would be having trouble handling it.

You could perhaps add the ability to choose what vram you have and allow the texturer to set the minimum requirement in an config file or something.

I'm sure this came out messy but i'm tired so i'll just leave it as it is.
That suggestion was made in this post as for choosing I suppose one could use atomagic methods to determine it and if you can't determine it allow someone to set it. There are ways of doing that. Since most windows machines have direct X on them I suppose using DX to find the VRAM amount is one way (See this POST by mudlord). I'm not thrilled by the thought but technically if you have a lot of VRAM to begin with this is a good indicator of memory for textures.

I think you should reconsider your idea that "HEY I have RAM I should have big textures". Unless you plan on doing all those textures yourself. Doing hirez textures is a heck of a lot of work. It's easy to make a hirez texture but it's not easy to make a good one (IE better than the original with a bilinear or bicubic filter on it).
As I mentioned after a certain amount, you won't be able to even enjoy the higher resolution textures. So you can have 256 1 M size textures, will this actually make it look better at 1280x1024 or 1600x1200? Really you are reaching beyond the models too look good. As a marketing expert once said "well you can only improve the looks of a VW so much before it looks absurd".

squall_leonhart
Mmmm read here where Rice discusses in great detail about big textures (I suggest you BOTH read it please).

The problem is you are reaching the limits of the game itself not just the textures.

Cyb
 
Last edited:
Let's get this straight, i'm taking about the really stretched out textures when i talk 8x+, those who looks blurry no matter how much you work with it (a situation mentioned in that thread), not the ones already looking good at 4x the size, so you won't be needing to zoom out alot to be able to enjoy it as it will have the same sharpness as the surroinding textures wich are at 4x, and as i've stated before that's just very few textures if any for each game but it will make a huge difference.
And yes, i'm pretty confident in my PS skills to be able to make really high res textures, the problem would be making a huge amount of them, especially without a proper digital camera, i'd have to rely on free textures on internet, but that's not what i'm talking about either, just a few super high res a game and the rest just normal high res or maybe even standard res, kinda like this guy demonstrates.

Also, what i meant with choosing VRAM was to pick whatever ram you have from a list with less than 32 VRAM to 1GB VRAM or more, compare your choice with the config file provided in the texture pack and then have it warn if it's too low or whatever.
But yeah, nothing as fancy as your suggestion no :p
 
Last edited:

Cyberman

Moderator
Moderator
Let's get this straight, i'm taking about the really stretched out textures when i talk 8x+, those who looks blurry no matter how much you work with it (a situation mentioned in that thread), not the ones already looking good at 4x the size, so you won't be needing to zoom out alot to be able to enjoy it as it will have the same sharpness as the surroinding textures wich are at 4x, and as i've stated before that's just very few textures if any for each game but it will make a huge difference.
And yes, i'm pretty confident in my PS skills to be able to make really high res textures, the problem would be making a huge amount of them, especially without a proper digital camera, i'd have to rely on free textures on internet, but that's not what i'm talking about either, just a few super high res a game and the rest just normal high res or maybe even standard res, kinda like this guy demonstrates.

Also, what i meant with choosing VRAM was to pick whatever ram you have from a list with less than 32 VRAM to 1GB VRAM or more, compare your choice with the config file provided in the texture pack and then have it warn if it's too low or whatever.
But yeah, nothing as fancy as your suggestion no :p
Hehehe at least you read it! I thought I was just talking to walls!

I'm uncertain what "PS skills" are but I'm sure you can make a texture :D

As I suggested to mudlord keeping tabs on textures in RAM and roughly basing what to do with a hirez texture should be adequate to allow big textures. If the plugin doesn't think it has enough memory for an HR texture downsampling it works fine. From how you were going on it sounded like you wanted all huge textures (which as you agree is absurd). As long as the plugin can get a clue about the VRAM size on the card then it has a clue how much memory it can use for textures. This is what GPU plugins for the playstation do (although none have hirez texture abilities). This requires little if any configuration (apart from maybe having to tell the plugin how much VRAM the card has) either

Cyb.
 

Doomulation

?????????????????????????
The way I see it is this... why put a limit on what the creators can do? If they want to do useless 4096x4096 textures, then why not let them? Who are we to say it's right or wrong?
If the card can't handle it, then you already mentioned downscaling them might help.
Not saying it's necessary or that they should be, but what if an author, for whatever reason, wants them? as long as it doesn't put sticks in the wheel, why not allow it?
 

Sirmatto

Member
I don't know the possibility of this, but I randomly thought of it the other day. With Jabo's new plugin coming out, supporting a new way of using hi-res textures, there are a lot of texture-makers holding out until the new plugin is released, which makes me think that it is actually a better way of handling textures. This sort of brings me to my point of: why don't you (Mudlord), maybe try to contact Jabo, and see if he's willing to work with you and unify, at least, the code base for textures. This way, there won't be two radically different sets of textures for the same game floating around, and it makes it so that when someone retextures a game, they can be confident that everyone can use them, no matter their plugin, emulator or even operating system choice.

This is somewhat like zilmar's (if I remember correctly) plugin specs a long time ago, which made plugins interchangeable, rather than having a bunch of incompatible plugins floating around.
 

Cyberman

Moderator
Moderator
That sounds like a good plan. I think the use of a key file (what textures are where they are located in the pack etc) and unified ID for a unique texture should do it. The key files could reside in a directory called hireztextures and the location of the data files would be relative to that path location (IE the correct ROM name) and the path's specified in the key file. The key file could be loaded up when the ROM is started and the GPU is initialized. At that point finding the HR textures is up to the GPU itself. Perhaps a 'Format' attribute as well to specify what the file type is might help as well.
How the key file is generated is a completely different matter. It just needs to do something like
Code:
<texturepack>
 <name>Djippi's Toilet Paper Link</name>
 <author>Djippi</author>
 <url></url>
 <texture>
  <name>Young Links Face</name>
  <ID>AC432567EE45DC12A0</ID>
  <Path>/Link/Young/Face.png</Path>
  <Type>PNG</Type>
 </texture>
 ...
</texturepack>
And the header needs to conform to XML standard. Paths will be an issue however since there is an incompatibility of Asian symbols and european/US symbols (it's not a big deal between europe and the US). So some sort of specific character encoding should be specified to keep the TP's universal.
Anyhow at startup I guess the plugin only needs to read the key file that represents the name of the game right? (IE "DUKE NUKEM ZERO HOUR.key"). This is a simple data base and it might work well for people trying to orginize there Texture Packs with a tool.

This does not include the fact some image textures are split between multiple textures. Adding multiple ID's too a texture could be an option by using something like <subtexture></subtexture> to wrap all the elements of the whole image into an array.

The two things that would need standardized then is texture Identification method and key file format/location.

Cyb
 
Last edited:

Doomulation

?????????????????????????
Unicode? It was built to work with all languages, after all.
It's quite possible to save files in Unicode format (Notepad in XP allows it, for example).
 

Cyberman

Moderator
Moderator
Unicode? It was built to work with all languages, after all.
It's quite possible to save files in Unicode format (Notepad in XP allows it, for example).
Errr Ummmm... Ok.. I think you kind of missed something. :D
XML header selects the document encoding format. All XML documents are based on a unicode standard. Unicode covers a large range of encoding formats. UTF-8 with western encoding is just one example. What was concerning me is someone using UTF-8 with asian symbols (legal to do with the mod symbols), and someone trying to extract a pack with said path encodings. This would be a disaster if for example someone tried that in the US because in the majority of cases said paths wouldn't be legal. I had this problem when trying to install a program localized for Japan once. It's not something easy to fix. I think specifying the encoding very specifically is necessary to prevent incompatibilities. This includes paths.
I don't want to get to the point of having to specify a TP format for distribution really. I suppose though it might become necessary. The Key File specifies where to find things from the HR directory and the Game ID. There is also the problem of ROMS being mangled (there names CRC's because of a bad dump etc). Even if you use a ROM CRC, this method of identification can vary. That is another issue as well. I don't know how to fix that kind of problem to be honest.
The only thing I can think of is having a way to see if the TP will work with a specific ROM ID. It might require creating a list of game keys it works with. IE Zelda 1.0 1.1 1.2 with the key information on them. Making it more specific is less flexible but things can degenerate quickly.

The key file's purpose is not obvious, but it is something to be generated by the plugin FOR a tool. So for example the tool loads up the key file and finds the paths specified (for the original textures) it then allows one to look for specific textures and organize things. The key file then can be moved to the HR directory after certain images are replaced the old paths can be ignored (because the key file is processed when the game is started not while the game is running and validating the existance of the texture path is part of that).

I've been working on making a tool to make texture matching with the games easier. However without a KEY file from the plugin this is quite difficult to do without it. The Key file can or should contain more information to help organize the data I guess is what I'm contemplating. It's not just a handy reference for the plugin to load textures.

Cyb
 

mudlord

Banned
I don't want to get to the point of having to specify a TP format for distribution really. I suppose though it might become necessary. The Key File specifies where to find things from the HR directory and the Game ID. There is also the problem of ROMS being mangled (there names CRC's because of a bad dump etc). Even if you use a ROM CRC, this method of identification can vary. That is another issue as well. I don't know how to fix that kind of problem to be honest.

The only thing I can think of is having a way to see if the TP will work with a specific ROM ID. It might require creating a list of game keys it works with. IE Zelda 1.0 1.1 1.2 with the key information on them. Making it more specific is less flexible but things can degenerate quickly.

From the ROM RDB perhaps?

Code:
<supported>
    <rdb>C3B6DE9D-65D2DE76-C:50</rdb>
    <rdb>2577C7D4-D18FAAAE-C:50</rdb>
    <rdb>6BFF4758-E5FF5D5E-C:4A</rdb>
    <rdb>C9C3A987-5810344C-C:4A</rdb>
    <rdb>3E5055B6-2E92DA52-C:45</rdb>
  </supported>

EDIT: I also made a small list of what has to be change for compatibility with Jabo's texture packs:

* Textures have to go in textures-load in emu directory
* Dumped textures have to go in textures-dump folder in emu directory
* Texture CRCs have to be identical
* Identifier methods have to be honoured
* Tex format guidelines have to be honoured
(such as all JPG, all PNG, NO BMP)
* The video plugin has to hook a menu in emu ROM browser for texture ZIP pack selection
* Video plugin MUST read from ZIPs if packs are bundled in ZIPs
* The plugin has to load "pack.xml" if bundled in ZIP
(otherwise, can load from textures-load arbitrary, but cannot be turned off)
 
Last edited:
OP
Enzo Dragon

Enzo Dragon

STFU, NAVI
I'm still alive and still check this thread often, first off.

Second, I fixed the issue I was having with selecting my render engine and all of that, but I still haven't been able to get the beta aspect ratio bit working. However, this might have to do with the render engine; I read somewhere on here that PJ64 wasn't the best with openGL?

Either way, I'm going to try a different emu later to see if that fixes my problems - otherwise, it may be that I screwed up my OGL libraries when I tried to "fix" them.
 
Last edited:

mudlord

Banned
I'm still alive and still check this thread often, first off.

Second, I fixed the issue I was having with selecting my render engine and all of that, but I still haven't been able to get the beta aspect ratio bit working. However, this might have to do with the render engine; I read somewhere on here that PJ64 wasn't the best with openGL?

Either way, I'm going to try a different emu later to see if that fixes my problems - otherwise, it may be that I screwed up my OGL libraries when I tried to "fix" them.

Thanks for dropping on this, I'm still working on this, ya know, its just for me college is really eating up my time atm (I got several assesments, including a Python programming proj).

I'm glad you fixed that renderer prob, but yeah, the aspect ratio control is as you say, beta...heheheh, I'm gonna see about doing a proper fix for it that still does it right..

And, I somewhat think PJ64 has a issue with OGL too..I dunno, its funny, I don't know yet if its to do with threading and such, or whether just Project64 1.6/1.7 plays not nice like other emus, with regards to OpenGL cleaning up on exit...
 
Last edited:

mudlord

Banned
Also, I had a look into the whole hi-res code. It seems Rice just set a 4x limit on the scaling factor of textures. I guess we could tweak that, to gain support for higher textures (we could through in compression on BMP CIs, since PNG supports colour indexing). But then we run into the issues as describe earlier. First things first though, getting BMGLib to work right on all games, and not just two...Which is plain weird, as I did nothing to mess with BMGLib when compiling a dynamic library like the official release..and upon testing that, the same thing happens with games other than SM64...:unsure:

EDIT: Okay, compatibility with Jabo's texture packs is not going to happen. There is really too much different variables (different things) if the packs are going to be working (and working right) between each plugin.
 
Last edited:

Cyberman

Moderator
Moderator
Texture Packs

Hmmm the TP format differences are that large?

It would be nice if one could converse with Jabo about it, as this would be a serious problem for repairing Rice's plugin.

Not a good thing really. Did Jabo give you his specs for TP's? If so where are the specs? The solution may be adding a 'import' tool to convert them. As you know no one will be developing new TP's for Rice's plugin due to Jabo's new plugin support for HR textures. This essentially means 0 support for the plugin by people making TP's as a consequence.

In your prior post the RDB data looks to be the best idea or method. Some textures might have different MD5 or CRC data. That could make things highly bothersome in make a single pack for all versions. There is the possibility of having multiple tables to hash the CRC's. It's highly unlikely 2 32 bit CRC's of seperate images will match. It's a 1 in 4 billion chance I believe. So a list of CRC's (seperate) with a reference to the proper texture file would work. This is becoming more a database application isn't it?


As a side note, is it possible to allow a program to link to the DLL via a socket to read data it's generating etc. ? Sort of like a Data pipe for debugging or in this case texture dumping information. The application might be able to handle creating much of the TP structure information while the game is 'played' through. IE things like load screens etc. Just an idea to have a seperate program from the DLL to handle that stuff. It would be interesting to know what kind of data one could get and how to arrange it etc. Might be useful for planing TP's as the game dumps data.

I digress though.

Cyb
 

mudlord

Banned
OK, I'm back..

Well, there are a huge amount of changes between Jabo's and Rice's plugin. With consultation with Jabo and Smiff on the PJ64 beta forums, its seems that technically it is unable to happen, due to how the two different plugins handle emulation....

The problem is, Jabo's plugin uses only MD5 checksums for texture identification. It uses no other identification methods, but its interesting how only PNG files are used for textures. Secondly, because of this sole use of a MD5 for checking, core differences between how the plugins handle texturing makes support not possible, or being optimistic, extremely difficult, because of how the two different plugins have thier very own methods of doing emulation. Also, Jabo explained how he doesn't want a "standard" of hi-res textures made by his plugin, so to comply with Jabo's ideals, the plugin won't switch h-res texturing methods. Besides, Jabo raised very valid points on how they are recruiting new talent to make new texture packs...:)..so that texture packs can be made natively for it. And like you said, no one would be making texture packs for this when the public release of 1.7 rocks around, mainly due to emulation improvements (which I admit, over the weeks I'm betatesting 1.7, are extremely nice).

In your prior post the RDB data looks to be the best idea or method. Some textures might have different MD5 or CRC data. That could make things highly bothersome in make a single pack for all versions. There is the possibility of having multiple tables to hash the CRC's. It's highly unlikely 2 32 bit CRC's of seperate images will match. It's a 1 in 4 billion chance I believe. So a list of CRC's (seperate) with a reference to the proper texture file would work. This is becoming more a database application isn't it?

Yeah, indeed. Its pretty unlikely for that method not to work :). And yep, its pretty much going that way ^^

As a side note, is it possible to allow a program to link to the DLL via a socket to read data it's generating etc. ? Sort of like a Data pipe for debugging or in this case texture dumping information. The application might be able to handle creating much of the TP structure information while the game is 'played' through. IE things like load screens etc. Just an idea to have a seperate program from the DLL to handle that stuff. It would be interesting to know what kind of data one could get and how to arrange it etc. Might be useful for planing TP's as the game dumps data.

That actually is a quite clever idea . I can see its applications, and heck, whatever helps me debug this is a good idea.
 

Cyberman

Moderator
Moderator
OK, I'm back..

Well, there are a huge amount of changes between Jabo's and Rice's plugin. With consultation with Jabo and Smiff on the PJ64 beta forums, its seems that technically it is unable to happen, due to how the two different plugins handle emulation....
You mean a large number of differences?
Hmmm that's too bad, however I don't think it is as bad as all that. However the problem might be in where and how the textures are being loaded. That is extremely difficult to fix unfortunately.
The problem is, Jabo's plugin uses only MD5 checksums for texture identification. It uses no other identification methods, but its interesting how only PNG files are used for textures. Secondly, because of this sole use of a MD5 for checking, core differences between how the plugins handle texturing makes support not possible, or being optimistic, extremely difficult, because of how the two different plugins have thier very own methods of doing emulation. Also, Jabo explained how he doesn't want a "standard" of hi-res textures made by his plugin, so to comply with Jabo's ideals, the plugin won't switch h-res texturing methods. Besides, Jabo raised very valid points on how they are recruiting new talent to make new texture packs...:)..so that texture packs can be made natively for it. And like you said, no one would be making texture packs for this when the public release of 1.7 rocks around, mainly due to emulation improvements (which I admit, over the weeks I'm betatesting 1.7, are extremely nice).
Are there patent issues on using MD5? It's actually superior to CRC in terms of acuracy. MD5 is just plane amazingly acurate, it is however extremely intensive on the CPU. If Jabo can provide the format information, I don't think conversion is impossible. Rice's method of storing the data is good, however there has always been something bugging me, portability and how to do it.

There is no easy solution, the portability issue brings up the morass of the library DLL needed for texture loading and dumping as well. Using raws for both is questionable but doable.

Is a binary format TP a good or bad idea is the next question. I don't like having to compile a whole bunch of data (it's slow). However it does keep the data structuring simpler for the plugin. It wouldn't have to look through a pile of files. Hmmmm. More to ponder. Porting the data might be an issue of prior packs, although parsing through a huge list of files might be adequate for a quicky porting util I suppose. Argh this makes my head spin LOL.

So I've heard it adds lots of things. The adjustable aspect ratio etc.

Yeah, indeed. Its pretty unlikely for that method not to work :). And yep, its pretty much going that way ^^
Ehh though it's too bad that Jabo's checks the textures likely in a different point so that makes MD5 data and CRC data compatibility improbable at best.

That actually is a quite clever idea . I can see its applications, and heck, whatever helps me debug this is a good idea.

I was thinking that it would offload creating the XML information reguarding the files and allow in real time a way of seeing what images are being dumped in game. IE a tree of the data and file list to allow you to pause the game and examine the textures dumped. That might help people with making TP's also.


Cyb
 
Last edited:

Top