Game Boy

[Rüdiger]

Pokemon Blue - May be related to STAT interrupts. I would imagine that one type of interrupt (likely H-Blank) is being triggered so the game can safely modify SCX for the scanline for the wave-like effects. Seems like all the GB Pokemon games (Red through Crystal) don't make the very first scanline move when attacks make those wave-like effects (LY=LYC test maybe? Perhaps the effect just couldn't be done when testing for LY=0). If it loops forever rather than crashing, it's usually a sign that you're not feeding it input it expects (an interrupt, a certain value in the MMIO registers, stuff like that). The best thing to do is look at the GB assembly in a debugger and compare it in an emulator. Time consuming, but effective, since eventually your emulator will diverge from a correct, working emulator and you can then pinpoint where exactly.

That was one of the reasons I wrote the profiler, just didn't have the time and motivation to dig through the output.
I also noticed that it does not move the second scanline, which looks weird and should probably not happen.
Edit: The issue was easy to find, he checks for LY==143, but LY resets after reaching 142.
A closer look at my code revealed that I triggered the Vblank interrupt at the end of scanline 143 instead of 144.
This fixed the loop but now the topmost 3 scanlines don't move...

This apparently fixed Pokemon Crystal too.

As for double-speed, I kinda struggled to implement it correctly too, in fact, I only knew my implementation was broken because the Oracle games were messed up. I solved it by simply halving the CPU cycles the LCD acknowledged after each instruction (my emulated APU is not cycle-based, not yet), but by sending the original amount of CPU cycles to the timers. The CPU runs at 2x its normal operating speed (meaning all timers run twice as fast too) but LCD operations, DMA transfers run at the same speed. The setup I just described follows that, but it took me a while to figure it out.

Three new lines and it seems to work fine :satisfied .
Edit: Some palette issues and the map corruption in Zelda DX is now fixed,
but remains in the DMG Zelda.

Other stuff - Sprite palettes in DMG games are off. You may know how to fix it already, but just make sure you're correctly reading OBP0 and OBP1. Also, seems like you still have BG/Sprite priority issues (Suicune in the intro of Crystal should be behind the grass). For reference, there are only 4 cases where a sprite's pixel will prevail over the BG's pixel (GBC only):

The color issues probably happened when I swapped color channels around to fix GBC games, but the palette lookups seem fine.
I check the OBJ-to-BG priority but not (yet) the BG-to-OBJ, so that could explain it.

 

[Shonumi]

Only two tests are finished at the moment. Apparently I didn't have as much free time as I imagined, thanks to my urge to play games rather than program stuff

The first test, LYC.gb works using the STAT interrupt, specifically the LY=LYC test. It draws exactly 16 lines of the background before disabling the background altogether. This test was based on Super Mario Land which gave me some headaches. A real Game Boy always performs a comparison of LY and LYC after LY is modified in any way. You have to 1st render the current scanline, then increment LY and do the comparison. I noticed some emulators were rendering, comparing, then incrementing LY. This isn't documented clearly as far as I know, but from a hardware perspective, the first (and correct method) is the most logical. This test basically makes sure the order is correct. Output should be 16 lines of alternating color (last color should be the lighter one of the two) and after that the rest of the screen should be blank (white basically in a pure B/W monochrome palette).

The second test, sprite_suite.gb, focuses mainly on sprites. It's still a WIP with only 1 test programmed (the 10 sprites per line limitation), but more are expected. This ROM does rely heavily on STAT interrupts (OAM interrupts) as an early warning for the program to abort writing to VRAM before it becomes inaccessible and continue on the next VBlank. Basically the 11th sprite just scrolls vertically over an over again, and every time the Game Boy has to render a line with more than 10 sprites, the 11th sprite's lines disappear.

Both have been verified on real hardware (CGB-001 if you're interested, but I have a Game Boy Pocket, Game Boy Advance, and Game Boy Player that can be tested as well).

 

[Rüdiger]

I found a rather stupid bug thanks to AddressSanitizer. I did not limit VBK to 1 bit which caused Oracle to write all over the memory.
This fixed the ingame graphics but the opening and some menus are still garbled.

The wrong colors in DMG mode were easy to fix but it took a lot of time to find the actual issue.
As it turned out, the two bits of all colors were swapped, completely messing up the otherwise correct palette lookups... :whistling

While the overall compatibility with DMG games is pretty good, I still found some additional games that have more or less serious issues:
Prehistorik Man: I know that this game does some mid-scanline changes that I can't handle at the moment, but besides that the game shows flickering and occasionally broken sprites.
Mystic Quest: Flickering between sequences.
Parodius: Does a JP HL straight into the character RAM and hits an invalid instruction.

Shonumi: Your test roms look good so far. Disabling the background on my emulator currently fills the scanline with color 0 from BGP instead of white so it is not really visible, but it turns off after 16 lines.
Did you write them in C or assembly and are there any good toolchains around? So far I only looked at gbdk, but the newest release is from 2002.

 

[Shonumi]

Congrats on your progress. I myself have been extraordinarily lazy as of late, and haven't put the finishing touchs on GBC support. It's ready to merge, almost.

I heard about Prehistorik Man being a pain to emulate. Perhaps the VBA-M source code has some comments in the code. It may give you a clue. I haven't tested that game. The scanline pixel data itself shouldn't be able to change after a certain point in the LCD's operation (i.e. VRAM access from the CPU is disabled), but I don't know how changing things like SCX or SCY affect rendering mid-scanline. Coming up with a test for that might be tricky.

As for the tests, I'll see if I can add more this weekend. It's not coded in anything but the GB's Z80 assembly. I opened a blank 32 KB file in a hex editor, then started plugging away hexadecimal values. I've been doing it this way for a while, and I've just gotten used to it. It's not very readable, but it works

 

[Rüdiger]

I rewrote my LCD state machine. That fixed the Oracle intro, the Link's Awakening intro, Parodius, sprites in Shantae...
and even Prehistorik Man looks a tad better. It did however not fix the map corruption in the DMG Zelda.

Next up will probably be the HDMA code. I have an implementation that does all transfers instantly and one that
does GP-DMA and HDMA seperately with 0x10 bytes per Hblank.
Needless to say, the first variant performs better for almost all games with Pokemon Crystal as notable exception...

As for the tests, I'll see if I can add more this weekend. It's not coded in anything but the GB's Z80 assembly. I opened a blank 32 KB file in a hex editor, then started plugging away hexadecimal values. I've been doing it this way for a while, and I've just gotten used to it. It's not very readable, but it works

Maybe you should try butterflies next?

 

[Shonumi]

Being lazy again (playing Bravely Default), so I haven't updated the tests. I know coding a GB ROM in a hex editor sounds a little extreme or "hard core", but it's really nothing special. Since the opcodes are 8-bit (the most common ones anyway), it doesn't take much to get used to it. I just don't want to take the time to get involved with an assembler when the actual code for a few test ROMs takes up only a few kilobytes of actual written code (as you can see, most of the 32KB are blank ). Coding for the GB's Z80 variant in a hex editor is actually quite enjoyable and fast once you get used to it. Coding 32-bit ARM instructions in a hex editor, however, gets really troublesome by comparison (yes I've actually done that, I'm exploring GBA emulation by studying the ARM7 architecture right now).

Made some progress reducing CPU usage in GBC mode, think I can begin merging it into master, then start hunting bugs and implementing HD custom tiles. Once that feature is finished, I think I'll work on emulating the Link Cable over a network. I know accuracy and optimization are important, but so are cool features. And I really want to trade Pokemon with myself...

Regarding Prehistorik Man, it looks like the actual data in VRAM doesn't change as I suspected, rather the game is changing the LCDC midscanline, disabling/enabling the BG (and OBJs I think). The relevant code seems to be coming from a STAT interrupt (LY=LYC). It seems to increment LYC constantly, and depending on that line, it disable/enable the BG for certain amounts of time. This is speculation since I have not thoroughly looked over the code, but it may be changing the BG palette so that all colors are black (or sometimes the second darkest shade of gray).

I figure this is highly relevant and useful for getting this game to work: https://bitbucket.org/arkhaix/emunisce/issue/52/mid-scanline-rendering-support Fixing this game in particular is below on my TODO list; did I mention I really want to trade Pokemon?

 

[Rüdiger]

Bravely Default? Looks I have to upgrade to a 3DS sooner or later...

I added support for the DMG, SGB and GBC bios. Not very useful except for the additional color palettes.
Found two more games that do not work properly, Duck Tales shows no background and Bubble Bobble hits an illegal instruction in the first level.

HD custom tiles: A friend suggested something similar, are there any tilesets for GB games around?
The link cable was not on my todo list, guess I never thought of trading Pokemon with myself. :down:
I wonder how important the timing and latencies are. Is there a (documented) protocol for linking over network?

I also dug up my GBC but the screen is flickering. Disassembling and reassembling didn't help either.

 

[Shonumi]

I haven't tested the GBC bios a great deal. It runs properly for the most part, but then it starts executing NOPs in uninitialized memory. I guess I just need to take the time to properly look at what it's trying to do. Personally, I have no interest whatsoever with the SGB, so I haven't even touched it. If I ever do get around to it, it's on the bottom of my long list.

I don't think there are any GB tilesets sitting around, at least I don't know of any. Any work in this field involves dumping the tiles myself and editing them. The infrastructure for HD tiles is setup already (it can load 1:1 custom tiles just fine), but it needs to take the scaling of the HD tileset into account when drawing.

I have a lot of old hardware sitting around my room myself. The old Game Boy Pocket is somewhere in the closet, along with the Pocket Printer. GBC's sitting on my desk with a flashcart in it (for development purposes only). MY GBA is sitting not too far off as well, and the Game Boy Player is around the corner in the next room. I'm surrounded with Game Boys I still have the box the GBC came in (I'm a hoarder...)

 

[Rüdiger]

Merged my old SGB code that I put aside some weeks ago and expanded it far enough to load SGB borders. Next up are the palettes.
Edit: Much better.

 

[Rüdiger]

It now supports CHR_TRN, PCT_TRN, PAL_TRN, PAL_SET, ATTR_TRN, ATTR_SET and PAL(01,12,03,23). The code is still a complete mess, but it's enough to get correct colors in most games.

 

[fleroviux]

Had anyone of you the problem that pokemon gold is crashing when warping?
I'm having this issue and I don't know how to fix it..

 

[Shonumi]

Had anyone of you the problem that pokemon gold is crashing when warping?
I'm having this issue and I don't know how to fix it..

Still having this issue? By "warping" do you mean:

1) Using the FLY HM to go to another city?
2) Using a Pokemon that knows Teleport?
3) Stepping on the warp tiles (like those in the Saffron City Gym or the one in the Team Rocket Hideout in Mahogany Town)?

I have no issues doing any of the above. Looks like you'll have to do some debugging to find out what component is the cause of the problem.

Rüdiger - That looks nice, great work. Is the SGB difficult to emulate (in your opinion)? I remember reading that the frequencies of sounds on the SGB is around 2% higher than the GB. I wonder if you could ignore changing anything related to the APU and still be fine.

 

[Rüdiger]

Shonumi - That really depends on how complete and accurate one wants to emulate the SGB. Emulating enough to get borders and palettes is doable and that is what I am aiming for at the moment.

The SGB is kind of weird and documentation is sparse, so my code is not very accurate and some guesswork is required. Communication with the SGB is performed over a serial protocol by writing two bits in the keypad register. Larger transfers happen over VRAM and - in principle - would require to reconstruct the VRAM content from the rendered image. But it seems that all? games use an 1:1 mapping and the content can be copied directly.

A lot of games also use the SGB sound functions but still output normal sound, probably some additional effects but I have not investigated it further and I am not sure it would be worth the trouble.
The SGB runs about 2.4% faster and I ... completely ignore it, no issues so far. But it would be interesting to investigate if some games try to compensate for the higher speed.

Complete SGB emulation would require a SNES emulator, as the games can - theoretically - access the complete hardware. But there are only 2-3 games that make use of it.
For most games it would be the best to load the border and then reboot in CGB mode (and would make the SGB emulation a complete overkill just for some borders... ).
But there are at least a few games that don't support CGB and benefit from the SGB mode.

Development has stopped at the moment, I'm too busy with other stuff.

Flerovium - I have nothing similar in pokemon.
Does it hang (in an endless loop) or crash by executing random code?
In the latter case it could be worthwhile to check the status of interrupts (does it crash inside of an interrupt, is it actually enabled, ...) and writes to the MBC prior to the crash (does it crash in the higher 16K of the ROM, does it crash shortly after switching the bank, ...). But thats really just a guess - could be anything...

 

[fleroviux]

Currently I only know that it resets, when stepping on the warp tiles. By reset I mean that it simply restarts.
The entrypoint (150h) is also the main loop (and vblank interrupt) from my current point of knowledge.
Therefore it's hard to find out, where exactly it's restarting. For my personal luck there exists a disassembly of pokémon crystal
with comments, methods names etc. Maybe I'll have to look for a variable containing a id that tells wich scene to render.

 

[Rüdiger]

Already one moth since the last post...

I dug out my emulator yesterday and fixed some palette issues.
There are still some SGB commands missing, but the graphics related functions used by most games are implemented.
I also added ATTR_BLK for Pokemon, but the colors are still off.

 

[Shonumi]

Nice work Rüdiger. Those screenshots are looking pretty awesome

I've been taking a long break from development myself, just got back into things by fixing a bug I didn't even know existed (took less than 10 minutes to solve, I'm back and on fire! ) Apparently my HDMAs would continue on indefinitely instead of only executing for a set number of scanlines. Oops.

I actually have a long to-do list, just need to get up and start coding things. With the HDMA fixed, I think I'll try to get the GBC BIOS working (low-hanging fruit you know).

 

[Rüdiger]

Shonumi: I still have at least one bug in my HDMA code, but as it only affects Pokemon Crystal, I have not bothered fixing it.
I have an idea what could be the cause, but fixing it would require a rewrite of the gpu/cpu timing code. :doh:

I found a really stupid bug in ATTR_BLK and now Pokemon Blue has (mostly?) the right colors. Gold has still some weird issues. The battle screen looks fine but almost everything else has wrong colors...
As it turns out, fixing the border in Blue has broken the borders of at least two other games. It seems that there is no documentation on some details and that I am stuck with guessing and digging in other emulators.

I want to fix and implement some more SGB commands and then merge everything with the (C)GB stuff.
Link support and maybe my own sound engine are on my to-do list, but otherwise, it is not very long. ( Or maybe I am not very inspired )

 

[Shonumi]

Have you had a look at the official GB programming manual? It has a lengthy section about SGB functions (obviously since game developers needed that info, and only Nintendo could provide it). That's about as complete as you can get for documentation.

About the HDMA, Pokemon Crystal was the first game I actually tried and tested to verify correct HDMA functionality. If you're worried about something like performing the actual transfer in a cycle-accurate manner, don't worry about that. For HDMAs, I just transfer all 16-bits per scanline instantly and I don't get any graphical glitches. As far as I know, most games should not be sensitive to the timing of the HDMA transfer, as long as it happens sooner rather than later. But I can't guess what's wrong on your end without seeing a picture or something

I would make sure that the HDMA addresses are correctly masked. The destination is always in VRAM, and certain bits in the start and destination 16-bit addresses are ignored because of this. So check where you're writing the HDMA to as well. Though if it really is a GPU/CPU timing issue (H-blanks not happening at the correct times) then yeah, you already know what's wrong like you said.

Got the GBC BIOS to work now. The code is a mess and needs to be re-worked before committing it to my master branch. It was fun diving into it though. Now the GBC palettes show up on DMG games.

 

[Rüdiger]

Have you had a look at the official GB programming manual? It has a lengthy section about SGB functions (obviously since game developers needed that info, and only Nintendo could provide it). That's about as complete as you can get for documentation.

Yes, but it is surprisingly unhelpful on some details. For example transparency in the SGB border. My solution which seems to work in all tested games (after 2 other where fixing one broke another...) is to make color 0 of all palettes transparent inside the GB window and white outside (or transparent with a white background).

And my ATTR_BLK seems to be correct after all, some games just have weird colors in SGB mode. (Left is my emulator and right is vbam. The white part of the border is color 0.)

I also added MASK_EN. ATTR_LIN, ATTR_DIV and ATTR_CHR are next and should be done quickly, as they are very similar to ATTR_BLK.
All sound functions and some other stuff is missing, but they are probably not worth the trouble.

 

[venge]

I heard about Prehistorik Man being a pain to emulate. Perhaps the VBA-M source code has some comments in the code. It may give you a clue. I haven't tested that game. The scanline pixel data itself shouldn't be able to change after a certain point in the LCD's operation (i.e. VRAM access from the CPU is disabled), but I don't know how changing things like SCX or SCY affect rendering mid-scanline. Coming up with a test for that might be tricky.

To correctly emulate prehistorik man, you should do exactly what a real game boy does during mode 3 (OAM - VRAM reading). Instead of reading all the pixels with your pointer for(i = 0; i <160; i++) at the END of the mode, you must think of how you will implement a pointer that follows this time given to the LCD to read from the VRAM. A VRAM acccess COULD change a pixel during a scanline but as you mentioned it's not possible. But a pallete change can do that . And if you look carefuly, it is exactly what happens during the prehistorik's scanline. Dynamic pallete changes.

*spoiler* That solution includes the last instruction clocks too